From 342b1f6e98d1ca94a4d909f6f140ea56e62cb0c9 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 04:31:08 +0530 Subject: [PATCH 01/73] fix(playwright): refuse a test run twice, and show keploy's skip reason keploy runs each test once in a run and charges a test's calls to its one window. A Playwright retry (from --retries, the config or test.describe.configure) or a --repeat-each repeat began the test a second time. Such a run now fails before its begin, saying why and how to turn it off, and keploy is told (POST /agent/refusal), so its verdict on the run names the test. The message names the test, not the attempt, so the agent keeps one refusal per test however often Playwright runs it again. keploy's reason for skipping a test was only in the skip annotation, which Playwright's list, line and dot reporters do not print. The test's output now shows it as well. A begin, an end or a refusal answered with a plain 404 (keploy running without KEPLOY_PROVEN=1) said only that the agent answered 404. It now names the switch, as the claim's 404 already did. Signed-off-by: slayerjain --- playwright/README.md | 6 +- playwright/src/agent.ts | 43 +++++++--- playwright/src/index.ts | 48 ++++++++++- playwright/test/fixtures/retry.spec.ts | 11 +++ playwright/test/rows.test.mjs | 114 +++++++++++++++++++++++-- playwright/test/unit.test.mjs | 59 ++++++++++++- 6 files changed, 254 insertions(+), 27 deletions(-) create mode 100644 playwright/test/fixtures/retry.spec.ts diff --git a/playwright/README.md b/playwright/README.md index a2b4861..3e90fa7 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -30,9 +30,9 @@ KEPLOY_PROVEN=1 keploy mock record -c "npx playwright test --workers=1 --retries KEPLOY_PROVEN=1 keploy mock replay -c "npx playwright test --workers=1 --retries=0" ``` -`KEPLOY_PROVEN=1` is keploy's switch for its proven model. keploy speaks this package's wire, scope protocol v1 (the claim, begin, end and given), only under it. Without it, keploy answers the worker's claim with a plain 404, and every test of the worker fails with `keploy: this keploy agent answered 404 to POST /agent/instance, so this worker cannot claim keploy's run: ...`, which names the switch. +`KEPLOY_PROVEN=1` is keploy's switch for its proven model. keploy speaks this package's wire, scope protocol v1 (the claim, begin, end, given and refusal), only under it. Without it, keploy answers the worker's claim with a plain 404, and every test of the worker fails with `keploy: this keploy agent answered 404 to POST /agent/instance, so this worker cannot claim keploy's run: ...`, which names the switch. A begin, an end or a refusal answered with a plain 404 names the switch the same way. -keploy's first release runs one test process at a time. A second worker that is still alive while another runs is refused by keploy, and its tests fail. Turn retries off: keploy sets aside a test it cannot prove, with its reason, and fails a test whose calls changed, and a retry changes neither. +keploy's first release runs one test process at a time. A second worker that is still alive while another runs is refused by keploy, and its tests fail. Turn retries off: keploy sets aside a test it cannot prove, with its reason, and fails a test whose calls changed, and a retry changes neither. keploy runs each test once in a run, so a test Playwright runs again (a retry, from `--retries`, the config or `test.describe.configure`, or a repeat of `--repeat-each`) fails without beginning, saying why, and keploy is told (`POST /agent/refusal`), so its verdict on the run names the test. ## Without keploy @@ -42,7 +42,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. -- **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). +- **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped )`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. - **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see the limits below). diff --git a/playwright/src/agent.ts b/playwright/src/agent.ts index fe2a24e..c523f49 100644 --- a/playwright/src/agent.ts +++ b/playwright/src/agent.ts @@ -1,6 +1,6 @@ // The client of keploy's agent: scope protocol v1, the same wire go-sdk's // keploytest speaks (POST /agent/instance, /agent/scope/begin, -// /agent/scope/end, /agent/scope/given). +// /agent/scope/end, /agent/scope/given, /agent/refusal). // // Every answer of the agent carries the protocol version as "v"; an answer // without it, or with another version, is an error, so this package on an @@ -14,6 +14,7 @@ export const PATH_INSTANCE = '/agent/instance'; export const PATH_BEGIN = '/agent/scope/begin'; export const PATH_END = '/agent/scope/end'; export const PATH_GIVEN = '/agent/scope/given'; +export const PATH_REFUSAL = '/agent/refusal'; export const ACTION_SKIP = 'skip'; export const ACTION_FAIL = 'fail'; @@ -106,25 +107,30 @@ export class AgentClient { return a as Answer; } - /** Claims keploy's run for this test process (a Playwright worker). */ - async claim(key: string, dir: string): Promise { - let a: Answer; + /** + * Sends a call of the scope protocol's first release. keploy speaks it + * only under its proven switch: with the switch off (keploy's default), + * the agent has no route for it and answers the mux's plain 404, as a + * keploy from before the protocol does. So a plain 404 names the switch, + * saying what it stopped (what). + */ + private async onWire(path: string, body: unknown, what: string): Promise { try { - a = await this.post(PATH_INSTANCE, { key, dir }); + return await this.post(path, body); } catch (err) { if (err instanceof KeployError && err.status === 404 && err.code === '') { - // keploy speaks scope protocol v1 only under its proven switch. With - // it off (keploy's default) the agent has no claim route and answers - // the mux's plain 404, as a keploy from before the protocol does - // (its begin and end answer an older, unversioned wire). The claim is - // this package's first call, so it is the one that names the switch. - throw new KeployError(`keploy: this keploy agent answered 404 to POST ${PATH_INSTANCE}, so this worker cannot claim keploy's run: ` + + throw new KeployError(`keploy: this keploy agent answered 404 to POST ${path}, so ${what}: ` + 'keploy speaks @keploy/playwright\'s wire, scope protocol v1, only when it runs with KEPLOY_PROVEN=1 ' + '(KEPLOY_PROVEN=1 keploy mock record|replay -c "npx playwright test --workers=1 --retries=0"); ' + 'a keploy run without KEPLOY_PROVEN=1, or one from before scope protocol v1, answers 404 there', 404); } throw err; } + } + + /** Claims keploy's run for this test process (a Playwright worker). */ + async claim(key: string, dir: string): Promise { + const a = await this.onWire(PATH_INSTANCE, { key, dir }, 'this worker cannot claim keploy\'s run'); if (!a.instance) { throw new KeployError('keploy: the keploy agent answered the claim of this test process without an instance'); } @@ -137,7 +143,7 @@ export class AgentClient { /** Asks the agent to open a test. */ async begin(test: TestRef): Promise { - const a = await this.post(PATH_BEGIN, { ...test, canSkip: true }); + const a = await this.onWire(PATH_BEGIN, { ...test, canSkip: true }, `${test.name} cannot begin`); if (a.status !== 'ok') { throw new KeployError(`keploy: the keploy agent answered the begin of ${test.name} with status ${JSON.stringify(a.status ?? '')}`); } @@ -149,7 +155,7 @@ export class AgentClient { /** Tells the agent the test ended, with its outcome. */ async end(test: TestRef, outcome: Outcome): Promise { - const a = await this.post(PATH_END, { ...test, outcome }); + const a = await this.onWire(PATH_END, { ...test, outcome }, `${test.name} cannot end`); if (a.status !== 'ok') { throw new KeployError(`keploy: the keploy agent answered the end of ${test.name} with status ${JSON.stringify(a.status ?? '')}`); } @@ -159,6 +165,17 @@ export class AgentClient { return a; } + /** + * Tells keploy this package refused something keploy cannot judge (a + * test Playwright retried), so keploy's verdict on the run names it. + */ + async refuse(code: string, message: string, remedy: string): Promise { + const a = await this.onWire(PATH_REFUSAL, { code, message, remedy }, 'keploy cannot be told of the refusal'); + if (a.status !== 'ok') { + throw new KeployError(`keploy: the keploy agent answered the refusal ${code} with status ${JSON.stringify(a.status ?? '')}`); + } + } + /** Gives the open test a value it made itself. Returns the agent's variable name. */ async given(name: string, value: string, source: string): Promise { let a: Answer; diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 29c0c14..60f0125 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -21,7 +21,11 @@ // project folder (KEPLOY_MOCK_PROJECT), which its test files are named // from; the current folder only when keploy names none; // - each test begins (POST /agent/scope/begin) before its beforeEach hooks; -// a test keploy set aside is skipped with keploy's reason; +// a test keploy set aside is skipped with keploy's reason, which the +// test's output shows; +// - a test Playwright runs again in the run (a retry, a --repeat-each +// repeat) fails without beginning, and keploy is told (POST +// /agent/refusal), since keploy runs each test once in a run; // - before the test's browser context closes, the test waits until none of // its pages' calls is in flight (settle), so its calls end inside it; // - the test ends (POST /agent/scope/end) after its last teardown, with @@ -104,10 +108,28 @@ export const test = base.extend({ return; } const ref = testRef(testInfo); + const again = runAgain(ref, testInfo); + if (again !== null) { + // keploy runs each test once in a run: a second run of it is failed + // before its begin, and keploy is told, so its verdict names the test. + // The message names no attempt, so the agent keeps one refusal for a + // test however often it is run again (each retry is a new worker). + let told = ''; + try { + await c.refuse(REFUSE_RUN_AGAIN, again.message, again.remedy); + } catch (err) { + told = `; keploy could not be told: ${err instanceof Error ? err.message : String(err)}`; + } + throw new KeployError(`keploy: ${again.message} (${again.remedy})${told}`); + } const b = await c.begin(ref); if (b.action === ACTION_SKIP) { // The framework's own skip, with keploy's reason; no end follows. - testInfo.skip(true, keployText(b.reason, 'keploy: set aside (the agent gave no reason)')); + // Playwright's list, line and dot reporters print a test's output but + // not its skip's reason, so the reason is printed as well. + const why = keployText(b.reason, 'keploy: set aside (the agent gave no reason)'); + process.stderr.write(`${why} (keploy skipped ${ref.name})\n`); + testInfo.skip(true, why); return; } open = ref; @@ -160,6 +182,28 @@ export function keployText(reason: string | undefined, fallback: string): string return reason.startsWith('keploy: ') ? reason : `keploy: ${reason}`; } +/** The code of the refusal of a test Playwright runs again in one run. */ +export const REFUSE_RUN_AGAIN = 'test_run_again'; + +/** + * Why this run of the test is refused, or null for its first run: keploy + * charges a test's calls to its one window in a run (scope protocol v1, + * section 4), so a Playwright retry, or a repeat of --repeat-each, is not + * begun. + */ +export function runAgain(ref: TestRef, ti: Pick): { message: string; remedy: string } | null { + const id = `${ref.project ? `${ref.project}::` : ''}${ref.file}::${ref.name}`; + if (ti.retry > 0) { + return { message: `Playwright retried ${id}, and keploy runs each test once in a run`, + remedy: 'run the suite with --retries=0, and with no retries in its config or its test.describe.configure' }; + } + if (ti.repeatEachIndex > 0) { + return { message: `Playwright ran ${id} again for --repeat-each, and keploy runs each test once in a run`, + remedy: 'run the suite without --repeat-each, and with no repeatEach in its config' }; + } + return null; +} + /** The test's own verdict, as keploy's end takes it. */ export function outcomeOf(ti: Pick): Outcome { if (ti.status === 'skipped') return 'skipped'; diff --git a/playwright/test/fixtures/retry.spec.ts b/playwright/test/fixtures/retry.spec.ts new file mode 100644 index 0000000..b684a71 --- /dev/null +++ b/playwright/test/fixtures/retry.spec.ts @@ -0,0 +1,11 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +test.describe('retried', () => { + // A retry of this group's test, whatever the command line says. + test.describe.configure({ retries: 1 }); + test('fails its first attempt', async ({}, testInfo) => { + fs.writeFileSync(`${process.env.MARKER_DIR}/attempt-${testInfo.retry}`, '1'); + if (testInfo.retry === 0) throw new Error('the first attempt fails'); + }); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 0421c59..7e452fa 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -115,14 +115,21 @@ async function playwright(specs, env, cwd = PKG) { const walk = (suite, titles) => { for (const s of suite.suites ?? []) walk(s, s.file && s.title === s.file ? titles : [...titles, s.title]); for (const spec of suite.specs ?? []) { - const t = spec.tests[0]; - const r = t.results[t.results.length - 1] ?? {}; - tests[[...titles, spec.title].join(' › ')] = { - status: r.status ?? 'none', - error: (r.errors ?? []).map((e) => e.message ?? '').join('\n'), - annotations: r.annotations ?? t.annotations ?? [], - steps: steps.filter((x) => x.test === spec.title).map((x) => x.step), - }; + // Each run of the test (--repeat-each lists one spec for each), its + // last attempt. + const runs = spec.tests.map((t) => { + const r = t.results[t.results.length - 1] ?? {}; + return { + status: r.status ?? 'none', + error: (r.errors ?? []).map((e) => e.message ?? '').join('\n'), + annotations: r.annotations ?? t.annotations ?? [], + stderr: (r.stderr ?? []).map((e) => e.text ?? '').join(''), + attempts: t.results.length, + }; + }); + const key = [...titles, spec.title].join(' › '); + if (tests[key]) tests[key].runs.push(...runs); + else tests[key] = { ...runs[0], runs, steps: steps.filter((x) => x.test === spec.title).map((x) => x.step) }; } }; for (const s of JSON.parse(fs.readFileSync(report, 'utf8')).suites) walk(s, []); @@ -184,6 +191,10 @@ test('keploy skips a set-aside test with its reason, fails a judged-red one, and const { tests, tmp } = await playwright(['agent-says.spec.ts'], keployEnv(a.url)); assert.equal(tests['set aside'].status, 'skipped'); assert.deepEqual(tests['set aside'].annotations.filter((x) => x.type === 'skip').map((x) => x.description), ['keploy: flaky/request_changed: the body changed']); + // Playwright's terminal reporters show a test's output, not its skip's + // reason: the reason is in the output. + assert.equal(tests['set aside'].stderr, 'keploy: flaky/request_changed: the body changed (keploy skipped set aside)\n'); + assert.equal(tests['judged red'].stderr, ''); assert.ok(!fs.existsSync(path.join(tmp, 'set-aside-ran')), 'the set-aside test ran'); assert.equal(tests['judged red'].status, 'failed'); // The agent's judging reason already starts with "keploy: "; the test shows it once (gap13 showed it twice). @@ -197,6 +208,93 @@ test('keploy skips a set-aside test with its reason, fails a judged-red one, and } finally { await a.close(); } }); +test('keploy\'s skip reason is in the output Playwright\'s list reporter prints', async () => { + const j = journal(); + const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.name === 'set aside' + ? { body: { v: 1, status: 'ok', action: 'skip', reason: 'flaky/request_changed: the body changed' } } : undefined)); + try { + const { tests, out } = await playwright(['agent-says.spec.ts', '-g', 'set aside', '--reporter=list,json'], keployEnv(a.url)); + assert.equal(tests['set aside'].status, 'skipped', out); + assert.match(out, /keploy: flaky\/request_changed: the body changed \(keploy skipped set aside\)/); + } finally { await a.close(); } +}); + +// retry.spec.ts's test fails its first attempt, and its group retries it +// once whatever the command line says. +const retried = 'retried › fails its first attempt'; +const retryRefusal = { + code: 'test_run_again', + message: `Playwright retried p1::test/fixtures/retry.spec.ts::${retried}, and keploy runs each test once in a run`, + remedy: 'run the suite with --retries=0, and with no retries in its config or its test.describe.configure', +}; + +test('a test Playwright retries is not begun again: the retry fails, saying why, and keploy is told', async () => { + const j = journal(); + const a = await fakeAgent(j); + try { + const { tests, tmp, out } = await playwright(['retry.spec.ts'], keployEnv(a.url)); + const t = tests[retried]; + assert.equal(t.attempts, 2, out); + assert.equal(t.status, 'failed'); + assert.equal(t.error.split('\n')[0], `KeployError: keploy: ${retryRefusal.message} (${retryRefusal.remedy})`); + // The retry's body did not run. + assert.ok(fs.existsSync(path.join(tmp, 'attempt-0'))); + assert.ok(!fs.existsSync(path.join(tmp, 'attempt-1')), 'the retry ran'); + const r = ref('retry.spec.ts', retried); + const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; + assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + ['/agent/instance', claim], + ['/agent/scope/begin', { ...r, canSkip: true }], ['/agent/scope/end', { ...r, outcome: 'failed' }], + // The retry runs in a new worker, which claims the run again. + ['/agent/instance', claim], + ['/agent/refusal', retryRefusal], + ]); + } finally { await a.close(); } +}); + +test('a retry keploy could not be told of still fails, saying why and what the agent answered', async () => { + const j = journal(); + const a = await fakeAgent(j, (c) => (c.path === '/agent/refusal' ? { status: 500, body: 'boom' } : undefined)); + try { + const { tests, tmp, out } = await playwright(['retry.spec.ts'], keployEnv(a.url)); + const t = tests[retried]; + assert.equal(t.attempts, 2, out); + assert.equal(t.status, 'failed'); + assert.equal(t.error.split('\n')[0], `KeployError: keploy: ${retryRefusal.message} (${retryRefusal.remedy}); keploy could not be told: ` + + 'keploy: the keploy agent answered 500 Internal Server Error to /agent/refusal: "\\"boom\\""'); + assert.ok(!fs.existsSync(path.join(tmp, 'attempt-1')), 'the retry ran'); + assert.equal(calls(j).filter((c) => c.path === '/agent/scope/begin').length, 1); + } finally { await a.close(); } +}); + +test('without keploy a retried test is retried as written', async () => { + const { tests, tmp, out } = await playwright(['retry.spec.ts'], { KEPLOY_MOCK_AGENT: undefined, KEPLOY_TEST_PID: undefined }); + assert.equal(tests[retried].attempts, 2, out); + assert.equal(tests[retried].status, 'passed'); + assert.ok(fs.existsSync(path.join(tmp, 'attempt-1'))); +}); + +test('a test --repeat-each runs again is not begun again: the repeat fails, saying why, and keploy is told', async () => { + const j = journal(); + const a = await fakeAgent(j); + try { + const { tests, out } = await playwright(['agent-says.spec.ts', '-g', 'judged red', '--repeat-each=2'], keployEnv(a.url)); + assert.deepEqual(tests['judged red'].runs.map((t) => t.status), ['passed', 'failed'], out); + const message = 'Playwright ran p1::test/fixtures/agent-says.spec.ts::judged red again for --repeat-each, and keploy runs each test once in a run'; + const remedy = 'run the suite without --repeat-each, and with no repeatEach in its config'; + assert.equal(tests['judged red'].runs[1].error.split('\n')[0], `KeployError: keploy: ${message} (${remedy})`); + const r = ref('agent-says.spec.ts', 'judged red'); + const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; + assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + ['/agent/instance', claim], + ['/agent/scope/begin', { ...r, canSkip: true }], ['/agent/scope/end', { ...r, outcome: 'passed' }], + // Playwright runs each repeat in a worker of its own. + ['/agent/instance', claim], + ['/agent/refusal', { code: 'test_run_again', message, remedy }], + ]); + } finally { await a.close(); } +}); + test('a refused claim fails every test of the worker, and no test begins', async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/instance' diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 2265e0c..2ead81f 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -10,7 +10,7 @@ import { AgentClient, KeployError } from '../dist/agent.js'; import { decide } from '../dist/join.js'; import { Inflight } from '../dist/settle.js'; import { EventEmitter } from 'node:events'; -import { testRef, outcomeOf, sourceOf, keployText } from '../dist/index.js'; +import { testRef, outcomeOf, sourceOf, keployText, runAgain } from '../dist/index.js'; // One fake agent answering every call with answer(path, body). async function agent(answer) { @@ -133,6 +133,63 @@ test('a claim answered with a plain 404 names the route and the switch keploy sp } finally { await a.close(); } }); +test('a begin, an end or a refusal answered with a plain 404 names the route and the switch, as the claim does; keploy\'s own 404 shows keploy\'s reason', async () => { + const t = { name: 'g › a', file: 'a.spec.ts' }; + const proven = (route, what) => new RegExp(`^keploy: this keploy agent answered 404 to POST ${route.replace(/\//g, '\\/')}, so ${what}: ` + + 'keploy speaks @keploy\\/playwright\'s wire, scope protocol v1, only when it runs with KEPLOY_PROVEN=1 ' + + '\\(KEPLOY_PROVEN=1 keploy mock record\\|replay -c "npx playwright test --workers=1 --retries=0"\\); ' + + 'a keploy run without KEPLOY_PROVEN=1, or one from before scope protocol v1, answers 404 there$'); + const calls = [ + [(c) => c.begin(t), proven('/agent/scope/begin', 'g › a cannot begin')], + [(c) => c.end(t, 'passed'), proven('/agent/scope/end', 'g › a cannot end')], + [(c) => c.refuse('test_run_again', 'm', 'r'), proven('/agent/refusal', 'keploy cannot be told of the refusal')], + ]; + for (const [call, re] of calls) { + let a = await agent(() => ({ status: 404, body: '404 page not found' })); + try { + await rejects(call(new AgentClient(a.url, '')), re); + } finally { await a.close(); } + a = await agent(() => ({ status: 404, body: { error: 'not_open', message: 'g › a is not open', remedy: 'begin it first' } })); + try { + await rejects(call(new AgentClient(a.url, '')), /^keploy: g › a is not open \(begin it first\)$/); + } finally { await a.close(); } + // Any other plain answer keeps the generic text. + a = await agent(() => ({ status: 500, body: 'boom' })); + try { + await rejects(call(new AgentClient(a.url, '')), /^keploy: the keploy agent answered 500 Internal Server Error to \/agent\/[a-z/]+: "boom"$/); + } finally { await a.close(); } + } +}); + +test('refuse sends the code, the message and the remedy, and takes only ok', async () => { + let a = await agent(() => ({ body: { v: 1, status: 'ok' } })); + try { + await new AgentClient(a.url, 'tok').refuse('test_run_again', 'Playwright retried a', 'run with --retries=0'); + assert.deepEqual(a.calls.map((x) => [x.path, x.auth, x.body]), + [['/agent/refusal', 'Bearer tok', { code: 'test_run_again', message: 'Playwright retried a', remedy: 'run with --retries=0' }]]); + } finally { await a.close(); } + for (const body of [{ v: 1 }, { v: 1, status: 'busy' }]) { + a = await agent(() => ({ body })); + try { + await rejects(new AgentClient(a.url, '').refuse('test_run_again', 'm', 'r'), /^keploy: the keploy agent answered the refusal test_run_again with status "(busy)?"$/); + } finally { await a.close(); } + } +}); + +test('runAgain: a retry and a repeat of --repeat-each are refused, naming the test by its keploy ID; a first run is not', () => { + const ref = { name: 'g › a', file: 'tests/a.spec.ts', project: 'chromium' }; + assert.equal(runAgain(ref, { retry: 0, repeatEachIndex: 0 }), null); + const retry = { message: 'Playwright retried chromium::tests/a.spec.ts::g › a, and keploy runs each test once in a run', + remedy: 'run the suite with --retries=0, and with no retries in its config or its test.describe.configure' }; + assert.deepEqual(runAgain(ref, { retry: 1, repeatEachIndex: 0 }), retry); + // The message names no attempt: the agent keeps one refusal per test. + assert.deepEqual(runAgain(ref, { retry: 2, repeatEachIndex: 0 }), retry); + assert.deepEqual(runAgain(ref, { retry: 1, repeatEachIndex: 1 }), retry); + assert.deepEqual(runAgain({ name: 'g › a', file: 'tests/a.spec.ts' }, { retry: 0, repeatEachIndex: 1 }), { + message: 'Playwright ran tests/a.spec.ts::g › a again for --repeat-each, and keploy runs each test once in a run', + remedy: 'run the suite without --repeat-each, and with no repeatEach in its config' }); +}); + test('an unreachable agent is an error', async () => { const a = await agent(() => ({ body: {} })); await a.close(); From 44541b0062613be636479c7a3241511922f4505d Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 04:31:08 +0530 Subject: [PATCH 02/73] chore(playwright): release 0.1.2 Signed-off-by: slayerjain --- playwright/package-lock.json | 4 ++-- playwright/package.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/playwright/package-lock.json b/playwright/package-lock.json index 6ab487c..1259dd4 100644 --- a/playwright/package-lock.json +++ b/playwright/package-lock.json @@ -1,12 +1,12 @@ { "name": "@keploy/playwright", - "version": "0.1.1", + "version": "0.1.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@keploy/playwright", - "version": "0.1.1", + "version": "0.1.2", "license": "Apache-2.0", "devDependencies": { "@playwright/test": "1.62.1", diff --git a/playwright/package.json b/playwright/package.json index 6b7cd7a..9556839 100644 --- a/playwright/package.json +++ b/playwright/package.json @@ -1,6 +1,6 @@ { "name": "@keploy/playwright", - "version": "0.1.1", + "version": "0.1.2", "description": "Join a Playwright suite to keploy's Universal Mocker: keploy mock record and keploy mock replay judge each test.", "license": "Apache-2.0", "author": "Keploy ", From ca9f96a01f920eec0f754d23e4a5c2cced3db0d0 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 15:25:32 +0530 Subject: [PATCH 03/73] feat(playwright): seed each test's browser random numbers from keploy A page that draws random numbers can make other calls each run, and no recording answers a call it never saw: stripe.js flips a 50/50 A/B coin from crypto.randomUUID() at each page load, and its two sides load files that never overlap, so a test on such a page fails about half its replays. An ID or idempotency key the browser makes differs too. Under keploy, each test's begin now sends `random: "seeded"` and a seed drawn afresh (128 bits as hex); keploy answers the seed to draw from: in a record the begin's own, which it keeps with the test's recording, in a replay the recording's. The test's context gets an init script, before its first page, that puts seeded streams behind Math.random, crypto.getRandomValues and crypto.randomUUID in every frame before any script of the page runs. Each function has its own stream, and each document its own, named by its page (in the order the test opened its pages; a popup by the window its opener opened), its frame (by its element's path) and how many documents of that path the tab loaded before: a reload draws new numbers, the same ones each run. The end sends what the test did (`seeded` or `own`). A document the seeding cannot place (a popup opened with noopener or at another origin, a frame that cannot count its documents, a top document with no session storage) draws the browser's own numbers, never another page's, and the test names it once per address. A begin answered with no seed (an older recording or an older keploy) draws the browser's own numbers and says so. On by default. KEPLOY_SEED_RANDOM=off turns it off for the run, and the option keploySeedRandom: false for a project, a file or a test; the begin then sends `random: "own"` and no seed. Without keploy nothing changes. Web Workers and Service Workers keep the browser's own numbers (README limits). No row checks a retry's random numbers: a retry fails before its begin, since keploy runs each test once in a run. The rows run in Chromium, Firefox and WebKit, so CI installs all three. Signed-off-by: slayerjain --- .github/workflows/playwright-sdk.yml | 4 +- playwright/README.md | 42 + playwright/src/agent.ts | 43 +- playwright/src/index.ts | 167 +++- playwright/src/random.ts | 713 +++++++++++++++++ playwright/test/draws.mjs | 67 ++ playwright/test/fixtures/engines.spec.ts | 113 +++ playwright/test/fixtures/first.spec.ts | 34 + playwright/test/fixtures/many.spec.ts | 177 +++++ playwright/test/fixtures/parse.spec.ts | 38 + playwright/test/fixtures/places.spec.ts | 99 +++ playwright/test/fixtures/playwright.config.ts | 13 +- playwright/test/fixtures/seeded.spec.ts | 40 + playwright/test/fixtures/tabs.spec.ts | 61 ++ playwright/test/random.test.mjs | 681 ++++++++++++++++ playwright/test/rows.test.mjs | 737 +++++++++++++++++- playwright/test/unit.test.mjs | 24 + 17 files changed, 3008 insertions(+), 45 deletions(-) create mode 100644 playwright/src/random.ts create mode 100644 playwright/test/draws.mjs create mode 100644 playwright/test/fixtures/engines.spec.ts create mode 100644 playwright/test/fixtures/first.spec.ts create mode 100644 playwright/test/fixtures/many.spec.ts create mode 100644 playwright/test/fixtures/parse.spec.ts create mode 100644 playwright/test/fixtures/places.spec.ts create mode 100644 playwright/test/fixtures/seeded.spec.ts create mode 100644 playwright/test/fixtures/tabs.spec.ts create mode 100644 playwright/test/random.test.mjs diff --git a/.github/workflows/playwright-sdk.yml b/.github/workflows/playwright-sdk.yml index 5dc9b0a..4b86de7 100644 --- a/.github/workflows/playwright-sdk.yml +++ b/.github/workflows/playwright-sdk.yml @@ -36,5 +36,7 @@ jobs: with: node-version: ${{ matrix.node }} - run: npm ci - - run: npx playwright install --with-deps chromium + # The rows run in Chromium, and the seeding's engine rows in Firefox and + # WebKit too. + - run: npx playwright install --with-deps chromium firefox webkit - run: npm test diff --git a/playwright/README.md b/playwright/README.md index 3e90fa7..98c180c 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -47,15 +47,57 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see the limits below). +- **Seeded random numbers.** Each test's browser draws its random numbers (`Math.random`, `crypto.getRandomValues`, `crypto.randomUUID`) from a seeded stream, so a replay draws the numbers its recording drew. A page that picks what to load at random (an A/B coin, such as stripe.js's at each page load) then makes the same calls in replay as in its recording, which keploy can answer. + - **Where the seed comes from.** Each test's begin sends a seed drawn afresh (`seed`, 128 bits as hex). keploy answers the seed to draw from. A record answers the begin's own, and keeps it in the test's mapping entry (`seed:`, beside its `at:`). A replay (and keploy's auto-replay) answers the one the recording kept. So two tests draw different numbers, and recording a test again draws new ones: an ID the browser makes is not sent twice to a real server. The seed is not derived from the test's name, so a test whose entry is renamed keeps its seed with the rest of the entry. + - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser then draws its own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser draws its own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. + - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser's pages change: not the app, its server or the browser's workers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **How a page is named.** By its place in the test's own order, the same in every run: + - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; + - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); + - a frame of its parent document's origin, by that document and by its place among the document's frames: the path its element asks for (`src`, or `about:srcdoc`, or `about:blank` for none) and how many frames the document had put in at that path before it, counted in the order the document holds the frame elements when it numbers them (as the task that put them in ends, or as a frame's document begins), never by their windows (the document finds a frame by its `frameElement`). For frames one task puts in out of document order, that is document order in Chromium and WebKit and the order they were put in in Firefox: the same each run, for a given app and engine. Frames at other paths, and the order the frames' documents load in, do not move it; it keeps its place whatever it loads, and one put in after a frame's removal takes the next number at its path. A frame in an open shadow root is placed so too. An `about:blank` frame does not draw its parent document's numbers; + - a frame under a parent of another origin, which it cannot read (a cross-site frame), by the page it is in and its path. One with no session storage to count its documents in (a sandboxed frame without `allow-same-origin`) is never seeded (below). + + A document the seeding cannot place draws the browser's own numbers, never another page's. The test then prints `keploy: in , the document at
drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. The address is the document's origin and path; for an opaque origin (`about:blank`, `about:srcdoc`, a `data:` URL, a sandboxed frame or page) it is the document's URL with no query or fragment, and `about:blank` for a window's first document that has no URL yet (Firefox runs init scripts there before it has one). The limits below list those documents. + - **What keploy is told.** The begin says what the test means to do (`random: "seeded"` or `"own"`). The end says what its pages drew from: `"seeded"` when a page opened in its seeded `context`, `"own"` when a page opened there unseeded, or when keploy answered no seed. When a test's calls still vary between runs, keploy can then say whether its pages were seeded, so the variation comes from somewhere the seeding does not reach, or whether seeding them is left to try. + - **On by default.** To turn it off, for record and replay alike: + - for the whole run: `KEPLOY_SEED_RANDOM=off` (or `0`, `false`; unset, `on`, `1` or `true` seed; any other value fails every test, naming the variable); + - for a project, a file or a test: the option `keploySeedRandom: false`, in the config's or a project's `use`, or with `test.use({ keploySeedRandom: false })`. Its type is `KeployOptions`: `defineConfig({ use: { keploySeedRandom: false } })`. + Any answer from the agent that this package cannot trust fails the test. That covers an agent it cannot reach, an error answer, an unreadable answer, and an answer that does not speak scope protocol v1. Nothing is retried. ## Limits of this version - Settle covers the `context` and `page` fixtures. A context the test makes with `browser.newContext()` is the test's own to close. - Settle's rows run in Chromium. Which calls the browser itself ends (a removed frame's) is proven there only; Firefox and WebKit are not rowed. +- Seeding's rows run in Chromium. Its rows for frames, for a popup that a page's first document opens, for top documents with no session storage, and for the test's own context init script run in Firefox and WebKit too. A popup that a page's first document opens with `window.open('about:blank')` draws the browser's own numbers and is named in every engine: in Chromium it has no session storage (its opener's origin is opaque), and in Firefox and WebKit it has no place, since its document begins before `window.open` returns. +- Seeding covers the pages of the `context` and `page` fixtures. These draws keep the browser's own random numbers, or the app's: + - the app's server, and the test's own code in Node; + - Web Workers, Shared Workers and Service Workers, since an init script does not reach them; + - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; + - `crypto.subtle`'s keys. + + A page whose own draws vary in number before another script draws (a draw per timer tick) still shifts that function's stream within the document. + + Two frames at one path can still trade numbers, unnamed, where the order they are counted in is not the same each run: + - a frame's number at its path is how many frames its parent document had put in at that path before it, so frames at one path that the page or a third-party script puts in, in an order or a number that varies run to run (as their scripts load, in network order; a re-render that makes a varying number of frames), trade numbers, unnamed. Chat and support widgets often put in `about:blank` frames as their scripts load; + - frames under a parent of another origin are counted by their origin in the tab, so two of one origin at one path take their numbers in the order their documents begin, which may change at replay; + - a frame put into a shadow root already in the page is not seen as it is put in, since the parent document watches only the nodes added to or removed from it outside its shadow roots: the frame is numbered at the document's next numbering (when the app next puts a frame in outside its shadow roots, or when a document of a frame not yet numbered, its own included, begins), so one put there before another frame at its path, at a moment that varies run to run, may take that frame's number. +- Seeding cannot place these documents, so they draw the browser's own numbers and are named (above): + - a popup opened with noopener, as a link with `target="_blank"` opens one, or opened other than by `window.open` (a form, or a link with `rel="opener"`, that targets a new window); + - a popup at another origin than its opener's page, or whose opener's page has loaded another document by the time the popup's begins, or whose opener the seeding could not place (below); + - a frame of another origin inside one of those popups; + - a frame its parent document cannot list (one in a closed shadow root, or in an `object` or `embed` element), one whose document began after the frame was removed, and a frame inside one; + - a frame under a parent of another origin that has no session storage to count its documents in (a sandboxed frame without `allow-same-origin`), alone under its parent or not, and a frame inside one: nothing tells its documents apart from the next ones at its path there (a reload, or a frame the app puts in again at its place) or from those of a frame beside it, so seeded, they could draw the same numbers. Its line says how many other frames its parent holds, counted once the parent's parser has ended (its `DOMContentLoaded`), so frames its markup puts in after a script that holds the parser count even when this frame drew, and loaded, before they came; when the parser has not ended by the test's end, counted then; when the frame is gone by then (its parent moved on to another document), the line names no other frame; + - a top document with no session storage that seeding can keep its count in, and a frame of its origin inside one. A document of an opaque origin has none: an `about:blank` loaded after its page's first document, a `data:` URL, a page served with a CSP `sandbox` without `allow-same-origin`, or a popup that a page's first document opens; nor has a document whose app has filled its session storage. Nothing tells such a document apart from the next one at its path in the tab (a reload, or another visit to that path), so seeded, they could draw the same numbers. So a test that loads `about:blank` (`page.goto('about:blank')`) and writes its page into it with `page.setContent` draws the browser's own numbers there: write into the page's first document (before its first `goto`), or serve the page from the app's origin, to seed it. A page's first document has no session storage in any engine either, but it is seeded: it is the one document of its kind in its tab (no later document there is marked, so the rule above holds for each of them), and before `newPage` returns the fixture marks it as the page's first with a value drawn afresh for each context and given only to that context's init script, so a page's own script cannot mark another document so. A frame of another origin inside such a document is placed as any frame under a parent of another origin (above). + + The end still says `"seeded"` for such a test: its `context` was seeded. A context init script of the test's own that draws random numbers as a document begins runs before the page is named, so that document has no place either; its line says it is in a page the test opened. +- Seeding's cost in a page: each document keeps a `MutationObserver` on its nodes (`childList`, its whole subtree) and looks at each node put in (and, while the document holds a numbered frame, each node taken out) for a frame; it walks the document (and its open shadow roots) to number its frames only when the app puts a frame in, or when a document of a frame not yet numbered begins. Most of the cost is the browser's own for such an observer, for each node put in, the parser's included. Measured as the median of five runs: in Chromium, seeding adds about 3 µs a node put in, about 170 ms to the load of a page of 40,000 elements, of which an observer that does nothing takes about 160 ms; in Firefox it adds about 45 ms to that load (27 for that observer), and in WebKit about 40 ms (29). A page that changes 300 times beside 20,000 elements or 2,000 open shadow roots takes within 2 ms of an unseeded one in Chromium and Firefox. +- The end reports only what the `context` fixture's pages drew from. A test none of whose pages came from it (an API-only test, a context it made itself) says nothing at its end, so keploy keeps what its begin said. +- Seeding keeps a count of each document path of a top document, and of a frame under a parent of another origin, in its origin's session storage in the tab, under the key `__keploy_random_loads`; a frame of its parent's origin is counted by its parent document. A frame under a parent of another origin with no session storage (a sandboxed frame) is never seeded (above). A top document with no session storage (an opaque origin) is never seeded either, but for a page's first document (above). The count is kept per origin, so the first documents of one page, or of frames under parents of other origins, at the same path in two origins draw the same numbers. Seeding also keeps two properties on each window, not enumerable: `__keploy_random_page` (the page's name) and `__keploy_random_doc` (what a popup or a frame reads to take its place). In a seeded page, `Math.random`, `crypto.getRandomValues`, `crypto.randomUUID` and `window.open` (which records the popups a document opens) are seeding's own functions, not the browser's native ones: `String(window.open)` shows no `[native code]`. - Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. - The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim. Each test is skipped at its begin, after the group's hooks have started; skipping the whole group before its hooks run needs a way to ask keploy for the set-aside tests first, which this version does not have. - `given` needs the agent route `POST /agent/scope/given` (keploy/keploy#5016). keploy v3.6.192 and later serve it; with the enterprise keploy, that is an enterprise release on keploy v3.6.192 or later. An older keploy answers that route with a plain 404, and `given` fails the test with `keploy: this keploy agent answered 404 to POST /agent/scope/given, so it cannot take the given value : given needs keploy v3.6.192 or later (keploy/keploy#5016), or an enterprise release on keploy v3.6.192 or later; an older keploy answers 404 there`. Claim, begin, settle and end do not need that route. +- Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser draws its own random numbers and prints the line that says so (above). Claim, begin, settle and end work with it as before. ## Releasing diff --git a/playwright/src/agent.ts b/playwright/src/agent.ts index c523f49..1b81fe5 100644 --- a/playwright/src/agent.ts +++ b/playwright/src/agent.ts @@ -32,6 +32,7 @@ export interface Answer { instance?: string; timeFile?: string; variable?: string; + seed?: string; } /** An answer this package cannot trust, or the agent's own refusal. */ @@ -141,21 +142,37 @@ export class AgentClient { return a; } - /** Asks the agent to open a test. */ - async begin(test: TestRef): Promise { - const a = await this.onWire(PATH_BEGIN, { ...test, canSkip: true }, `${test.name} cannot begin`); + /** + * Asks the agent to open a test. random says how the test's browser means + * to draw its random numbers, so a finding that the test's calls vary + * between runs can say whether seeding them is left to try. seed, sent + * when it means to seed them, is one drawn afresh for this begin; the + * answer's seed is the one to draw from: in a record the begin's own, in a + * replay the one the test's recording kept, and none when it kept none. An + * agent from before either field ignores it and answers no seed. + */ + async begin(test: TestRef, random?: Random, seed?: string): Promise { + const a = await this.onWire(PATH_BEGIN, { ...test, canSkip: true, ...(random ? { random } : {}), ...(seed ? { seed } : {}) }, `${test.name} cannot begin`); if (a.status !== 'ok') { throw new KeployError(`keploy: the keploy agent answered the begin of ${test.name} with status ${JSON.stringify(a.status ?? '')}`); } if (a.action && a.action !== ACTION_SKIP) { throw new KeployError(`keploy: the keploy agent answered the begin of ${test.name} with the unknown action ${JSON.stringify(a.action)}`); } + if (a.seed !== undefined && (typeof a.seed !== 'string' || (a.seed !== '' && !SEED.test(a.seed)))) { + throw new KeployError(`keploy: the keploy agent answered the begin of ${test.name} with the seed ${quoted(typeof a.seed === 'string' ? a.seed : String(JSON.stringify(a.seed)))}, ` + + 'which is no seed keploy keeps (8 to 64 lower-case hex digits)'); + } return a; } - /** Tells the agent the test ended, with its outcome. */ - async end(test: TestRef, outcome: Outcome): Promise { - const a = await this.onWire(PATH_END, { ...test, outcome }, `${test.name} cannot end`); + /** + * Tells the agent the test ended, with its outcome, and, when it is known, + * how its browser drew its random numbers (random), which replaces what its + * begin meant to do. + */ + async end(test: TestRef, outcome: Outcome, random?: Random): Promise { + const a = await this.onWire(PATH_END, { ...test, outcome, ...(random ? { random } : {}) }, `${test.name} cannot end`); if (a.status !== 'ok') { throw new KeployError(`keploy: the keploy agent answered the end of ${test.name} with status ${JSON.stringify(a.status ?? '')}`); } @@ -209,6 +226,20 @@ export interface TestRef { export type Outcome = 'passed' | 'failed' | 'skipped'; +/** + * How a test's browser draws its random numbers: RANDOM_SEEDED from the + * seed keploy answered its begin with, RANDOM_OWN the browser's own + * (KEPLOY_SEED_RANDOM=off, the keploySeedRandom option false, or keploy + * answered no seed). Sent with the begin, it is what the test means to do; + * with the end, what its pages did. + */ +export type Random = typeof RANDOM_SEEDED | typeof RANDOM_OWN; +export const RANDOM_SEEDED = 'seeded'; +export const RANDOM_OWN = 'own'; + +/** A seed as keploy keeps and answers one: 8 to 64 lower-case hex digits. */ +export const SEED = /^[0-9a-f]{8,64}$/; + async function readBounded(res: Response): Promise { if (!res.body) return ''; const reader = res.body.getReader(); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 60f0125..7c65a01 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -20,31 +20,42 @@ // /agent/instance), under its Playwright project's name, with keploy's // project folder (KEPLOY_MOCK_PROJECT), which its test files are named // from; the current folder only when keploy names none; -// - each test begins (POST /agent/scope/begin) before its beforeEach hooks; -// a test keploy set aside is skipped with keploy's reason, which the -// test's output shows; +// - each test begins (POST /agent/scope/begin) before its beforeEach hooks, +// saying whether its browser's random numbers are to be seeded (random: +// "seeded" or "own") and, when they are, with a seed drawn afresh; keploy +// answers the seed to draw from; a test keploy set aside is skipped with +// keploy's reason, which the test's output shows; // - a test Playwright runs again in the run (a retry, a --repeat-each // repeat) fails without beginning, and keploy is told (POST // /agent/refusal), since keploy runs each test once in a run; // - before the test's browser context closes, the test waits until none of // its pages' calls is in flight (settle), so its calls end inside it; // - the test ends (POST /agent/scope/end) after its last teardown, with -// its outcome; when keploy saw a call with no recorded answer, or a +// its outcome and what its context's pages drew their random numbers +// from (random); when keploy saw a call with no recorded answer, or a // recorded call the test no longer made, the test fails with keploy's // text; // - given(name, value) tells keploy a value the test made itself (a // session token it signs, a name it stamps with the time), which keploy -// follows where the test's calls carry it. +// follows where the test's calls carry it; +// - each test's browser draws its random numbers (Math.random, +// crypto.getRandomValues, crypto.randomUUID) from a stream seeded by the +// seed keploy answered its begin with, which a record keeps with the +// test's recording, so a replay's pages make the calls the recording's +// made; each page draws its own, named by its place in the test's +// order; KEPLOY_SEED_RANDOM=off or the option keploySeedRandom: false +// turns it off. // Any answer of the agent this package cannot trust fails the test. import { test as base, expect } from '@playwright/test'; -import type { TestInfo } from '@playwright/test'; +import type { Page, TestInfo } from '@playwright/test'; import fs from 'node:fs'; import path from 'node:path'; -import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError } from './agent'; -import type { Outcome, TestRef } from './agent'; +import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError, RANDOM_OWN, RANDOM_SEEDED } from './agent'; +import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; +import { SEEDED_LINE, besideOf, freshSeed, initScript, othersNow, pageAddress, placePages, seedingFromEnv, unplacedLine, unseededLine, type Unplaced } from './random'; import { Inflight } from './settle'; export { expect, KeployError }; @@ -62,15 +73,64 @@ let client: AgentClient | null = null; let projectDir = ''; // The test this worker has open, if any. let open: TestRef | null = null; +// Whether the environment leaves the test browser's seeding on. +let seedingOn = false; +// Whether this worker has said that it seeds the test browser. +let saidSeeded = false; +// How many contexts this worker seeded: each one's report function has its +// own name. +let reports = 0; + +// How a test's browser draws its random numbers. +type PageSeeding = { + // Whether its begin asked keploy for a seed (seeding on, not opted out). + asked: boolean; + // The seed keploy answered; '' for none. + seed: string; + // Whether a page opened in its context fixture's context, seeded + // (seededPage) or not (ownPage). + seededPage: boolean; + ownPage: boolean; +}; +// The open test's, while one is open. +let seeding: PageSeeding | null = null; + +/** + * What the test's end says its browser drew from: seeded when a page of its + * context was seeded, own when a page of its context was not, or when it + * asked for a seed and keploy answered none. Nothing for a test whose pages + * did not come from the context fixture (an API-only test, a context it made + * itself): keploy keeps what its begin said. + */ +function randomDone(s: PageSeeding): Random | undefined { + if (s.seededPage) return RANDOM_SEEDED; + if (s.ownPage || (s.asked && s.seed === '')) return RANDOM_OWN; + return undefined; +} + +/** @keploy/playwright's options, for a config's or a file's use. */ +export type KeployOptions = { + /** + * Seed the test browser's random numbers per test under keploy (default + * true), so a replay's pages draw what the recording's drew. Set false, + * for record and replay alike, to give a project, a file or a test the + * browser's own random numbers; KEPLOY_SEED_RANDOM=off does so for the + * whole run. + */ + keploySeedRandom: boolean; +}; type WorkerFixtures = { keployWorker: void }; -type TestFixtures = { keployTest: void }; +type TestFixtures = { keployTest: void } & KeployOptions; export const test = base.extend({ keployWorker: [async ({}, use, workerInfo) => { const j = joining(); if (j.mode === 'refused') throw new KeployError(j.refusal); if (j.mode === 'active') { + const s = seedingFromEnv(process.env); + if (s.mode === 'refused') throw new KeployError(s.refusal); + seedingOn = s.mode === 'seeded'; const c = new AgentClient(j.url, j.token); projectDir = realpath(j.project || process.cwd()); // The claim's key names the slice of the recording this worker's @@ -86,13 +146,86 @@ export const test = base.extend({ await use(); }, { scope: 'worker', auto: true }], - context: async ({ context }, use) => { - if (client === null) { + keploySeedRandom: [true, { option: true }], + + // After keployTest: its begin's answer holds the seed. + context: async ({ context, keployTest }, use) => { + void keployTest; + const s = seeding; + if (client === null || s === null || open === null) { await use(context); return; } + const name = open.name; + let restore = (): void => {}; + // The lines that wait for a parent's parser to end, each said at the + // test's end if it has not. + const waiting = new Set<() => void>(); + if (s.seed !== '') { + // A document the seeding cannot place draws the browser's own numbers + // and says so here, once for each address and reason in the test. + const report = `__keploy_random_unplaced_${++reports}`; + const told = new Set(); + let opened: (page: Page | null | undefined) => Promise = async () => false; + await context.exposeBinding(report, async (source, at: unknown, why: unknown, above: unknown) => { + const a = pageAddress(at); + const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' ? why : (await opened(source.page)) ? 'own' : 'popup'; + // Once for each address and reason: two documents at one address + // (about:blank) may have no place for two reasons. + const k = `${w}\n${a}`; + if (told.has(k)) return; + told.add(k); + if (w !== 'uncounted') { + process.stdout.write(`${unplacedLine(name, a, w)}\n`); + return; + } + // Its line says how many frames are beside its frame once their + // parent's parser has ended, or at the test's end. + const b = besideOf(source.frame, Number(above)); + let said = false; + const say = (): void => { + if (said) return; + said = true; + waiting.delete(say); + process.stdout.write(`${unplacedLine(name, a, w, othersNow(b))}\n`); + }; + waiting.add(say); + if (b !== null) { + try { + await b[1].waitForLoadState('domcontentloaded', { timeout: 0 }); + } catch { + // The parent, or its page, is gone. + } + } + say(); + }); + // Before the test's first page: an init script runs in every frame's + // document before its own scripts. + // The value by which the page fixture's script marks a page's first + // document; the page's own scripts do not know it. + const mark = freshSeed(); + await context.addInitScript({ content: initScript(s.seed, report, mark) }); + const placing = placePages(context, mark); + restore = placing.restore; + opened = placing.opened; + if (!saidSeeded) { + saidSeeded = true; + process.stdout.write(`${SEEDED_LINE}\n`); + } + } else if (s.asked) { + process.stdout.write(`${unseededLine(name)}\n`); + } + const seeded = s.seed !== ''; + const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; + if (context.pages().length > 0) opened(); + context.on('page', opened); const inflight = new Inflight(context); - await use(context); + try { + await use(context); + } finally { + restore(); + for (const say of [...waiting]) say(); + } if (inflight.size > 0) { const calls = inflight.describe(); const shown = calls.slice(0, 5).join(', ') + (calls.length > 5 ? `, and ${calls.length - 5} more` : ''); @@ -100,7 +233,7 @@ export const test = base.extend({ } }, - keployTest: [async ({ keployWorker }, use, testInfo) => { + keployTest: [async ({ keployWorker, keploySeedRandom }, use, testInfo) => { void keployWorker; const c = client; if (c === null) { @@ -122,7 +255,8 @@ export const test = base.extend({ } throw new KeployError(`keploy: ${again.message} (${again.remedy})${told}`); } - const b = await c.begin(ref); + const asked = seedingOn && keploySeedRandom; + const b = await c.begin(ref, asked ? RANDOM_SEEDED : RANDOM_OWN, asked ? freshSeed() : undefined); if (b.action === ACTION_SKIP) { // The framework's own skip, with keploy's reason; no end follows. // Playwright's list, line and dot reporters print a test's output but @@ -132,10 +266,13 @@ export const test = base.extend({ testInfo.skip(true, why); return; } + const s: PageSeeding = { asked, seed: asked ? b.seed ?? '' : '', seededPage: false, ownPage: false }; open = ref; + seeding = s; await use(); open = null; - const e = await c.end(ref, outcomeOf(testInfo)); + seeding = null; + const e = await c.end(ref, outcomeOf(testInfo), randomDone(s)); if (e.action === ACTION_FAIL) throw new KeployError(keployText(e.reason, 'keploy: the keploy agent failed this test and gave no reason')); }, { auto: true }], }); diff --git a/playwright/src/random.ts b/playwright/src/random.ts new file mode 100644 index 0000000..66e2aba --- /dev/null +++ b/playwright/src/random.ts @@ -0,0 +1,713 @@ +// Seeded randomness for the test browser. A page that draws random numbers +// (stripe.js picks one of two builds from crypto.randomUUID() at each page +// load) makes other calls each run, and no recording can answer a call it +// never saw. So under keploy each test's browser draws from a seeded stream +// instead. Each test's begin sends a seed drawn afresh, and keploy answers +// the seed to draw from: a record keeps the begin's with the test's +// recording, and a replay answers the recording's. So record and replay +// draw the same numbers and pick the same side, two tests draw different +// ones, and a record made again draws new ones (a browser-made ID is not sent +// twice to a real server). Each page of a test draws its own numbers, so two +// tabs do not make one ID. Only the test browser's pages change; the app, +// its server and the browser's workers do not. +// +// On by default; KEPLOY_SEED_RANDOM=off or the fixture option +// keploySeedRandom: false turns it off. + +import type { BrowserContext, Frame, Page } from '@playwright/test'; +import crypto from 'node:crypto'; + +export const ENV_SEED_RANDOM = 'KEPLOY_SEED_RANDOM'; + +/** The option that turns seeding off for a project, a file or a test. */ +export const OPTION_SEED_RANDOM = 'keploySeedRandom'; + +export type Seeding = { mode: 'seeded' } | { mode: 'off' } | { mode: 'refused'; refusal: string }; + +/** Whether the environment leaves seeding on (unset, on) or turns it off. */ +export function seedingFromEnv(env: NodeJS.ProcessEnv): Seeding { + const v = (env[ENV_SEED_RANDOM] ?? '').trim(); + const l = v.toLowerCase(); + if (l === '' || l === 'on' || l === '1' || l === 'true') return { mode: 'seeded' }; + if (l === 'off' || l === '0' || l === 'false') return { mode: 'off' }; + return { + mode: 'refused', + refusal: `keploy: ${ENV_SEED_RANDOM}=${JSON.stringify(v)} is neither on nor off; set it to off to give the test browser its own random numbers, ` + + 'or unset it to seed them per test', + }; +} + +/** The line a worker shows once, at the first test whose browser it seeds. */ +export const SEEDED_LINE = 'keploy: the test browser\'s random numbers (Math.random, crypto.getRandomValues, crypto.randomUUID) are seeded per test ' + + 'from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew; ' + + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named). ' + + `To turn it off, record and replay with ${ENV_SEED_RANDOM}=off, or set use: { ${OPTION_SEED_RANDOM}: false } in the Playwright config.`; + +/** + * The line a test shows when it asked keploy for a seed and got none, so its + * pages draw the browser's own random numbers. + */ +export function unseededLine(name: string): string { + return `keploy: keploy answered the begin of ${name} with no seed, so its browser draws its own random numbers: ` + + 'its recording kept none (it was recorded before keploy kept seeds, or with seeding off: record the test again with seeding on, the default, to seed it), ' + + 'or the test was pulled from a Keploy branch, which keeps no seed, or this keploy predates seeds.'; +} + +/** + * An address a page sent, as the run's output shows it: one line (each + * control character a ?), of at most 512 characters. + */ +export function pageAddress(at: unknown): string { + return String(at).replace(/[\u0000-\u001f\u007f]/g, '?').slice(0, 512); +} + +/** + * Why the seeding could not place a document: 'own' for one in a page the + * test opened that drew before keploy named the page, 'frame' for a frame + * it cannot place among its parent's frames (initScript), 'popup' for one + * in a page the app opened, 'uncounted' for a frame under a parent of + * another origin that has no session storage to count its documents in, or + * a frame inside one, 'nostorage' for a top document with no session + * storage to count its documents in (other than a page's first document), + * or a frame of its origin inside one. + */ +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage'; + +const UNPLACED: Record, string> = { + own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', + frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + + 'which its parent document cannot list, or one whose document began after its removal), or a frame inside one.', + popup: 'it is in a page the app opened that keploy cannot place in the test\'s order (a popup opened with noopener, as a link with target=_blank opens one, ' + + 'or at another origin than its opener\'s page, or after that page moved on, or by a document keploy could not place; or a frame of another origin in one).', + nostorage: 'it is a top document with no session storage that keploy can keep its count in (a document of an opaque origin has none: an about:blank loaded after ' + + 'its page\'s first document, a data: URL, a page served with a CSP sandbox without allow-same-origin, or a popup that a page\'s first document opened; ' + + 'or the app has filled it), or a frame of its origin inside one: keploy cannot count the documents loaded at its path in the tab, so seeded, ' + + 'the next one there (a reload, or another visit to that path) could draw the same numbers.', +}; + +/** + * The line a test shows, once for each address and reason, when a document + * of its seeded context drew the browser's own random numbers because the + * seeding could not place it among the test's pages (initScript); others: for + * 'uncounted', how many other frames are beside the frame with no session + * storage (under its parent): the document's own frame, or the one it is in + * (othersNow). + */ +export function unplacedLine(name: string, at: string, why: Unplaced = 'popup', others = 0): string { + const n = Math.floor(Number(others)); + const reason = why === 'uncounted' + ? 'it is a frame under a parent of another origin with no session storage (a sandboxed frame without allow-same-origin), or a frame inside one: ' + + 'keploy cannot count the documents that frame loads, so seeded, the next document there (a reload, or a frame the app puts in again at its place)' + + (n > 0 ? ` or, at its path, one of the ${n} other frame(s) beside that frame` : '') + ' could draw the same numbers.' + : UNPLACED[why]; + return `keploy: in ${name}, the document at ${at} drew the browser's own random numbers, so its draws may differ at replay: ${reason}`; +} + +/** + * The frame with no session storage that an 'uncounted' line names, and its + * parent: frame is the document's own frame, and above how many frames up + * from it that one is (initScript; 0 for its own). null when it has no + * parent (a top document, or one further up than the top). + */ +export function besideOf(frame: Frame, above: number): [Frame, Frame] | null { + let f: Frame | null = frame; + for (let i = 0; i < above && f !== null; i++) f = f.parentFrame(); + const parent = f === null ? null : f.parentFrame(); + return f === null || parent === null ? null : [f, parent]; +} + +/** + * How many other frames are beside that frame under its parent (besideOf) + * now; 0, so the line names no other frame, when there is no such frame or + * it is gone (its parent may have moved on to another document, which holds + * other frames). + * The fixture counts once the parent's parser has ended (its + * DOMContentLoaded), or at the test's end if it has not: a frame draws as it + * loads, and may draw and load while its parent's parser holds the frames + * after it in its markup (behind a script it loads), so a count taken at + * its draw, or at its own load, would depend on the network. + */ +export function othersNow(beside: [Frame, Frame] | null): number { + return beside === null || beside[0].isDetached() ? 0 : beside[1].childFrames().length - 1; +} + +/** A seed drawn afresh for a test's begin: 128 bits, as hex. */ +export function freshSeed(): string { + return crypto.randomBytes(16).toString('hex'); +} + +// A key of the page's session storage: how many documents of each path the +// tab has loaded from the document's origin, so that a document loaded again +// in the same tab (a reload, a second visit) draws new numbers, as it would +// unseeded, and still the same ones each run. +export const LOADS_KEY = '__keploy_random_loads'; + +// The window property, not enumerable, that names the page a document is +// in: set by pageScript in every document of a page the test opened. +export const PAGE_KEY = '__keploy_random_page'; + +// The window property, not enumerable, that holds a document's place for the +// windows it opens and the frames it holds. +export const DOC_KEY = '__keploy_random_doc'; + +/** + * The init script that names the page a test opened (context.newPage(), the + * page fixture's among them) in each of its documents and frames: its place + * among the pages the test opened in its context, in the order the test + * opened them. Before newPage returns, the fixture adds it to the page, for + * each document the page loads, and runs it in the page's first document + * (its about:blank, which setContent writes into): initScript reads the + * name at the document's first draw. + */ +export function pageScript(id: string): string { + return `Object.defineProperty(globalThis, ${JSON.stringify(PAGE_KEY)}, { value: ${JSON.stringify(id)}, enumerable: false, writable: false, configurable: true });\n`; +} + +/** + * pageScript for the page's first document (its about:blank, which + * setContent writes into), run in it as newPage returns: it also tells the + * seeding in that document (initScript) that the document is the page's + * first, with mark, the value the context's init script was given, which + * the page's own scripts do not know. No engine gives that document session + * storage, and it is the one document of its kind in its tab, so it is + * seeded with no count; any other top document with no session storage is + * not. + */ +export function firstScript(id: string, mark: string): string { + return `${pageScript(id)}try { globalThis[${JSON.stringify(DOC_KEY)}].first(${JSON.stringify(mark)}); } catch (e) {}\n`; +} + +/** + * Names each page the test opens in context (context.newPage(), the page + * fixture's among them) by its place in the order the test opened them, in + * every document of the page, its first about:blank included, before + * newPage returns: two tabs of a test draw their own numbers, the same ones + * at replay. A popup is placed by its opener's document in the page itself + * (initScript). mark: the value the context's init script was given, by + * which it knows a page's first document (firstScript). Returns what undoes + * it, and whether a page is one the test opened, once every newPage the + * test has asked for has come. + */ +export function placePages(context: BrowserContext, mark: string): { restore: () => void; opened: (page: Page | null | undefined) => Promise } { + const had = Object.prototype.hasOwnProperty.call(context, 'newPage'); + const newPage = context.newPage; + let made = 0; + // The test's pages, and those it has asked for that have not come yet. + const pages = new WeakSet(); + const coming = new Set>(); + context.newPage = async function placed(this: BrowserContext): Promise { + // Its place is taken when the test asks for it, not when the page + // comes: two pages asked for at once keep the order the test asked in. + const id = `t${made++}`; + const asked = newPage.call(context); + coming.add(asked); + let p: Page; + try { + p = await asked; + } finally { + coming.delete(asked); + } + pages.add(p); + // The documents the page loads, and the one it has: an init script + // does not run in a document that already exists. + await p.addInitScript({ content: pageScript(id) }); + await p.evaluate(firstScript(id, mark)); + return p; + }; + return { + restore: () => { + if (had) context.newPage = newPage; + else delete (context as { newPage?: unknown }).newPage; + }, + // A page's first document may tell before its newPage has come. + opened: async (page) => { + const came = await Promise.allSettled([...coming]); + return pages.has(page as Page) || came.some((c) => c.status === 'fulfilled' && c.value === page); + }, + }; +} + +/** + * The init script that seeds a document: one stream for each of + * Math.random, crypto.getRandomValues and crypto.randomUUID, from the seed, + * the document's page, its path, and how many documents of that path + * loaded before it in its frame (for a top document, or a frame under a + * parent of another origin: its origin in the tab). + * + * The page keeps two tabs of a test from drawing the same numbers (a + * browser-made tab ID or idempotency key reaches a real server in a record), + * and is the same at replay: + * - a page the test opened is named by pageScript: its place in the order + * the test opened its pages. A top document with no session storage to + * count its documents in (an opaque origin) is never seeded, but for + * the page's first document, which firstScript marks: it draws the + * browser's own numbers and is named, since seeded, the next document + * at its path in the tab (a reload, another visit) could draw its + * numbers; + * - a popup is the window its opener's document opened with window.open, + * in the order that document opened them, read when the popup's + * document begins (or, for a document that took over its window's + * first about:blank and so ran no init script, at its first draw); it + * needs an opener of its own origin; + * - a frame of its parent's origin is in its parent document, at the + * path its element asks for and its number among the parent's frames + * that were put in at that path, by element, in the order the parent + * document holds them when it numbers them (as the task that put them + * in ends, or as a frame's document begins): frames at other paths, and + * the order the frames' documents load in, do not move it, and it keeps its place whatever it loads. An + * about:blank frame does not draw its parent document's numbers; + * - a frame under a parent of another origin, which it cannot read, is + * in the page the test opened, by its path and how many documents of + * that path its origin loaded in the tab before. One with no session + * storage to count them in (a sandboxed frame) is never seeded: it + * draws the browser's own numbers and is named, since seeded, its next + * document (a reload, or a frame put in again at its place) or a frame + * beside it could draw its numbers. + * A document with no place (a popup opened with noopener, as a link with + * target=_blank opens one, or of another origin than its opener's page, or + * after its opener's page moved on; a frame of another origin in one; a + * frame its parent cannot list: one in a closed shadow root, or in an + * object or embed; a frame whose document began after its removal) draws + * the browser's own numbers, never another page's, and says so through + * report, the name of a function the fixture exposed ('' for none), at its + * first draw, with its address, why ('frame' for a frame its parent + * cannot place, 'uncounted' for a frame under a parent of another origin + * with no session storage, or one in such a frame, 'nostorage' for a top + * document with no session storage, or a frame of its origin in one, else + * '') and, for 'uncounted', how many frames up from the document's own + * frame the frame with no session storage is (0 for its own), so the + * fixture can count the frames beside that one (othersNow). + * + * The path alone names the document in its frame: a replay may serve the + * app at another host or port than its record did, and a query often + * carries a time. A stream for each function keeps one's draws from + * shifting another's: a timer's Math.random does not move the next + * crypto.randomUUID. + * + * mark: the value by which firstScript marks a page's first document ('' for + * none, so no document is marked). + */ +export function initScript(seed: string, report = '', mark = ''): string { + return `(() => { + const seed = ${JSON.stringify(seed)}; + const mark = ${JSON.stringify(mark)}; + const g = globalThis; + const imul = Math.imul; + // cyrb128: a string's 128-bit hash, a stream's four words. + const hash = (s) => { + let h1 = 1779033703, h2 = 3144134277, h3 = 1013904242, h4 = 2773480762; + for (let i = 0; i < s.length; i++) { + const k = s.charCodeAt(i); + h1 = h2 ^ imul(h1 ^ k, 597399067); + h2 = h3 ^ imul(h2 ^ k, 2869860233); + h3 = h4 ^ imul(h3 ^ k, 951274213); + h4 = h1 ^ imul(h4 ^ k, 2716044179); + } + h1 = imul(h3 ^ (h1 >>> 18), 597399067); + h2 = imul(h4 ^ (h2 >>> 22), 2869860233); + h3 = imul(h1 ^ (h3 >>> 17), 951274213); + h4 = imul(h2 ^ (h4 >>> 19), 2716044179); + h1 ^= h2 ^ h3 ^ h4; h2 ^= h1; h3 ^= h1; h4 ^= h1; + return [h1 >>> 0, h2 >>> 0, h3 >>> 0, h4 >>> 0]; + }; + // sfc32 over those words: 32 random bits a call. + const stream = (s) => { + let [a, b, c, d] = hash(s); + const next = () => { + let t = (a + b) | 0; + a = b ^ (b >>> 9); + b = (c + (c << 3)) | 0; + c = (c << 21) | (c >>> 11); + d = (d + 1) | 0; + t = (t + d) | 0; + c = (c + t) | 0; + return t >>> 0; + }; + for (let i = 0; i < 15; i++) next(); + return next; + }; + // The fixture's function that hears of a document with no place, taken + // off the window so the page does not see it. + let tell = null; + ${report === '' ? '' : `try { const f = g[${JSON.stringify(report)}]; if (typeof f === 'function') tell = f; delete g[${JSON.stringify(report)}]; } catch (e) { tell = null; }`} + // Taken as the script begins: the page's own scripts run after it. + const WM = WeakMap; + const Mp = Map; + const U = g.URL; + const MO = g.MutationObserver; + // A frame's element, read through the window's own getter, taken now: the + // parent finds the frame by it, and never reads another frame's window. + let elementGetter = null; + try { + const d = Object.getOwnPropertyDescriptor(g, 'frameElement'); + if (d && typeof d.get === 'function') elementGetter = d.get; + } catch (e) { elementGetter = null; } + const frameElementOf = () => { try { return elementGetter !== null ? elementGetter.call(g) : g.frameElement; } catch (e) { return null; } }; + // The name pageScript gives the page the test opened, in each of its + // documents and frames (it runs after this script); null for none. + const named = () => { try { const v = g[${JSON.stringify(PAGE_KEY)}]; return typeof v === 'string' ? v : null; } catch (e) { return null; } }; + // The path a frame element asks its frame to load: the src's path (its + // host, port and query are not, as a document's are not), about:srcdoc, + // or about:blank for none. + // A src of another scheme names its scheme only (a blob: URL carries a + // browser-made ID); one keploy cannot read is ?. + const srcOf = (el, doc) => { + try { + if (el.hasAttribute('srcdoc')) return 'about:srcdoc'; + const a = el.getAttribute('src'); + if (a === null || a === '') return 'about:blank'; + const u = new U(a, doc.baseURI); + return u.protocol === 'http:' || u.protocol === 'https:' ? u.pathname : u.protocol; + } catch (e) { + return '?'; + } + }; + // The place a frame element takes at the path it asks for: how many + // frames of this document were put in at that path before it. + const next = (s, el) => { + const at = srcOf(el, s.doc); + const n = s.numbers.get(at) || 0; + s.numbers.set(at, n + 1); + return 'f' + at + '\\n' + n; + }; + const isFrame = (n) => { const t = n.localName; return t === 'iframe' || t === 'frame'; }; + // Numbers the frame elements of the document that have none yet, in the + // order of the elements in the document (an open shadow root's at its + // host). It reads elements alone, never a frame's window: reading one + // makes Chromium begin that frame's first document, whose init script + // would ask this document again, a frame deeper each time. + const number = (s) => { + const visit = (root) => { + let all = null; + try { all = root.querySelectorAll('*'); } catch (e) { return; } + for (let i = 0; i < all.length; i++) { + const el = all[i]; + if (isFrame(el) && !s.els.has(el)) { s.els.set(el, next(s, el)); s.held++; } + let r = null; + try { r = el.shadowRoot; } catch (e) { r = null; } + if (r) visit(r); + } + }; + visit(s.doc); + }; + // Whether a node the app put in or took out is a frame element, or holds + // one outside its shadow roots. A node with no element child (a text + // node, a leaf element) holds none, and costs no query. + const holds = (n) => { + try { return isFrame(n) || (n.firstElementChild != null && n.querySelector('iframe,frame') !== null); } catch (e) { return false; } + }; + // A frame element taken out loses its number: put back, it holds another + // window, and takes the next number at its path. + const drop = (s, el) => { if (s.els.delete(el)) s.held--; s.bound.delete(el); }; + const forget = (s, n) => { + try { + if (isFrame(n)) drop(s, n); + const all = n.querySelectorAll('iframe,frame'); + for (let i = 0; i < all.length; i++) drop(s, all[i]); + } catch (e) { + // A node with no elements. + } + }; + // What the app's changes to the document's nodes, in order, do to its + // frames: one taken out is forgotten (looked for only while the document + // holds a numbered frame), and the document numbers its frames only when + // one was put in (a change that puts in no frame costs no walk). + const take = (s, records) => { + let put = false; + for (let i = 0; i < records.length; i++) { + const r = records[i]; + if (s.held > 0) { + const out = r.removedNodes; + for (let j = 0; j < out.length; j++) if (holds(out[j])) forget(s, out[j]); + } + if (!put) { + const added = r.addedNodes; + for (let j = 0; j < added.length; j++) if (holds(added[j])) { put = true; break; } + } + } + if (put) number(s); + }; + // Numbers a document's frames as the app puts them in (at the end of the + // task that put them, or as a frame's document begins, whichever comes + // first), so that the order their documents load in (the network's) does + // not decide their numbers. The observer watches the document's childList + // outside its shadow roots: a frame put into a shadow root already in the + // page is numbered when the app next puts a frame in outside them, or when + // a document of one of the parent's frames, its own included, begins. + const watch = (s) => { + try { + const mo = new MO((records) => { take(s, records); }); + mo.observe(s.doc, { childList: true, subtree: true }); + s.mo = mo; + } catch (e) { + // No document or no observer: frames are numbered as their documents + // begin. + } + }; + // A document's state: its path; how many documents of that path loaded + // before it in its frame, or for a top document or a frame under a parent + // of another origin in its origin in the tab; what it is (a top document, + // a frame of its parent's origin, or one under a parent of another + // origin); a frame's parent and its place among the parent's frames; the + // place its opener gives a top document; the windows it opened; its own + // frames' places and loads; and its streams. Made as the document begins; + // made again at the next draw when the window has a new document that ran + // no init script: a popup's first document of its opener's origin takes + // over the window of its first about:blank. + const begin = (doc) => { + let place = ''; + try { place = location.pathname; } catch (e) { place = ''; } + // A frame's parent document, when keploy can read it (of the frame's + // origin): it places the frame and counts its loads. A frame under a + // parent of another origin cannot read it. + let kind = 'top'; + let up = null; + let frame = null; + let loads = 0; + // The parent as the document begins, before any script of the page has + // run: a page script may replace the global parent (var parent = ...). + try { + const p = g.parent; + if (p !== g) { + kind = 'frame'; + if (p !== null && p !== undefined) { + let d = null; + try { d = p[${JSON.stringify(DOC_KEY)}]; } catch (e) { kind = 'other'; } + if (d) { + const e = d.enter(frameElementOf(), g, place); + up = d; + frame = e[0]; + loads = e[1]; + } + } + } + } catch (e) { + // A parent keploy cannot ask: no place. + kind = 'frame'; + } + // Whether session storage counted this document's loads. + let counted = false; + if (kind !== 'frame') { + try { + const store = sessionStorage; + let seen = {}; + try { seen = JSON.parse(store.getItem(${JSON.stringify(LOADS_KEY)}) || '{}') || {}; } catch (e) { seen = {}; } + if (typeof seen !== 'object' || Array.isArray(seen)) seen = {}; + loads = typeof seen[place] === 'number' ? seen[place] : 0; + seen[place] = loads + 1; + store.setItem(${JSON.stringify(LOADS_KEY)}, JSON.stringify(seen)); + counted = true; + } catch (e) { + // No session storage here (a sandboxed frame, an opaque origin): every + // document of this path in the tab would draw the same numbers, so + // neither a frame under a parent of another origin nor a top + // document (but a page's first) is seeded (pageOf). + } + } + // A top document's opener, read as the document begins: an opener may + // move on later (a frame's is read too, and not used). + let given = null; + try { + const o = g.opener; + if (o) { + const d = o[${JSON.stringify(DOC_KEY)}]; + const i = d ? d.opened().indexOf(g) : -1; + const k = i >= 0 ? d.key() : null; + if (k !== null) given = k + 'w' + i; + } + } catch (e) { + // An opener of another origin. + given = null; + } + const s = { doc, place, loads, counted, kind, up, frame, given, page: undefined, first: false, uncounted: false, opened: [], els: new WM(), held: 0, bound: new WM(), mo: null, numbers: new Mp(), counts: new Mp(), streams: null }; + watch(s); + return s; + }; + const docNow = () => { try { return document; } catch (e) { return null; } }; + let state = begin(docNow()); + const now = () => { + const doc = docNow(); + if (state.doc !== doc) state = begin(doc); + return state; + }; + // The place frame element el of this document takes, for its window w, + // and how many documents at path w loaded before in that place: [place, + // loads]; [null, 0] when el is not among this document's frames (one in a + // closed shadow root, in an object or embed element, or taken out). The + // app's changes not yet seen are taken first, in order. An element whose + // number another window holds was put back where the observer does not + // see (in a shadow root): it takes the next number at its path. + const enter = (el, w, path) => { + const s = now(); + if (s.mo !== null) take(s, s.mo.takeRecords()); + if (!s.els.has(el)) number(s); + if (!s.els.has(el)) return [null, 0]; + const had = s.bound.get(el); + if (had !== undefined && had !== w) s.els.set(el, next(s, el)); + s.bound.set(el, w); + const f = s.els.get(el); + const c = f + '\\n' + path; + const n = s.counts.get(c) || 0; + s.counts.set(c, n + 1); + return [f, n]; + }; + // The document's page, when first asked: for a top document, the page + // the test opened (the page script runs after this one, before the + // page's own scripts), or the place its opener gives it; for a frame of + // its parent's origin, its parent document's key and its place among + // that document's frames; for a frame under a parent of another origin, + // the page the test opened, marked as such a frame. null for none, and + // for a frame under a parent of another origin whose loads session + // storage did not count: nothing tells its documents at one path apart + // (a reload, a frame put in again at its place, another frame beside it), + // and seeded they would draw the same numbers (a browser-made ID sent + // twice to a real server in a record). + const pageOf = () => { + const s = now(); + if (s.page === undefined) { + let p = null; + if (s.kind === 'top') { + p = named(); + if (p === null) p = s.given; + // A top document whose loads session storage did not count (an + // opaque origin) is not seeded: nothing tells it apart from the next + // document at its path in the tab (a reload, another visit, a popup + // opened again), and seeded they would draw the same numbers. A + // page's first document is the one of its kind in its tab. + if (p !== null && !s.counted && !s.first) { s.uncounted = true; p = null; } + } else if (s.kind === 'other') { + const t = named(); + if (t !== null && !s.counted) s.uncounted = true; + p = t === null || s.uncounted ? null : t + 'x'; + } else if (s.frame !== null) { + const k = s.up.key(); + p = k === null ? null : k + s.frame; + } + s.page = p; + } + return s.page; + }; + // Why a document with no page has none: 'frame' for a frame its parent + // cannot place, 'uncounted' for a frame under a parent of another origin + // with no session storage, or one in such a frame, 'nostorage' for a top + // document with no session storage, or a frame of its origin in one, else + // ''. aboveFor: for 'uncounted', how many frames up from the document's + // own frame that frame with no session storage is (0 for its own): the + // fixture counts the frames beside it under its parent (othersNow). + const whyOf = () => { + const s = now(); + if (s.kind === 'top') return s.uncounted ? 'nostorage' : ''; + if (s.kind === 'other') return s.uncounted ? 'uncounted' : ''; + if (s.kind !== 'frame') return ''; + return s.frame === null ? 'frame' : s.up.why(); + }; + const aboveFor = () => { + const s = now(); + return s.kind === 'frame' && s.frame !== null ? s.up.above() + 1 : 0; + }; + const key = () => { const p = pageOf(); const s = now(); return p === null ? null : seed + '\\n' + p + '\\n' + s.place + '\\n' + s.loads + '\\n'; }; + let mine; + try { + // first: firstScript, in a page's first document, marks it as such + // (its state, so a later document that takes over the window is not), + // with the mark the fixture gave this script, which a page's own script + // does not know. + const d = Object.freeze({ key, why: whyOf, above: aboveFor, enter, opened: () => now().opened, first: (m) => { if (mark !== '' && m === mark) now().first = true; } }); + Object.defineProperty(g, ${JSON.stringify(DOC_KEY)}, { value: d, enumerable: false, writable: false, configurable: true }); + mine = d; + } catch (e) { + // A window that takes no property: nothing it opens or holds has a place. + } + // Whether a later run of this script in the window has taken over: WebKit + // runs it again in a document that takes over its window's first + // about:blank, over this run's functions. This run then only passes on + // the browser's own, so the document begins once. + const stale = () => { try { return g[${JSON.stringify(DOC_KEY)}] !== mine; } catch (e) { return false; } }; + // The document's stream for name, made at its first draw; null when it + // has no place, so it draws the browser's own numbers, and says so once. + const from = (name) => { + if (stale()) return null; + const s = now(); + if (s.streams === null) { + const k = key(); + s.streams = k === null ? false : { + random: stream(k + 'Math.random'), + grv: stream(k + 'crypto.getRandomValues'), + uuid: stream(k + 'crypto.randomUUID'), + }; + if (s.streams === false && tell !== null) { + // Its address: origin and path, or for an opaque origin (about:blank, + // about:srcdoc, a data: URL, a sandboxed frame or page) its URL, with + // no query or fragment. A window's first document has no URL yet + // while Firefox runs its init scripts: it is about:blank. Why: whyOf. + let at = ''; + try { + const o = location.origin; + at = o && o !== 'null' ? o + location.pathname : String(location.href).split(/[?#]/)[0]; + } catch (e) { at = ''; } + if (at === '') at = 'about:blank'; + const why = whyOf(); + try { const r = tell(at, why, aboveFor()); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} + } + } + return s.streams === false ? null : s.streams[name]; + }; + const define = (o, name, value) => { + const d = Object.getOwnPropertyDescriptor(o, name); + Object.defineProperty(o, name, { value, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); + }; + const fill = (next, u) => { + for (let i = 0; i < u.length; i += 4) { + let w = next(); + for (let j = 0; j < 4 && i + j < u.length; j++) { u[i + j] = w & 255; w >>>= 8; } + } + }; + const realOpen = g.open; + if (typeof realOpen === 'function') { + define(g, 'open', { open() { + const w = realOpen.apply(this, arguments); + try { if (!stale()) { const opened = now().opened; if (w && opened.indexOf(w) < 0) opened.push(w); } } catch (e) {} + return w; + } }.open); + } + const realRandom = Math.random; + // 53 bits, as the browser's own Math.random gives. + define(Math, 'random', { random() { + const rnd = from('random'); + if (rnd === null) return realRandom(); + return ((rnd() >>> 5) * 67108864 + (rnd() >>> 6)) / 9007199254740992; + } }.random); + const C = typeof Crypto === 'function' ? Crypto.prototype : null; + if (C && typeof C.getRandomValues === 'function') { + const real = C.getRandomValues; + define(C, 'getRandomValues', { getRandomValues(array) { + // The browser's own call first: it refuses what the browser refuses + // (a float array, more than 65536 bytes, a this that is no Crypto). + const out = real.call(this, array); + const grv = from('grv'); + if (grv !== null) fill(grv, new Uint8Array(out.buffer, out.byteOffset, out.byteLength)); + return out; + } }.getRandomValues); + } + if (C && typeof C.randomUUID === 'function') { + const realUUID = C.randomUUID; + define(C, 'randomUUID', { randomUUID() { + const own = realUUID.call(this); + const uuid = from('uuid'); + if (uuid === null) return own; + const b = new Uint8Array(16); + fill(uuid, b); + b[6] = (b[6] & 15) | 64; + b[8] = (b[8] & 63) | 128; + let h = ''; + for (let i = 0; i < 16; i++) { + h += (b[i] + 256).toString(16).slice(1); + if (i === 3 || i === 5 || i === 7 || i === 9) h += '-'; + } + return h; + } }.randomUUID); + } +})(); +`; +} diff --git a/playwright/test/draws.mjs b/playwright/test/draws.mjs new file mode 100644 index 0000000..1cae0a3 --- /dev/null +++ b/playwright/test/draws.mjs @@ -0,0 +1,67 @@ +// The test browser's seeding run in a Node vm context in place of a page: +// a fake Crypto (over Node's own randomness), a location and a session +// storage. Shared by the unit rows and the browser rows, which compare the +// browser's draws with these. +import vm from 'node:vm'; +import { webcrypto } from 'node:crypto'; +import { pageScript } from '../dist/random.js'; + +// The draws the rows' pages make at load (test/fixtures/seeded.spec.ts's +// page server sends the same expression). +export const DRAWS = '({ random: [Math.random(), Math.random()], bytes: Array.from(crypto.getRandomValues(new Uint8Array(8))), uuid: crypto.randomUUID() })'; +// The same without crypto.randomUUID, for about:blank, which is no secure +// context (test/fixtures/first.spec.ts draws it). +export const BLANK_DRAWS = '({ random: [Math.random(), Math.random()], bytes: Array.from(crypto.getRandomValues(new Uint8Array(8))) })'; + +/** + * A document at path whose init script is script. storage is the tab's + * session storage (a Map), shared by the documents given the same one; + * null: a document with no session storage (reading it throws). page is the + * page the fixture named it in (pageScript, run after script as the browser + * runs a page's init scripts after its context's); null: a document no + * page script names (a popup with no opener). told collects what the + * document tells the fixture's function named report: its address, or + * [address, why] when it says why ([address, why, above] for 'uncounted'). + * origin and href are its location's. + * parent is the window's parent: the window itself (a top window) when + * undefined; null for a removed frame; OTHER for a parent of another origin + * (reading it throws); or an object, a parent of the frame's origin. frames: + * how many frames a parent of another origin says it holds (its length, + * which another origin may read, and the seeding does not); undefined: + * reading it throws too. element + * is the window's frameElement, read through a getter. doc is its document + * (with URL beside it), for a document whose frames the row makes. opener + * is the window's opener (none when undefined). + */ +export const OTHER = Symbol('a parent of another origin'); +export function documentAt(script, path = '/p', storage = new Map(), page = 't0', { report = '', told = [], origin = 'http://h', href = undefined, parent = undefined, doc = undefined, frames = undefined, element = undefined, opener = undefined } = {}) { + // As the browser's: a this that is no Crypto is refused. + class Crypto { + getRandomValues(a) { if (!(this instanceof Crypto)) throw new TypeError('Illegal invocation'); return webcrypto.getRandomValues(a); } + randomUUID() { if (!(this instanceof Crypto)) throw new TypeError('Illegal invocation'); return webcrypto.randomUUID(); } + } + const ctx = { location: { origin, pathname: path, href: href ?? `${origin}${path}` }, Crypto, crypto: new Crypto() }; + if (storage === null) { + Object.defineProperty(ctx, 'sessionStorage', { get() { throw new Error('SecurityError: no session storage here'); } }); + } else { + ctx.sessionStorage = { getItem: (k) => (storage.has(k) ? storage.get(k) : null), setItem: (k, v) => { storage.set(k, String(v)); } }; + } + if (report) ctx[report] = (at, why, above) => { told.push(why === 'uncounted' ? [at, why, above] : why ? [at, why] : at); return Promise.resolve(); }; + if (element !== undefined) Object.defineProperty(ctx, 'frameElement', { get() { return element; }, configurable: true, enumerable: true }); + if (doc !== undefined) { ctx.document = doc; ctx.URL = URL; } + if (opener !== undefined) ctx.opener = opener; + vm.createContext(ctx); + // A top window: its own parent and top. + vm.runInContext('globalThis.parent = globalThis; globalThis.top = globalThis;', ctx); + if (parent !== undefined) { + ctx.parent = parent === OTHER ? new Proxy({}, { + get(t, k) { + if (k === 'length' && frames !== undefined) return frames; + throw new Error('SecurityError: a parent of another origin'); + }, + }) : parent; + } + if (script) vm.runInContext(script, ctx); + if (page !== null) vm.runInContext(pageScript(page), ctx); + return { ctx, run: (code) => vm.runInContext(code, ctx), draws: () => JSON.parse(JSON.stringify(vm.runInContext(DRAWS, ctx))) }; +} diff --git a/playwright/test/fixtures/engines.spec.ts b/playwright/test/fixtures/engines.spec.ts new file mode 100644 index 0000000..4dfab0d --- /dev/null +++ b/playwright/test/fixtures/engines.spec.ts @@ -0,0 +1,113 @@ +import fs from 'node:fs'; +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). Frames of a page's +// first document: two srcdoc (the first holding a frame of its own), one +// about:blank with no src, then one the app puts where that one was after +// removing it; a popup the first document opens with +// window.open('about:blank'); the test's own context init script, which +// draws in each document before keploy names the page; a frame in an open +// shadow root and one in a closed one; a frame's Math.random drawn from +// once the frame is removed; and top documents with no session storage. +// Each test keeps what its pages drew. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +// test/draws.mjs's BLANK_DRAWS: about:blank is no secure context, so it has +// no crypto.randomUUID. +const BLANK_DRAWS = '({ random: [Math.random(), Math.random()], bytes: Array.from(crypto.getRandomValues(new Uint8Array(8))) })'; + +const frame = (p: Page, name: string) => { + const f = p.frame(name); + if (!f) throw new Error(`no frame ${name} among ${p.frames().map((x) => x.name()).join(', ')}`); + return f; +}; + +test('frames', async ({ page }) => { + // n holds a frame of its own, nn. + await page.setContent(''); + await page.waitForLoadState('load'); + const drew: Record = { top: await page.evaluate(BLANK_DRAWS) }; + for (const n of ['n', 'nn', 's', 'b']) drew[n] = { url: frame(page, n).url(), draws: await frame(page, n).evaluate(BLANK_DRAWS) }; + // The frame at b's index is removed, and another put in its place. + await page.evaluate(() => { + document.querySelector('iframe[name="b"]')!.remove(); + const c = document.createElement('iframe'); + c.name = 'c'; + document.body.appendChild(c); + }); + drew.c = await frame(page, 'c').evaluate(BLANK_DRAWS); + keep('frames', drew); +}); + +test('popupfromblank', async ({ page, context }) => { + const [p] = await Promise.all([context.waitForEvent('page'), page.evaluate(() => { window.open('about:blank'); })]); + keep('popupfromblank', { url: p.url(), draws: await p.evaluate(BLANK_DRAWS) }); +}); + +test('ownctxinit', async ({ context }) => { + await context.addInitScript('window.__early = Math.random();'); + const p = await context.newPage(); + const early = await p.evaluate(() => (window as unknown as { __early: number }).__early); + // A popup its first document opens: at about:blank too, with another + // reason (its opener has no place). + const [q] = await Promise.all([context.waitForEvent('page'), p.evaluate(() => { window.open('about:blank'); })]); + await q.waitForLoadState('load'); + await p.goto(`${base}/draws`); + keep('ownctxinit', { early, url: p.url() }); +}); + +test('shadow', async ({ page }) => { + // A frame inside an open shadow root, and one inside a closed one: no + // engine lists either among its parent's frames. + await page.setContent('
'); + await page.evaluate(() => { + for (const [id, name, mode] of [['h', 'w', 'open'], ['k', 'v', 'closed']] as const) { + const f = document.createElement('iframe'); + f.name = name; + document.getElementById(id)!.attachShadow({ mode }).appendChild(f); + } + }); + const f = frame(page, 'w'); + const index = await f.evaluate(() => Array.from({ length: parent.length }, (_, j) => parent[j]).indexOf(window)); + keep('shadow', { index, open: await f.evaluate(BLANK_DRAWS), closed: await frame(page, 'v').evaluate(BLANK_DRAWS) }); +}); + +test('nostorage', async ({ page }) => { + // The page's first document draws; then, after a page of its origin, + // about:blank loaded twice, a srcdoc frame of its origin in the first, a page + // served with a CSP sandbox (/csp) loaded twice, and a data: URL loaded + // twice: none has session storage. + const drew: Record = { first: await page.evaluate(BLANK_DRAWS), blank: [], csp: [], data: [] }; + await page.goto(`${base}/fa`); + for (let i = 0; i < 2; i++) { + await page.goto('about:blank'); + (drew.blank as unknown[]).push(await page.evaluate(BLANK_DRAWS)); + if (i === 0) { + await page.evaluate(() => new Promise((done) => { const f = document.createElement('iframe'); f.name = 'in'; f.srcdoc = '

in

'; f.onload = done; document.body.appendChild(f); })); + drew.inner = await frame(page, 'in').evaluate(BLANK_DRAWS); + } + } + for (let i = 0; i < 2; i++) { + await page.goto(`${base}/csp`); + (drew.csp as unknown[]).push(await page.evaluate(() => (window as unknown as { __draws: unknown }).__draws)); + } + for (let i = 0; i < 2; i++) { + await page.goto('data:text/html,d'); + (drew.data as unknown[]).push(await page.evaluate(BLANK_DRAWS)); + } + keep('nostorage', drew); +}); + +test('removed', async ({ page }) => { + // Functions taken from a frame, then drawn from once it is removed. + await page.setContent('

x

'); + const drew = await page.evaluate(() => { + const f = document.createElement('iframe'); + document.body.appendChild(f); + const r = (f.contentWindow as unknown as { Math: Math }).Math.random; + f.remove(); + return [r(), r()]; + }); + keep('removed', drew); +}); diff --git a/playwright/test/fixtures/first.spec.ts b/playwright/test/fixtures/first.spec.ts new file mode 100644 index 0000000..b22ad5f --- /dev/null +++ b/playwright/test/fixtures/first.spec.ts @@ -0,0 +1,34 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +// A page's first document: in the page fixture's page, a draw before its +// first goto, then the page's own script in what setContent writes into that +// document; in one context.newPage() makes, setContent at once, then a draw. And a popup opened with noopener +// at an overlong address (DRAWS_BASE's /long/...), which keploy cannot place. +// Each test keeps what its pages drew. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +// test/draws.mjs's BLANK_DRAWS: about:blank is no secure context, so it has +// no crypto.randomUUID. +const BLANK_DRAWS = '({ random: [Math.random(), Math.random()], bytes: Array.from(crypto.getRandomValues(new Uint8Array(8))) })'; + +test('blank', async ({ page, context }) => { + const set = ``; + const read = (): unknown => (window as unknown as { __set: unknown }).__set; + // Written by setContent as soon as newPage returns, then a draw after. + const other = await context.newPage(); + await other.setContent(set); + const drew: Record = { other: { url: other.url(), set: await other.evaluate(read), after: await other.evaluate(BLANK_DRAWS) } }; + // A draw before the first goto, then setContent. + const first = await page.evaluate(BLANK_DRAWS); + await page.setContent(set); + drew.page = { url: page.url(), first, set: await page.evaluate(read) }; + keep('blank', drew); +}); + +test('longpath', async ({ page, context }) => { + await page.goto(`${base}/draws`); + const [p] = await Promise.all([context.waitForEvent('page'), page.evaluate((u) => { window.open(u, '_blank', 'noopener'); }, `${base}/long/${'x'.repeat(600)}`)]); + await p.waitForLoadState('load'); + keep('longpath', { url: p.url() }); +}); diff --git a/playwright/test/fixtures/many.spec.ts b/playwright/test/fixtures/many.spec.ts new file mode 100644 index 0000000..4857c95 --- /dev/null +++ b/playwright/test/fixtures/many.spec.ts @@ -0,0 +1,177 @@ +import fs from 'node:fs'; +import type { Frame, Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). A page that holds many +// frames of its origin, in its markup or put in by one script; sandboxed +// frames, which have no session storage (some whose own script replaces the +// global parent, some holding a srcdoc frame); and what seeding costs a page +// whose DOM changes often. Each test keeps what it saw. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +const read = (): unknown => (window as unknown as { __draws: unknown }).__draws; + +// /many's frames m0..m(k-1), once each has drawn: null for one that has +// not, or whose element holds no window. +async function frames(p: Page, k: number): Promise { + const look = (n: number): unknown[] => Array.from({ length: n }, (_, i) => { + const w = (document.getElementsByName(`m${i}`)[0] as HTMLIFrameElement | undefined)?.contentWindow as unknown as { __draws?: unknown } | null; + return w && w.__draws !== undefined ? JSON.parse(JSON.stringify(w.__draws)) : null; + }); + let got: unknown[] = []; + for (let i = 0; i < 1200; i++) { + got = await p.evaluate(look, k); + if (got.every((x) => x !== null)) return got; + await p.waitForTimeout(50); + } + return got; +} + +for (const how of ['markup', 'script']) { + test(how, async ({ page }) => { + const errors: string[] = []; + page.on('pageerror', (e) => errors.push(String(e))); + await page.goto(`${base}/many?k=300&how=${how}`); + const drew = await frames(page, 300); + keep(how, { drew, errors }); + }); +} + +async function loaded(p: Page, n: string, path: string): Promise { + for (let i = 0; i < 400; i++) { + const f = p.frame(n); + if (f && new URL(f.url(), 'http://x').pathname === path) { + try { if (await f.evaluate(() => (window as unknown as { __draws?: unknown }).__draws !== undefined)) return f; } catch { /* not yet */ } + } + await p.waitForTimeout(25); + } + throw new Error(`no frame ${n} at ${path}`); +} + +test('sandboxed', async ({ page }) => { + // Three sandboxed frames at one path, then a page with one. + await page.goto(`${base}/sbx?n=3`); + const three: Record = {}; + for (const n of ['s0', 's1', 's2']) three[n] = await (await loaded(page, n, '/fs')).evaluate(read); + await page.goto(`${base}/sbx?n=1`); + keep('sandboxed', { three, one: await (await loaded(page, 's0', '/fs')).evaluate(read) }); +}); + +test('lone', async ({ page }) => { + // A lone sandboxed frame, then taken out, and another put in at its path. + await page.goto(`${base}/sbx?n=1`); + const first = await (await loaded(page, 's0', '/fs')).evaluate(read); + await page.evaluate(() => { + document.getElementsByName('s0')[0].remove(); + const f = document.createElement('iframe'); + f.name = 'r0'; + f.setAttribute('sandbox', 'allow-scripts'); + f.src = '/fs'; + document.body.appendChild(f); + }); + keep('lone', { first, again: await (await loaded(page, 'r0', '/fs')).evaluate(read) }); +}); + +test('swapped', async ({ page }) => { + // Nine sandboxed frames, three at each of /fsv, /fss and /fso, whose own + // script replaces the global parent before it draws. + await page.goto(`${base}/sbx?n=3&swap=1`); + const drew: Record = {}; + for (const [k, path] of [['v', '/fsv'], ['s', '/fss'], ['o', '/fso']]) { + for (let i = 0; i < 3; i++) drew[`${k}${i}`] = await (await loaded(page, `${k}${i}`, path)).evaluate(read); + } + keep('swapped', drew); +}); + +test('held', async ({ page }) => { + // Three sandboxed frames at /fh: the parent's parser puts in the second + // and third only once the first has drawn and loaded. + await page.goto(`${base}/held`); + const drew: Record = {}; + for (const n of ['h0', 'h1', 'h2']) drew[n] = await (await loaded(page, n, '/fh')).evaluate(read); + keep('held', drew); +}); + +test('nested', async ({ page }) => { + // Two sandboxed frames at /fsn, each holding a srcdoc frame of its origin. + await page.goto(`${base}/sbx?n=2&nest=1`); + const drew: Record = {}; + for (const n of ['s0', 's1']) { + const f = await loaded(page, n, '/fsn'); + let child: unknown; + for (let i = 0; i < 400 && child === undefined; i++) { + const c = f.childFrames()[0]; + if (c) { try { child = await c.evaluate(() => (window as unknown as { __draws?: unknown }).__draws); } catch { /* not yet */ } } + if (child === undefined) await page.waitForTimeout(25); + } + if (child === undefined) throw new Error(`no srcdoc frame drew in ${n}`); + drew[n] = { own: await f.evaluate(read), child }; + } + keep('nested', drew); +}); + +// A workload in the page: 'grow' adds 20000 rows in 500 tasks; 'churn' +// adds and removes one span in each of 300 tasks, on a page of 20000 +// elements; 'shadow' does the same on a page of 2000 open shadow roots of +// 10 elements each. It answers its time in ms, and how many times the +// seeding walked the document (a querySelectorAll('*') on it) during the +// workload, and then after one frame is put in. +const work = (kind: string): Promise<{ ms: number; walks: number; framed: number }> => new Promise((res) => { + let walks = 0; + const all = Document.prototype.querySelectorAll; + Document.prototype.querySelectorAll = function (this: Document, q: string) { + if (q === '*' && this === document) walks++; + return all.call(this, q); + } as typeof all; + const box = document.getElementById('box')!; + const mc = new MessageChannel(); + let i = 0; + let last: Element | null = null; + const T = kind === 'grow' ? 500 : 300; + const t0 = performance.now(); + mc.port1.onmessage = () => { + if (kind === 'grow') { + for (let k = 0; k < 40; k++) { const d = document.createElement('div'); d.textContent = 'r'; box.appendChild(d); } + } else { + if (last) last.remove(); + last = document.createElement('span'); + box.appendChild(last); + } + if (++i < T) { mc.port2.postMessage(0); return; } + setTimeout(() => { + const ms = performance.now() - t0; + const during = walks; + box.appendChild(document.createElement('iframe')); + setTimeout(() => { Document.prototype.querySelectorAll = all; res({ ms, walks: during, framed: walks - during }); }, 0); + }, 0); + }; + mc.port2.postMessage(0); +}); + +test('cost', async ({ page, browser }) => { + // The same workloads in the test's page (seeded) and in a page of a + // context the test made itself (not seeded), taken in turn; the least + // time of two runs each, and the walks of the seeded page's runs. + // WebKit takes 3 to 6 s for one 'grow' run, seeded or not, so on a + // busy machine the twelve runs (four of them 'grow') can take more + // than the 30 s a test gets by default. + test.setTimeout(120_000); + const own = await browser.newContext(); + try { + const mine = await own.newPage(); + const out: Record = {}; + for (const [kind, at] of [['grow', '/cost?n=0'], ['churn', '/cost?n=20000'], ['shadow', '/cost?shadow=2000']]) { + const t = { seeded: Infinity, own: Infinity, walks: [] as number[], framed: [] as number[], ownWalks: [] as number[] }; + for (let r = 0; r < 2; r++) { + for (const [which, p] of [['seeded', page], ['own', mine]] as const) { + await p.goto(`${base}${at}`); + const got = await p.evaluate(work, kind); + t[which] = Math.min(t[which], got.ms); + if (which === 'seeded') { t.walks.push(got.walks); t.framed.push(got.framed); } else t.ownWalks.push(got.walks + got.framed); + } + } + out[kind] = t; + } + keep('cost', out); + } finally { await own.close(); } +}); diff --git a/playwright/test/fixtures/parse.spec.ts b/playwright/test/fixtures/parse.spec.ts new file mode 100644 index 0000000..2001e4c --- /dev/null +++ b/playwright/test/fixtures/parse.spec.ts @@ -0,0 +1,38 @@ +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). Two sandboxed frames at +// /fs in a page whose parser never ends (it waits on a script the server +// never answers: unparsed) or whose load never comes (an image the server +// never answers: unloaded), and a third the test puts in once both have +// drawn. The test then ends, and its context waits for that call until the +// test times out. A 1 ms navigation timeout, so that the fixture's wait for +// the parse's end holds only by its own timeout of 0 (none). +const base = process.env.DRAWS_BASE ?? ''; +test.use({ navigationTimeout: 1 }); + +async function drawn(p: Page, n: string): Promise { + for (let i = 0; i < 400; i++) { + const f = p.frame(n); + if (f) { + try { if (await f.evaluate(() => (window as unknown as { __draws?: unknown }).__draws !== undefined)) return; } catch { /* not yet */ } + } + await p.waitForTimeout(25); + } + throw new Error(`frame ${n} drew nothing`); +} + +for (const [name, waitUntil] of [['unparsed', 'commit'], ['unloaded', 'domcontentloaded']] as const) { + test(name, async ({ page }) => { + await page.goto(`${base}/${name}`, { waitUntil, timeout: 0 }); + for (const n of ['u0', 'u1']) await drawn(page, n); + await page.evaluate(() => { + const f = document.createElement('iframe'); + f.name = 'u2'; + f.setAttribute('sandbox', 'allow-scripts'); + f.src = '/fs'; + document.body.appendChild(f); + }); + await drawn(page, 'u2'); + }); +} diff --git a/playwright/test/fixtures/places.spec.ts b/playwright/test/fixtures/places.spec.ts new file mode 100644 index 0000000..48cef09 --- /dev/null +++ b/playwright/test/fixtures/places.spec.ts @@ -0,0 +1,99 @@ +import fs from 'node:fs'; +import type { Frame, Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). Frames of a page that +// come, or load, in an order a replay may change: /host puts in a frame of +// its own origin (a, /fa) and two cross-site ones (c, /fc and w, /fw) in +// the order PLACES_ORDER names, as third-party scripts that load in network +// order put theirs in; /race holds two frames at one path, and the server +// answers the one PLACES_SLOW names late; /before puts a frame in before +// another at its path whose document is still loading; /set is a frameset. +// Each test keeps what its frames drew as they loaded. +const base = process.env.DRAWS_BASE ?? ''; +const order = process.env.PLACES_ORDER ?? 'acw'; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +const read = (): unknown => (window as unknown as { __draws: unknown }).__draws; + +// The frame named n once its document at path has drawn. +async function loaded(p: Page, n: string, path: string): Promise { + for (let i = 0; i < 400; i++) { + const f = p.frame(n); + if (f && new URL(f.url(), 'http://x').pathname === path) { + try { if (await f.evaluate(() => (window as unknown as { __draws?: unknown }).__draws !== undefined)) return f; } catch { /* not yet */ } + } + await p.waitForTimeout(25); + } + throw new Error(`no frame ${n} at ${path} among ${p.frames().map((x) => `${x.name()} ${x.url()}`).join(', ')}`); +} +const drawsOf = async (p: Page, n: string, path: string): Promise => (await loaded(p, n, path)).evaluate(read); + +test('order', async ({ page }) => { + await page.goto(`${base}/host?order=${order}`); + const out: Record = { top: await page.evaluate(read) }; + for (const [n, at] of [['a', '/fa'], ['c', '/fc'], ['w', '/fw']]) out[n] = await drawsOf(page, n, at); + keep('order', out); +}); + +test('race', async ({ page }) => { + await page.goto(`${base}/race?slow=${process.env.PLACES_SLOW ?? '1'}`); + const drew = { r0: await drawsOf(page, 'r0', '/fr'), r1: await drawsOf(page, 'r1', '/fr') }; + // Both taken out and put back in one task: each element holds another + // window, and their documents load in the server's order again. + await page.evaluate(() => { + const fs = ['r0', 'r1'].map((n) => document.querySelector(`iframe[name="${n}"]`)!); + for (const f of fs) f.remove(); + for (const f of fs) document.body.appendChild(f); + }); + keep('race', { ...drew, again0: await drawsOf(page, 'r0', '/fr'), again1: await drawsOf(page, 'r1', '/fr') }); +}); + +test('remount', async ({ page }) => { + // A frame of the page's origin (it has session storage) removed, and + // another put in at the same path and index. + await page.goto(`${base}/host?order=a`); + const first = await drawsOf(page, 'a', '/fa'); + await page.evaluate(() => { + document.querySelector('iframe[name="a"]')!.remove(); + const f = document.createElement('iframe'); + f.name = 'a2'; + f.src = '/fa'; + document.body.appendChild(f); + }); + keep('remount', { first, again: await drawsOf(page, 'a2', '/fa') }); +}); + +test('nested', async ({ page }) => { + // x and n at the top; inside n (the top's second frame), y, z and nn + // (n's third). + await page.goto(`${base}/nested`); + const out: Record = {}; + for (const [n, at] of [['x', '/fa'], ['n', '/fn'], ['y', '/fb'], ['z', '/fb'], ['nn', '/fa']]) out[n] = await drawsOf(page, n, at); + keep('nested', out); +}); + +test('navigate', async ({ page }) => { + // One frame loads /fa, then /fb, then /fa again: its element still asks + // for /fa. + await page.goto(`${base}/host?order=a`); + const drew = [await drawsOf(page, 'a', '/fa')]; + for (const at of ['/fb', '/fa']) { + await page.frame('a')!.goto(`${base}${at}`); + drew.push(await drawsOf(page, 'a', at)); + } + keep('navigate', drew); +}); + +for (const [name, wrap] of [['before', '0'], ['beforewrapped', '1']]) { + test(name, async ({ page }) => { + // b0 put in (in a div for beforewrapped), its document answered late; + // b1 put in before it 50 ms later, its document answered at once. + await page.goto(`${base}/before?wrap=${wrap}`); + keep(name, { b0: await drawsOf(page, 'b0', '/fr'), b1: await drawsOf(page, 'b1', '/fr') }); + }); +} + +test('frameset', async ({ page }) => { + await page.goto(`${base}/set`); + keep('frameset', { f0: await drawsOf(page, 'f0', '/fa'), f1: await drawsOf(page, 'f1', '/fa') }); +}); diff --git a/playwright/test/fixtures/playwright.config.ts b/playwright/test/fixtures/playwright.config.ts index 45f46a7..9fc9926 100644 --- a/playwright/test/fixtures/playwright.config.ts +++ b/playwright/test/fixtures/playwright.config.ts @@ -1,7 +1,8 @@ import { defineConfig } from '@playwright/test'; +import type { KeployOptions } from '../../dist'; // The rows' Playwright project: one worker, no retries, as keploy runs a suite. -export default defineConfig({ +export default defineConfig({ testDir: '.', workers: 1, retries: 0, @@ -9,10 +10,14 @@ export default defineConfig({ reporter: [['json'], ['./steps-reporter.cjs']], outputDir: process.env.PW_OUTPUT_DIR || 'test-results', // PW_UNNAMED=1: one unnamed project, Playwright's default when a config - // lists none. PW_BROWSER=firefox or webkit runs the rows in that browser - // (Chromium, Playwright's default, otherwise). + // lists none. PW_NOSEED=1: the project turns seeding off. PW_BROWSER: the + // engine (chromium, the default, firefox or webkit). projects: [{ name: process.env.PW_UNNAMED ? '' : 'p1', - use: { headless: true, browserName: (process.env.PW_BROWSER || 'chromium') as 'chromium' | 'firefox' | 'webkit' }, + use: { + headless: true, + ...(process.env.PW_BROWSER ? { browserName: process.env.PW_BROWSER as 'chromium' | 'firefox' | 'webkit' } : {}), + ...(process.env.PW_NOSEED ? { keploySeedRandom: false } : {}), + }, }], }); diff --git a/playwright/test/fixtures/seeded.spec.ts b/playwright/test/fixtures/seeded.spec.ts new file mode 100644 index 0000000..449e3a7 --- /dev/null +++ b/playwright/test/fixtures/seeded.spec.ts @@ -0,0 +1,40 @@ +import fs from 'node:fs'; +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// DRAWS_BASE serves /draws, a page whose first script draws random numbers as +// it loads, and which holds a cross-site frame (/frame on another host of +// the same server) that does the same. Each test keeps what its pages drew. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +async function load(page: Page, query = ''): Promise<{ top: unknown; frame: unknown }> { + await page.goto(`${base}/draws${query}`); + const frame = page.frames().find((f) => f.url().endsWith('/frame')); + if (!frame) throw new Error(`no /frame among ${page.frames().map((f) => f.url()).join(', ')}`); + const read = (): unknown => (window as unknown as { __draws: unknown }).__draws; + return { top: await page.evaluate(read), frame: await frame.evaluate(read) }; +} + +test('one', async ({ page }) => { + keep('one', await load(page)); + // Each seeded function, called with a this that is no Crypto. + keep('this', await page.evaluate(() => { + const thrown = (f: () => unknown): string => { try { f(); return 'no error'; } catch (e) { return String(e); } }; + return { + grv: thrown(() => crypto.getRandomValues.call({}, new Uint8Array(4))), + uuid: thrown(() => crypto.randomUUID.call({})), + drew: crypto.getRandomValues(new Uint8Array(4)).some((x) => x !== 0) || crypto.randomUUID().length === 36, + }; + })); +}); +test('two', async ({ page }) => { keep('two', await load(page)); }); +test('again', async ({ page }) => { + // The same page loaded again, then with a query: the path and its count + // name the document, the query does not. + keep('again', [await load(page), await load(page), await load(page, '?t=' + Date.now())]); +}); +test.describe('own', () => { + test.use({ keploySeedRandom: false }); + test('three', async ({ page }) => { keep('three', await load(page)); }); +}); diff --git a/playwright/test/fixtures/tabs.spec.ts b/playwright/test/fixtures/tabs.spec.ts new file mode 100644 index 0000000..810bfc7 --- /dev/null +++ b/playwright/test/fixtures/tabs.spec.ts @@ -0,0 +1,61 @@ +import fs from 'node:fs'; +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Pages of one test at one path (DRAWS_BASE's /draws, with its cross-site +// /frame): two tabs the test opens, two popups the first opens at one +// moment, two pages asked for at once, a frame of the page's own origin +// (/nest), in a tab and in a popup, and a popup opened with noopener. +// Each test keeps what its pages drew. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +const read = (): unknown => (window as unknown as { __draws: unknown }).__draws; + +async function drew(p: Page): Promise<{ top: unknown; frame: unknown }> { + await p.waitForLoadState('load'); + const frame = p.frames().find((f) => f.url().endsWith('/frame')); + if (!frame) throw new Error(`no /frame among ${p.frames().map((f) => f.url()).join(', ')}`); + return { top: await p.evaluate(read), frame: await frame.evaluate(read) }; +} + +test('tabs', async ({ page, context }) => { + await page.goto(`${base}/draws`); + const second = await context.newPage(); + await second.goto(`${base}/draws`); + // Two popups at one moment, told apart by the names they were opened with. + const popups: Page[] = []; + const both = new Promise((resolve) => { + context.on('page', (p) => { popups.push(p); if (popups.length === 2) resolve(); }); + }); + await page.evaluate((u) => { window.open(u, 'one'); window.open(u, 'two'); }, `${base}/draws`); + await both; + const named: Record = {}; + for (const p of popups) { await p.waitForLoadState('load'); named[await p.evaluate(() => window.name)] = p; } + keep('tabs', { first: await drew(page), second: await drew(second), one: await drew(named.one), two: await drew(named.two) }); +}); + +test('atonce', async ({ context }) => { + // Asked for at once: each keeps its place in the order the test asked. + const [a, b] = await Promise.all([context.newPage(), context.newPage()]); + await b.goto(`${base}/draws`); + await a.goto(`${base}/draws`); + keep('atonce', { a: await drew(a), b: await drew(b) }); +}); + +test('nest', async ({ page }) => { + await page.goto(`${base}/nest`); + keep('nest', await drew(page)); +}); + +test('popnest', async ({ page, context }) => { + // A popup at /nest, whose frame of its own origin is placed by it. + await page.goto(`${base}/draws`); + const [p] = await Promise.all([context.waitForEvent('page'), page.evaluate((u) => { window.open(u); }, `${base}/nest`)]); + keep('popnest', await drew(p)); +}); + +test('noopener', async ({ page, context }) => { + await page.goto(`${base}/draws`); + const [p] = await Promise.all([context.waitForEvent('page'), page.evaluate((u) => { window.open(u, '_blank', 'noopener'); }, `${base}/draws`)]); + keep('noopener', { first: await drew(page), popup: await drew(p) }); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs new file mode 100644 index 0000000..9f0b020 --- /dev/null +++ b/playwright/test/random.test.mjs @@ -0,0 +1,681 @@ +// Unit rows of the test browser's seeding: the env rule, the seed, and the +// init script run in a Node vm context in place of a page. No browser. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { DOC_KEY, ENV_SEED_RANDOM, LOADS_KEY, PAGE_KEY, SEEDED_LINE, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, seedingFromEnv, unplacedLine, unseededLine } from '../dist/random.js'; +import { SEED } from '../dist/agent.js'; +import { OTHER, documentAt } from './draws.mjs'; + +// Two seeds, as keploy answers them. +const S1 = '0123456789abcdef0123456789abcdef'; +const S2 = 'fedcba9876543210fedcba9876543210'; + +const UUID_V4 = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/; + +test('seeding is on unless KEPLOY_SEED_RANDOM turns it off; another value is refused, naming it', () => { + for (const v of [undefined, '', 'on', 'ON', '1', 'true', ' on ']) { + assert.deepEqual(seedingFromEnv({ [ENV_SEED_RANDOM]: v }), { mode: 'seeded' }, String(v)); + } + for (const v of ['off', 'OFF', 'Off', '0', 'false', ' off ']) { + assert.deepEqual(seedingFromEnv({ [ENV_SEED_RANDOM]: v }), { mode: 'off' }, v); + } + for (const v of ['yes', 'no', 'seeded', '2']) { + const s = seedingFromEnv({ [ENV_SEED_RANDOM]: v }); + assert.equal(s.mode, 'refused', v); + assert.equal(s.refusal, `keploy: KEPLOY_SEED_RANDOM=${JSON.stringify(v)} is neither on nor off; set it to off to give the test browser its own random numbers, or unset it to seed them per test`); + } +}); + +test('the lines say what is seeded, what is not, where the seed lives, and how to turn it off or on', () => { + assert.equal(SEEDED_LINE, 'keploy: the test browser\'s random numbers (Math.random, crypto.getRandomValues, crypto.randomUUID) are seeded per test ' + + 'from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew; ' + + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named). ' + + 'To turn it off, record and replay with KEPLOY_SEED_RANDOM=off, or set use: { keploySeedRandom: false } in the Playwright config.'); + assert.equal(unseededLine('g › t'), 'keploy: keploy answered the begin of g › t with no seed, so its browser draws its own random numbers: ' + + 'its recording kept none (it was recorded before keploy kept seeds, or with seeding off: record the test again with seeding on, the default, to seed it), ' + + 'or the test was pulled from a Keploy branch, which keeps no seed, or this keploy predates seeds.'); + assert.equal(unplacedLine('g › t', 'http://h/p'), 'keploy: in g › t, the document at http://h/p drew the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is in a page the app opened that keploy cannot place in the test\'s order (a popup opened with noopener, as a link with target=_blank opens one, ' + + 'or at another origin than its opener\'s page, or after that page moved on, or by a document keploy could not place; or a frame of another origin in one).'); + assert.equal(unplacedLine('g › t', 'http://h/p', 'popup'), unplacedLine('g › t', 'http://h/p')); + assert.equal(unplacedLine('g › t', 'about:blank', 'own'), 'keploy: in g › t, the document at about:blank drew the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.'); + const uncounted = (beside) => 'keploy: in g › t, the document at http://h/f drew the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is a frame under a parent of another origin with no session storage (a sandboxed frame without allow-same-origin), or a frame inside one: ' + + `keploy cannot count the documents that frame loads, so seeded, the next document there (a reload, or a frame the app puts in again at its place)${beside} could draw the same numbers.`; + assert.equal(unplacedLine('g › t', 'http://h/f', 'uncounted', 3), uncounted(' or, at its path, one of the 3 other frame(s) beside that frame')); + assert.equal(unplacedLine('g › t', 'http://h/f', 'uncounted', 1), uncounted(' or, at its path, one of the 1 other frame(s) beside that frame')); + // With no other frame beside it, or a count that is no whole number above + // 0, the line names no other frame; a count is cut to a whole number. + for (const n of [0, NaN, -3, undefined, 0.5]) assert.equal(unplacedLine('g › t', 'http://h/f', 'uncounted', n), uncounted(''), String(n)); + assert.equal(unplacedLine('g › t', 'http://h/f', 'uncounted'), uncounted('')); + assert.equal(unplacedLine('g › t', 'http://h/f', 'uncounted', 2.5), uncounted(' or, at its path, one of the 2 other frame(s) beside that frame')); + assert.equal(unplacedLine('g › t', 'about:blank', 'frame'), 'keploy: in g › t, the document at about:blank drew the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + + 'which its parent document cannot list, or one whose document began after its removal), or a frame inside one.'); + assert.equal(unplacedLine('g › t', 'about:blank', 'nostorage', 3), 'keploy: in g › t, the document at about:blank drew the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is a top document with no session storage that keploy can keep its count in (a document of an opaque origin has none: an about:blank loaded after ' + + 'its page\'s first document, a data: URL, a page served with a CSP sandbox without allow-same-origin, or a popup that a page\'s first document opened; ' + + 'or the app has filled it), or a frame of its origin inside one: keploy cannot count the documents loaded at its path in the tab, so seeded, ' + + 'the next one there (a reload, or another visit to that path) could draw the same numbers.'); +}); + +test('an address a page sends is shown on one line, each control character a ?, cut to 512 characters', () => { + assert.equal(pageAddress('http://h/p'), 'http://h/p'); + assert.equal(pageAddress('http://h/a\nkeploy: forged\r\u0000\u001f\u007f\tz'), 'http://h/a?keploy: forged?????z'); + assert.equal(pageAddress(`http://h/${'x'.repeat(600)}`), `http://h/${'x'.repeat(503)}`); + assert.equal(pageAddress(42), '42'); +}); + +// A frame of the fixture's frame tree: its parent, its child frames, and +// whether it is detached. +function fakeFrame(parent = null) { + const f = { kids: [], detached: false, parentFrame: () => parent, childFrames: () => f.kids.slice(), isDetached: () => f.detached }; + if (parent !== null) parent.kids.push(f); + return f; +} + +test('an uncounted frame\'s line counts the frames beside the frame with no session storage, under its parent; none when it is gone or has no parent', () => { + // Its own frame (0 up), one inside it (1 up), two down (2 up). + const p = fakeFrame(); + const u = fakeFrame(p); + for (let i = 0; i < 3; i++) fakeFrame(p); + const inner = fakeFrame(u); + fakeFrame(u); + const deep = fakeFrame(inner); + assert.deepEqual(besideOf(u, 0), [u, p]); + assert.deepEqual(besideOf(inner, 1), [u, p]); + assert.deepEqual(besideOf(deep, 2), [u, p]); + assert.deepEqual(besideOf(inner, 0), [inner, u]); + assert.equal(othersNow(besideOf(inner, 1)), 3); + assert.equal(othersNow(besideOf(inner, 0)), 1); + // Counted when asked: a frame put in since counts. + const b = besideOf(u, 0); + fakeFrame(p); + assert.equal(othersNow(b), 4); + // A lone frame: none beside it. + const lone = fakeFrame(); + assert.equal(othersNow(besideOf(fakeFrame(lone), 0)), 0); + // Gone (its parent moved on, or the app took it out): none, whatever its + // parent holds now. + u.detached = true; + assert.equal(othersNow(b), 0); + // A top document, or a frame further up than the top: no parent, and + // none beside it. + assert.equal(besideOf(p, 0), null); + assert.equal(besideOf(u, 1), null); + assert.equal(besideOf(u, 5), null); + assert.equal(othersNow(null), 0); + // What no script of keploy's sends (not a count of frames up): its own. + for (const bad of [NaN, -1, undefined]) assert.deepEqual(besideOf(inner, bad), [inner, u], String(bad)); +}); + +test('a begin\'s seed is drawn afresh each time: 128 bits of hex, a seed keploy keeps', () => { + const seen = new Set(); + for (let i = 0; i < 64; i++) { + const s = freshSeed(); + assert.match(s, /^[0-9a-f]{32}$/); + assert.match(s, SEED); + seen.add(s); + } + assert.equal(seen.size, 64); + // keploy keeps 8 to 64 lower-case hex digits, as it answers them. + for (const s of ['0123abcd', 'f'.repeat(64)]) assert.match(s, SEED, s); + for (const s of ['', '0123abc', 'f'.repeat(65), '0123ABCD', '0123abcg', ' 0123abcd']) assert.doesNotMatch(s, SEED, s); +}); + +test('a seeded document draws the same numbers for the same seed and path, others for another seed or path', () => { + const s1 = initScript(S1); + const a = documentAt(s1, '/p').draws(); + assert.deepEqual(documentAt(s1, '/p').draws(), a); + assert.notDeepEqual(documentAt(initScript(S2), '/p').draws(), a); + assert.notDeepEqual(documentAt(s1, '/q').draws(), a); + for (const r of a.random) assert.ok(r >= 0 && r < 1 && r !== 0, String(r)); + // 53 bits a draw, as the browser's own: not a 32-bit value scaled. + const many = JSON.parse(documentAt(s1).run('JSON.stringify(Array.from({ length: 16 }, () => Math.random()))')); + assert.ok(many.some((r) => (r * 4294967296) % 1 !== 0), many.join(' ')); + assert.match(a.uuid, UUID_V4); + assert.equal(a.bytes.length, 8); + // Unseeded draws differ every time. + assert.notDeepEqual(documentAt('', '/p').draws(), documentAt('', '/p').draws()); +}); + +test('each page the test opened draws its own numbers at one path, the same ones each run; its place is named in every document after the seeding script', () => { + const s = initScript(S1); + const first = documentAt(s, '/p', new Map(), 't0').draws(); + const second = documentAt(s, '/p', new Map(), 't1').draws(); + // Two tabs of one test, each in a fresh session storage, at one path: a + // browser-made tab ID or idempotency key is not sent twice. + assert.notEqual(second.uuid, first.uuid); + assert.notDeepEqual(second, first); + // Each page draws the same numbers each run (record and replay). + assert.deepEqual(documentAt(s, '/p', new Map(), 't0').draws(), first); + assert.deepEqual(documentAt(s, '/p', new Map(), 't1').draws(), second); + // The page's name is read at the first draw, so the page script, which + // the browser runs after the context's, still names it; the window shows + // neither the name nor the seeding's state to the page's own scripts. + const d = documentAt(s, '/p', new Map(), 't1'); + assert.deepEqual(JSON.parse(d.run(`JSON.stringify([Object.keys(globalThis).filter((k) => k.startsWith('__keploy')), globalThis[${JSON.stringify(PAGE_KEY)}], typeof globalThis[${JSON.stringify(DOC_KEY)}]])`)), + [[], 't1', 'object']); + assert.match(pageScript('t3'), /"t3"/); +}); + +test('a document no page names and no opener places draws the browser\'s own numbers, never another page\'s, and tells the fixture once, at its first draw', () => { + const told = []; + const s = initScript(S1, '__report'); + const d = documentAt(s, '/p', new Map(), null, { report: '__report', told }); + // Nothing is told before a draw, and the report function is taken off the + // window as the document begins. + assert.deepEqual(told, []); + assert.equal(d.run('typeof globalThis.__report'), 'undefined'); + const a = d.draws(); + assert.deepEqual(told, ['http://h/p']); + const b = d.draws(); + assert.deepEqual(told, ['http://h/p']); + assert.notDeepEqual(b, a); + // Not the draws of a page the test opened, at that path or any. + assert.notDeepEqual(documentAt(s, '/p', new Map(), null).draws(), documentAt(s, '/p', new Map(), null).draws()); + assert.notDeepEqual(a, documentAt(s, '/p', new Map(), 't0').draws()); + assert.match(a.uuid, UUID_V4); + assert.equal(a.bytes.length, 8); + for (const r of a.random) assert.ok(r >= 0 && r < 1, String(r)); + // With no function to tell, it still draws its own. + assert.notDeepEqual(documentAt(initScript(S1), '/p', new Map(), null).draws(), documentAt(initScript(S1), '/p', new Map(), null).draws()); + // An opaque origin tells its URL with no query or fragment, never its + // origin's "null" before its path. + for (const [href, at] of [['about:blank', 'about:blank'], ['about:srcdoc?q#f', 'about:srcdoc'], ['http://h/sandboxed#f', 'http://h/sandboxed']]) { + const o = []; + documentAt(s, 'blank', null, null, { report: '__report', told: o, origin: 'null', href }).draws(); + assert.deepEqual(o, [at], href); + } + const e = []; + documentAt(s, 'blank', null, null, { report: '__report', told: e, origin: '', href: 'about:blank#x' }).draws(); + assert.deepEqual(e, ['about:blank']); + // A window's first document with no URL yet (Firefox, as its init + // scripts run) is about:blank. + const none = []; + documentAt(s, '', null, null, { report: '__report', told: none, origin: '', href: '' }).draws(); + assert.deepEqual(none, ['about:blank']); +}); + +test('a document loaded again in the same tab draws new numbers, the same ones each run', () => { + const s = initScript(S1); + const run = () => { const tab = new Map(); return [documentAt(s, '/p', tab).draws(), documentAt(s, '/p', tab).draws(), documentAt(s, '/q', tab).draws()]; }; + const [first, again, other] = run(); + assert.notDeepEqual(again, first); + assert.deepEqual(run(), [first, again, other]); + // Another path's count is its own: /q is its first load, as in a fresh tab. + assert.deepEqual(other, documentAt(s, '/q').draws()); + const tab = new Map(); + documentAt(s, '/p', tab); documentAt(s, '/p', tab); + assert.deepEqual(JSON.parse(tab.get(LOADS_KEY)), { '/p': 2 }); +}); + +test('a top document with no session storage draws the browser\'s own numbers and says why, once, but for a page\'s first document, which its mark seeds; with an unreadable count, a document still draws seeded numbers', () => { + const s = initScript(S1); + const r = initScript(S1, 'rep', 'MK'); + // A page's document, then a popup its opener places: never seeded with no + // session storage, each told once, with why; seeded with it. + const opener = { [DOC_KEY]: { opened: () => ({ indexOf: () => 0 }), key: () => 'K\n' } }; + for (const [page, o, as] of [['t0', undefined, 't0'], [null, opener, 'K\nw0']]) { + const told = []; + const d = documentAt(r, '/p', null, page, { report: 'rep', told, opener: o }); + const a = d.draws(); + d.draws(); + assert.deepEqual(told, [['http://h/p', 'nostorage']], as); + assert.notDeepEqual(a, documentAt(s, '/p', new Map(), as).draws(), as); + assert.notDeepEqual(documentAt(s, '/p', null, page, { opener: o }).draws(), documentAt(s, '/p', null, page, { opener: o }).draws(), as); + const kept = []; + assert.deepEqual(documentAt(r, '/p', new Map(), page, { report: 'rep', told: kept, opener: o }).draws(), documentAt(s, '/p', new Map(), as).draws(), as); + assert.deepEqual(kept, [], as); + } + // Session storage the app has filled (setItem throws): its loads cannot be + // counted, so the same. + const full = new (class extends Map { set() { throw new Error('QuotaExceededError'); } })(); + const fullTold = []; + assert.notDeepEqual(documentAt(r, '/p', full, 't0', { report: 'rep', told: fullTold }).draws(), documentAt(s, '/p', new Map(), 't0').draws()); + assert.deepEqual(fullTold, [['http://h/p', 'nostorage']]); + // A page's first document, marked by firstScript as newPage returns, is + // seeded with no count, the same each run; a document that takes its + // window over after the mark is not, and says why. + const first = () => { + const told = []; + const d = documentAt(r, 'blank', null, null, { report: 'rep', told, origin: 'null', href: 'about:blank', doc: { querySelectorAll: () => [] } }); + d.run(firstScript('t0', 'MK')); + return { d, told }; + }; + const f = first(); + // The mark is the seeding's state for the document: the window holds no + // property of it beside the page's name and the document's place. + assert.deepEqual(JSON.parse(f.d.run(`JSON.stringify(Object.getOwnPropertyNames(globalThis).filter((k) => k.startsWith('__keploy')).sort())`)), [DOC_KEY, PAGE_KEY].sort()); + const a = f.d.draws(); + assert.deepEqual(a, documentAt(s, 'blank', new Map(), 't0').draws()); + assert.deepEqual(first().d.draws(), a); + assert.deepEqual(f.told, []); + const later = first(); + later.d.ctx.document = { querySelectorAll: () => [] }; + assert.notDeepEqual(later.d.draws(), a); + assert.deepEqual(later.told, [['about:blank', 'nostorage']]); + // The mark is for the document it runs in: one that took the window over + // before the mark ran in it is the one marked. + const over = documentAt(r, 'blank', null, null, { origin: 'null', href: 'about:blank', doc: { querySelectorAll: () => [] } }); + over.ctx.document = { querySelectorAll: () => [] }; + over.run(firstScript('t0', 'MK')); + assert.deepEqual(over.draws(), a); + // A page's own script that marks its document with another value, or + // with none, does not seed it; nor does the right one in a script given + // no mark. + for (const [script, m] of [[r, 'other'], [r, undefined], [r, ''], [initScript(S1, 'rep'), '']]) { + const told = []; + const d = documentAt(script, 'blank', null, null, { report: 'rep', told, origin: 'null', href: 'about:blank', doc: { querySelectorAll: () => [] } }); + d.run(pageScript('t0') + `globalThis[${JSON.stringify(DOC_KEY)}].first(${m === undefined ? '' : JSON.stringify(m)});`); + assert.notDeepEqual(d.draws(), a, String(m)); + assert.deepEqual(told, [['about:blank', 'nostorage']], String(m)); + } + // The page's name alone, with no mark: not its first document. + const named = []; + assert.notDeepEqual(documentAt(r, 'blank', null, 't0', { report: 'rep', told: named, origin: 'null', href: 'about:blank', doc: { querySelectorAll: () => [] } }).draws(), a); + assert.deepEqual(named, [['about:blank', 'nostorage']]); + for (const junk of ['not json', '[1,2]', 'null', '{"/p":"x"}']) { + assert.deepEqual(documentAt(s, '/p', new Map([[LOADS_KEY, junk]])).draws(), documentAt(s, '/p').draws(), junk); + } +}); + +test('each function has its own stream: extra draws of one do not move another\'s', () => { + const s = initScript(S1); + const d = documentAt(s); + const base = documentAt(s).draws(); + d.run('for (let i = 0; i < 7; i++) Math.random();'); + const after = d.draws(); + assert.deepEqual(after.bytes, base.bytes); + assert.equal(after.uuid, base.uuid); + assert.notDeepEqual(after.random, base.random); + const e = documentAt(s); + e.run('crypto.getRandomValues(new Uint8Array(3)); crypto.randomUUID();'); + assert.deepEqual(e.draws().random, base.random); +}); + +test('getRandomValues fills the array it was given and returns it; what the browser refuses it still refuses', () => { + const d = documentAt(initScript(S1)); + assert.equal(d.run('(() => { const a = new Uint32Array(5); return crypto.getRandomValues(a) === a && a.some((x) => x !== 0); })()'), true); + // A view into the middle of a buffer is filled there only. + assert.equal(d.run('(() => { const b = new Uint8Array(12); crypto.getRandomValues(new Uint8Array(b.buffer, 4, 4)); return b.slice(0, 4).every((x) => x === 0) && b.slice(8).every((x) => x === 0); })()'), true); + assert.throws(() => d.run('crypto.getRandomValues(new Float32Array(2))')); + assert.throws(() => d.run('crypto.getRandomValues(new Uint8Array(65537))')); + // Called on a this that is no Crypto, either function throws as the + // browser's own does, and draws nothing from its stream. + const f = documentAt(initScript(S1)); + assert.throws(() => f.run('crypto.getRandomValues.call({}, new Uint8Array(4))'), /Illegal invocation/); + assert.throws(() => f.run('crypto.randomUUID.call({})'), /Illegal invocation/); + assert.throws(() => f.run('Crypto.prototype.randomUUID()'), /Illegal invocation/); + assert.throws(() => f.run('crypto.getRandomValues(new Float32Array(2))')); + assert.deepEqual(f.draws(), documentAt(initScript(S1)).draws()); + // The functions keep their names and lengths, and stay on the prototype. + assert.deepEqual(JSON.parse(d.run('JSON.stringify([Math.random.name, Math.random.length, crypto.getRandomValues.name, crypto.getRandomValues.length, crypto.randomUUID.name, Object.keys(crypto).length])')), + ['random', 0, 'getRandomValues', 1, 'randomUUID', 0]); +}); + +test('a document without crypto.randomUUID (not a secure context) is not given one', () => { + class Crypto { getRandomValues(a) { return a; } } + const d = documentAt(''); + d.ctx.Crypto = Crypto; + d.ctx.crypto = new Crypto(); + d.run(initScript(S1)); + assert.equal(d.run('typeof crypto.randomUUID'), 'undefined'); + assert.equal(d.run('typeof Math.random()'), 'number'); +}); + +test('a page the test asked for is the test\'s own when its first document tells while its newPage has not come; it is named in the order the test asked', async () => { + // A context whose pages come only when the row says, so the order is the + // row's, not the browser's. + const asked = []; + const original = function newPage() { return new Promise((resolve) => asked.push(resolve)); }; + const context = { newPage: original }; + const fakePage = () => { + const scripts = []; + return { scripts, addInitScript: async (s) => { scripts.push(s.content); }, evaluate: async (s) => { scripts.push(s); } }; + }; + const placing = placePages(context, 'MK'); + const first = fakePage(), second = fakePage(), popup = fakePage(); + const a = context.newPage(), b = context.newPage(); + // Both first documents tell before either newPage has come. + const firstOwn = placing.opened(first), secondOwn = placing.opened(second), popupOwn = placing.opened(popup); + // The second comes first: each keeps the place it was asked in. + asked[1](second); + asked[0](first); + assert.equal(await a, first); + assert.equal(await b, second); + assert.deepEqual([await firstOwn, await secondOwn, await popupOwn], [true, true, false]); + // Its later documents' script, then its first document's, which marks it + // with the context's mark. + assert.deepEqual(first.scripts, [pageScript('t0'), firstScript('t0', 'MK')]); + assert.deepEqual(second.scripts, [pageScript('t1'), firstScript('t1', 'MK')]); + assert.ok(firstScript('t1', 'MK').startsWith(pageScript('t1')) && firstScript('t1', 'MK').includes(`${JSON.stringify(DOC_KEY)}].first("MK")`)); + // After they came, and with none coming. + assert.deepEqual([await placing.opened(first), await placing.opened(second), await placing.opened(popup), await placing.opened(null)], [true, true, false, false]); + placing.restore(); + assert.equal(context.newPage, original); + // A context whose newPage is its prototype's gets the prototype's back. + const proto = { newPage: original }; + const inherited = Object.create(proto); + placePages(inherited, 'MK').restore(); + assert.equal(Object.prototype.hasOwnProperty.call(inherited, 'newPage'), false); + assert.equal(inherited.newPage, original); +}); + +// A frame element of a document the row makes: its tag, its src (null for +// none), whether it has srcdoc, the shadow root it hosts, the frame +// elements under it, and whether it has element children (inner; it has +// when kids has a frame). Reading its window counts in reads: the +// numbering never reads one (in Chromium that begins the frame's first +// document). Each query for frames under it counts in queries. +let reads = 0; +let queries = 0; +const el = (localName, src = null, { srcdoc = false, shadow = null, kids = [], inner = kids.length > 0 } = {}) => ({ + localName, shadowRoot: shadow, + firstElementChild: inner ? (kids[0] ?? { localName: 'span' }) : null, + get contentWindow() { reads++; return null; }, + getAttribute: (n) => (n === 'src' ? src : null), + hasAttribute: (n) => n === 'srcdoc' && srcdoc, + querySelector: (q) => { queries++; assert.equal(q, 'iframe,frame'); return kids[0] ?? null; }, + querySelectorAll: (q) => { queries++; assert.equal(q, 'iframe,frame'); return kids; }, +}); +// A document (or shadow root) whose elements, in tree order, are els; walks +// counts its walks. +const tree = (els, baseURI = 'https://h/host') => { + const t = { baseURI, walks: 0, querySelectorAll: (q) => { assert.equal(q, '*'); t.walks++; return els; } }; + return t; +}; +// The seeding state of a document whose elements are those of doc: what +// its frames' documents ask it, and the observer it watches doc with (a +// fake one, when observe is true). +function parentOf(doc, observe = false) { + const fake = observe ? `globalThis.__mos = []; globalThis.MutationObserver = class { + constructor(cb) { this.cb = cb; this.pending = []; this.asked = null; __mos.push(this); } + observe(t, o) { this.asked = [t, o]; } + takeRecords() { const r = this.pending; this.pending = []; return r; } + };\n` : ''; + const d = documentAt(fake + initScript(S1), '/host', new Map(), 't0', { doc }); + const api = d.run(`globalThis[${JSON.stringify(DOC_KEY)}]`); + return { enter: (e, w, p) => { try { return [...api.enter(e, w, p)]; } catch (x) { return String(x); } }, mo: observe ? d.ctx.__mos[0] : null }; +} +const added = (...n) => ({ addedNodes: n, removedNodes: [] }); +const removed = (...n) => ({ addedNodes: [], removedNodes: n }); + +test('a document numbers its frame elements at the path each asks for, in element order, whatever order their documents begin in, never reading a frame\'s window, and counts each place\'s loads', () => { + reads = 0; + const w = (n) => ({ n }); + const E = { + r0: el('iframe', '/fr?n=0'), sh: el('iframe', '/fr'), a: el('iframe', '/fa'), r1: el('iframe', 'https://cdn.example/fr#x'), + sd: el('iframe', '/ignored', { srcdoc: true }), none: el('iframe'), empty: el('iframe', ''), blob: el('iframe', 'blob:https://h/5d3c'), + data: el('iframe', 'data:text/html,

'), other: el('iframe', 'mailto:x'), bad: el('iframe', 'http://['), fr: el('frame', '/fb'), + emb: el('embed', '/fa'), closed: el('iframe', '/fr'), gone: el('iframe', '/fr'), + }; + const W = Object.fromEntries(Object.keys(E).map((k, i) => [k, w(i)])); + const els = [ + E.r0, el('div', null, { shadow: tree([E.sh]) }), E.a, E.r1, E.sd, E.none, E.empty, E.blob, + E.data, E.other, E.bad, E.fr, E.emb, + ]; + const doc = tree(els); + const p = parentOf(doc); + const d = { enter: (x, at) => p.enter(E[x], W[x], at) }; + // r1's document begins first: still the second at /fr, after r0, and + // after sh, in the shadow root of the element before a. + assert.deepEqual(d.enter('r1', '/fr'), ['f/fr\n2', 0]); + assert.deepEqual(d.enter('sh', '/fr'), ['f/fr\n1', 0]); + assert.deepEqual(d.enter('r0', '/fr'), ['f/fr\n0', 0]); + // Numbered once: a numbered frame's document begins with no walk. + assert.equal(doc.walks, 1); + // Each place counts the documents of each path it loads. + assert.deepEqual(d.enter('r1', '/fr'), ['f/fr\n2', 1]); + assert.deepEqual(d.enter('r1', '/fb'), ['f/fr\n2', 0]); + assert.deepEqual(d.enter('r0', '/fr'), ['f/fr\n0', 1]); + assert.deepEqual(d.enter('a', '/fa'), ['f/fa\n0', 0]); + // srcdoc, no src, an empty src, other schemes (by scheme alone: a blob: + // URL carries a browser-made ID), a src keploy cannot read, a frameset's + // frame. + for (const [x, at] of [['sd', 'about:srcdoc\n0'], ['none', 'about:blank\n0'], ['empty', 'about:blank\n1'], ['blob', 'blob:\n0'], ['data', 'data:\n0'], ['other', 'mailto:\n0'], ['bad', '?\n0'], ['fr', '/fb\n0']]) { + assert.deepEqual(d.enter(x, 'p'), [`f${at}`, 0], at); + } + assert.equal(doc.walks, 1); + // An element the document does not list as a frame (an embed, one in a + // closed shadow root, a removed one), and no element, have no place. + for (const x of ['emb', 'closed', 'gone']) assert.deepEqual(d.enter(x, '/fa'), [null, 0]); + assert.deepEqual(p.enter(null, w(99), '/fa'), [null, 0]); + // a removed, and a2 put in at its path: a2 takes the next number there; + // a keeps its place. + const a2 = el('iframe', '/fa'); + els.splice(2, 1, a2); + assert.deepEqual(p.enter(a2, w(100), '/fa'), ['f/fa\n1', 0]); + assert.deepEqual(d.enter('a', '/fa'), ['f/fa\n0', 1]); + // a2's element now holds another window (put back where no observer + // sees it): the next number at its path, which that window keeps. + const again = w(101); + assert.deepEqual(p.enter(a2, again, '/fa'), ['f/fa\n2', 0]); + assert.deepEqual(p.enter(a2, again, '/fa'), ['f/fa\n2', 1]); + // Put in another order, the same frames take the same places. + const [b, x, s0, s1] = [el('iframe', '/fb'), el('iframe', '/fx'), el('iframe', '/fr'), el('iframe', '/fr')]; + const e2 = parentOf(tree([b, x, s0, s1])); + assert.deepEqual(e2.enter(s1, w(1), '/fr'), ['f/fr\n1', 0]); + assert.deepEqual(e2.enter(b, w(2), '/fb'), ['f/fb\n0', 0]); + assert.equal(reads, 0); +}); + +test('a document looks into the nodes taken out of it only while it holds a numbered frame, counted exactly', () => { + const w = (n) => ({ n }); + const els = []; + const doc = tree(els); + const p = parentOf(doc, true); + const row = el('div', null, { inner: true }); + const look = () => { queries = 0; p.mo.cb([removed(row)]); return queries; }; + // a numbered, then c put in and taken out in one change (never + // numbered), then a taken out: the document holds none, and looks no more. + const a = el('iframe', '/fa'); + els.push(a); + p.mo.cb([added(a)]); + assert.equal(look(), 1); + // A node that is no frame and has no element child is not looked into. + queries = 0; + p.mo.cb([removed(el('span'), { nodeName: '#text' })]); + assert.equal(queries, 0); + const c = el('iframe', '/fa'); + p.mo.cb([added(c), removed(c)]); + assert.equal(look(), 1); + // A div holding c (never numbered) taken out does not count a off. + p.mo.cb([removed(el('div', null, { kids: [c] }))]); + assert.equal(look(), 1); + els.splice(els.indexOf(a), 1); + p.mo.cb([removed(a)]); + assert.equal(look(), 0); + // Put back, a takes the next number, and is looked for again. + els.push(a); + p.mo.cb([added(a)]); + assert.deepEqual(p.enter(a, w(1), '/fa'), ['f/fa\n1', 0]); + assert.equal(look(), 1); +}); + +test('one change that puts a frame in and then takes a numbered one out forgets the one taken out: its document, beginning after, has no place, and put back, it takes the next number', () => { + const w = (n) => ({ n }); + const els = []; + const doc = tree(els); + const p = parentOf(doc, true); + // f is numbered as it is put in; its document has not begun. + const f = el('iframe', '/fa'); + els.push(f); + p.mo.cb([added(f)]); + // One change: g put in, then f taken out. + const g = el('iframe', '/fb'); + els.splice(0, 1, g); + p.mo.cb([added(g), removed(f)]); + assert.deepEqual(p.enter(f, w(0), '/fa'), [null, 0]); + assert.deepEqual(p.enter(g, w(1), '/fb'), ['f/fb\n0', 0]); + // f put back: the next number at /fa. + els.push(f); + p.mo.cb([added(f)]); + assert.deepEqual(p.enter(f, w(2), '/fa'), ['f/fa\n1', 0]); +}); + +test('a document\'s observer numbers its frames as the app puts them in, only when a change puts one in, forgets a frame taken out, and is read up to date as a frame\'s document begins', () => { + reads = 0; + const w = (n) => ({ n }); + const els = []; + const doc = tree(els); + const p = parentOf(doc, true); + assert.equal(p.mo.asked[0], doc); + assert.deepEqual({ ...p.mo.asked[1] }, { childList: true, subtree: true }); + // A change that puts in, or takes out, no frame costs no walk. A text + // node or an element with no element child costs no query; one with + // element children, one; and while the document holds no numbered frame, + // a node taken out is not looked into. + const span = el('span'); + const row = el('div', null, { inner: true }); + els.push(span, row); + queries = 0; + p.mo.cb([added(span, { nodeName: '#text' }, row), removed(span), removed(row)]); + assert.equal(doc.walks, 0); + assert.equal(queries, 1); + // b0 put in, then b1 before it: numbered as they come, whatever order + // their documents begin in. A div that holds a frame counts as one. + // Once a change has put a frame in, the rest of it is not looked into. + const [b0, b1] = [el('iframe', '/fr'), el('iframe', '/fr')]; + els.push(b0); + queries = 0; + p.mo.cb([added(b0, row), added(row)]); + assert.equal(queries, 0); + const box = el('div', null, { kids: [b1] }); + els.unshift(box, b1); + p.mo.cb([added(box)]); + assert.equal(doc.walks, 2); + assert.deepEqual(p.enter(b1, w(1), '/fr'), ['f/fr\n1', 0]); + assert.deepEqual(p.enter(b0, w(0), '/fr'), ['f/fr\n0', 0]); + assert.equal(doc.walks, 2); + // b0 taken out and put back after b1: it holds another window, and takes + // the next number as it is put back. A div taken out forgets its frames. + p.mo.cb([removed(b0), added(b0)]); + assert.deepEqual(p.enter(b0, w(2), '/fr'), ['f/fr\n2', 0]); + els.splice(els.indexOf(box), 1); + p.mo.cb([removed(box)]); + els.splice(els.indexOf(b1), 1); + assert.deepEqual(p.enter(b1, w(1), '/fr'), [null, 0]); + // While it holds a numbered frame (b0), a node taken out is looked into. + queries = 0; + p.mo.cb([removed(row)]); + assert.equal(queries, 1); + // Changes the observer has not yet delivered are read first, in order: + // y put back and z put in after it, z's document beginning first. + const [y, z] = [el('iframe', '/fy'), el('iframe', '/fy')]; + els.push(y); + p.mo.cb([added(y)]); + assert.deepEqual(p.enter(y, w(3), '/fy'), ['f/fy\n0', 0]); + els.push(z); + p.mo.pending = [removed(y), added(y), added(z)]; + assert.deepEqual(p.enter(z, w(4), '/fy'), ['f/fy\n2', 0]); + assert.deepEqual(p.enter(y, w(5), '/fy'), ['f/fy\n1', 0]); + assert.equal(p.mo.pending.length, 0); + assert.equal(reads, 0); +}); + +test('a frame of its parent\'s origin draws in its parent document\'s key and its place there; one under a parent of another origin, in its page by its path, and never with no session storage; one with no place draws its own and says why', () => { + const at = 'http://h/f'; + // Placed by its parent: the parent's key, its place, its own path and + // its loads in that place; session storage untouched. + // It names itself by its element, read through the window's own getter + // as the script began: a page script that later puts another frameElement + // on the window does not move it, when a document that ran no init + // script takes the window over. + const asked = []; + const mine = { tag: 'its element' }; + const parent = { [DOC_KEY]: { enter: (e, w, p) => { asked.push([e, p]); return ['f/f\n3', 2]; }, key: () => 'K\n', why: () => 'unused', above: () => 0 } }; + const store = new Map(); + const placed = documentAt(initScript(S1), '/f', store, null, { parent, element: mine, doc: { querySelectorAll: () => [] } }); + assert.deepEqual(placed.draws(), documentAt(initScript(S1), '/f', new Map([[LOADS_KEY, '{"/f":2}']]), 'K\nf/f\n3').draws()); + placed.run('Object.defineProperty(globalThis, "frameElement", { value: { tag: "the page\'s" }, configurable: true });'); + placed.ctx.document = { querySelectorAll: () => [] }; + placed.draws(); + assert.deepEqual(asked, [[mine, '/f'], [mine, '/f']]); + assert.equal(store.size, 0); + // A frameElement that is no getter: no element, so no place. + const plain = []; + const p2 = { [DOC_KEY]: { enter: (e, w, p) => { plain.push(e); return [null, 0]; }, key: () => 'K\n', why: () => '', above: () => 0 } }; + const d2 = documentAt(initScript(S1), '/f', new Map(), null, { parent: p2 }); + d2.ctx.frameElement = mine; + d2.run(initScript(S1)); + assert.deepEqual(plain, [undefined, mine]); + // Under a parent of another origin: its page, marked, its path, and its + // origin's count in the tab. + const tab = new Map([[LOADS_KEY, '{"/f":1}']]); + assert.deepEqual(documentAt(initScript(S1), '/f', tab, 't0', { parent: OTHER }).draws(), documentAt(initScript(S1), '/f', new Map([[LOADS_KEY, '{"/f":1}']]), 't0x').draws()); + assert.equal(tab.get(LOADS_KEY), '{"/f":2}'); + // In a page no page script names, it has no place, and says so with no + // reason of its own, with session storage or without. + for (const st of [new Map(), null]) { + const none = []; + const u = documentAt(initScript(S1, 'rep'), '/f', st, null, { parent: OTHER, report: 'rep', told: none, frames: 3 }); + assert.notDeepEqual(u.draws(), u.draws()); + assert.deepEqual(none, [at]); + } + // With no session storage to count its loads in, it is never seeded, + // alone under its parent or not: nothing tells its next document (a + // reload, a frame put in again at its place) from it. It draws its own + // and says it is that frame (0 frames up: the fixture counts the frames + // beside it, othersBeside), whatever its parent's length says. With + // session storage, it is seeded, whatever its parent holds. + assert.deepEqual(documentAt(initScript(S1), '/f', new Map(), 't0', { parent: OTHER, frames: 4 }).draws(), documentAt(initScript(S1), '/f', new Map(), 't0x').draws()); + for (const frames of [1, 2, 5, undefined, 'x']) { + const told = []; + const t = documentAt(initScript(S1, 'rep'), '/f', null, 't0', { parent: OTHER, frames, report: 'rep', told }); + const drew = t.draws(); + assert.notDeepEqual(drew, t.draws(), String(frames)); + assert.notDeepEqual(drew, documentAt(initScript(S1), '/f', new Map(), 't0x').draws(), String(frames)); + assert.deepEqual(told, [[at, 'uncounted', 0]], String(frames)); + } + // The parent is the one the document began with: a page script that + // replaces the global parent (var parent = 0, parent = self, a parent of + // its own making) before the first draw moves neither the rule nor what + // it tells. + for (const swap of ['var parent = 0;', 'parent = globalThis;', 'parent = { length: 1 };', 'parent = { get length() { throw new Error(); } };']) { + const told = []; + const t = documentAt(initScript(S1, 'rep'), '/f', null, 't0', { parent: OTHER, frames: 3, report: 'rep', told }); + t.run(swap); + const drew = t.draws(); + assert.notDeepEqual(drew, t.draws(), swap); + assert.deepEqual(told, [[at, 'uncounted', 0]], swap); + } + // A frame its parent cannot place, one whose parent is gone, and one + // whose parent has no seeding state: no place, the frame reason. + for (const p of [{ [DOC_KEY]: { enter: () => [null, 0], key: () => 'K\n', why: () => '', above: () => 0 } }, null, {}]) { + const told = []; + const f = documentAt(initScript(S1, 'rep'), '/f', new Map(), 't0', { parent: p, report: 'rep', told }); + let drew = []; + try { drew = [f.draws(), f.draws()]; } catch (e) { drew = [String(e)]; } + assert.deepEqual(told, [[at, 'frame']]); + assert.notDeepEqual(drew[0], drew[1]); + } + // A frame placed in a parent with no place: no place, its parent's + // reason; for 'uncounted', one frame further up than its parent's. + for (const why of ['', 'frame', 'uncounted', 'nostorage']) { + const told = []; + documentAt(initScript(S1, 'rep'), '/f', new Map(), 't0', { parent: { [DOC_KEY]: { enter: () => ['f/f\n0', 0], key: () => null, why: () => why, above: () => 3 } }, report: 'rep', told }).draws(); + assert.deepEqual(told, [why === 'uncounted' ? [at, why, 4] : why ? [at, why] : at], why); + } +}); + +test('a document that a second run of the script takes over (WebKit runs it in a document that takes over its window\'s first about:blank) begins once: the first run passes on the browser\'s own', () => { + const store = new Map(); + const blank = { baseURI: 'about:blank', querySelectorAll: () => [] }; + // The window's first about:blank runs the script; a window.open of its + // own is there to be wrapped. + const d = documentAt(`globalThis.open = function () { return {}; };\n${initScript(S1)}`, 'blank', store, 't0', { doc: blank }); + // Its document is taken over by /p, which runs the script again. + d.ctx.document = { baseURI: 'http://h/p', querySelectorAll: () => [] }; + d.ctx.location.pathname = '/p'; + d.run(initScript(S1)); + d.run(pageScript('t0')); + // getRandomValues and randomUUID call the first run's functions first, + // and a popup goes through its window.open: /p is still counted once. + const drew = d.draws(); + d.run('open()'); + assert.equal(store.get(LOADS_KEY), '{"blank":1,"/p":1}'); + assert.deepEqual(drew, documentAt(initScript(S1), '/p', new Map(), 't0').draws()); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 7e452fa..e4ba7a6 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1,6 +1,7 @@ // Rows: real Playwright runs of test/fixtures against a fake keploy agent, -// which records every call and answers as each row says. The settle rows -// need Chromium (npx playwright install chromium). +// which records every call and answers as each row says. They need +// Chromium, and the seeding's engine rows Firefox and WebKit too +// (npx playwright install chromium firefox webkit). import { test } from 'node:test'; import assert from 'node:assert/strict'; import { spawn } from 'node:child_process'; @@ -9,6 +10,8 @@ import http from 'node:http'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; +import { LOADS_KEY, SEEDED_LINE, initScript, unplacedLine, unseededLine } from '../dist/random.js'; +import { BLANK_DRAWS, DRAWS, documentAt } from './draws.mjs'; const PKG = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); const CLI = path.join(PKG, 'node_modules', '@playwright', 'test', 'cli.js'); @@ -21,7 +24,8 @@ function journal() { } // A fake keploy agent. answer(call) returns {status, body} or undefined for -// the protocol's plain "ok". +// the protocol's plain "ok", which answers a begin with the seed it sent, as +// a record does. async function fakeAgent(j, answer = () => undefined) { const srv = http.createServer((req, res) => { let body = ''; @@ -33,7 +37,8 @@ async function fakeAgent(j, answer = () => undefined) { if (!a) { a = { body: call.path === '/agent/instance' ? { v: 1, instance: 'run', timeFile: '' } : call.path === '/agent/scope/given' ? { v: 1, status: 'ok', variable: `given:${call.body.name}` } - : { v: 1, status: 'ok' } }; + : call.path === '/agent/scope/begin' && call.body.seed ? { v: 1, status: 'ok', seed: call.body.seed } + : { v: 1, status: 'ok' } }; } res.writeHead(a.status ?? 200, { 'content-type': 'application/json' }); res.end(JSON.stringify(a.body)); @@ -124,6 +129,7 @@ async function playwright(specs, env, cwd = PKG) { error: (r.errors ?? []).map((e) => e.message ?? '').join('\n'), annotations: r.annotations ?? t.annotations ?? [], stderr: (r.stderr ?? []).map((e) => e.text ?? '').join(''), + stdout: (r.stdout ?? []).map((x) => x.text ?? Buffer.from(x.buffer ?? '', 'base64').toString()).join(''), attempts: t.results.length, }; }); @@ -138,6 +144,15 @@ async function playwright(specs, env, cwd = PKG) { const keployEnv = (url) => ({ KEPLOY_MOCK_AGENT: url, KEPLOY_MOCK_AGENT_TOKEN: 'tok', KEPLOY_MOCK_PROJECT: PKG, KEPLOY_TEST_PID: undefined }); const calls = (j) => j.events.filter((e) => e.agent).map((e) => e.agent); +// Every call's path and body, each begin's seed shown as FRESH once it is +// checked to be a fresh one: 128 bits of hex, no two begins the same. +const FRESH = ''; +function wire(j) { + const seeds = calls(j).filter((c) => c.path === '/agent/scope/begin' && c.body.seed !== undefined).map((c) => c.body.seed); + for (const sd of seeds) assert.match(sd, /^[0-9a-f]{32}$/); + assert.equal(new Set(seeds).size, seeds.length, seeds.join(' ')); + return calls(j).map((c) => [c.path, c.body && c.body.seed !== undefined ? { ...c.body, seed: FRESH } : c.body]); +} const ref = (file, name) => ({ name, file: `test/fixtures/${file}`, project: 'p1' }); test('inert without keploy: no call, every test runs as written', async () => { @@ -165,13 +180,13 @@ test('a worker claims the run, each test begins before its beforeEach and ends w const given = { name: 'seed', value: 'seed-value-1', source: 'test/fixtures/basic.spec.ts:7' }; const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; // b fails, so Playwright runs c and d in a new worker, which claims again. - assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + assert.deepEqual(wire(j), [ ['/agent/instance', claim], - ['/agent/scope/begin', { ...r('a'), canSkip: true }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('a'), outcome: 'passed' }], - ['/agent/scope/begin', { ...r('b'), canSkip: true }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('b'), outcome: 'failed' }], + ['/agent/scope/begin', { ...r('a'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('a'), outcome: 'passed' }], + ['/agent/scope/begin', { ...r('b'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('b'), outcome: 'failed' }], ['/agent/instance', claim], - ['/agent/scope/begin', { ...r('c'), canSkip: true }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('c'), outcome: 'skipped' }], - ['/agent/scope/begin', { ...r('d'), canSkip: true }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('d'), outcome: 'passed' }], + ['/agent/scope/begin', { ...r('c'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('c'), outcome: 'skipped' }], + ['/agent/scope/begin', { ...r('d'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('d'), outcome: 'passed' }], ]); assert.ok(calls(j).every((c) => c.auth === 'Bearer tok')); // KEPLOY_TEST_PID named the claiming worker (test a checks it). @@ -242,9 +257,9 @@ test('a test Playwright retries is not begun again: the retry fails, saying why, assert.ok(!fs.existsSync(path.join(tmp, 'attempt-1')), 'the retry ran'); const r = ref('retry.spec.ts', retried); const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; - assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + assert.deepEqual(wire(j), [ ['/agent/instance', claim], - ['/agent/scope/begin', { ...r, canSkip: true }], ['/agent/scope/end', { ...r, outcome: 'failed' }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'failed' }], // The retry runs in a new worker, which claims the run again. ['/agent/instance', claim], ['/agent/refusal', retryRefusal], @@ -285,9 +300,9 @@ test('a test --repeat-each runs again is not begun again: the repeat fails, sayi assert.equal(tests['judged red'].runs[1].error.split('\n')[0], `KeployError: keploy: ${message} (${remedy})`); const r = ref('agent-says.spec.ts', 'judged red'); const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; - assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + assert.deepEqual(wire(j), [ ['/agent/instance', claim], - ['/agent/scope/begin', { ...r, canSkip: true }], ['/agent/scope/end', { ...r, outcome: 'passed' }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'passed' }], // Playwright runs each repeat in a worker of its own. ['/agent/instance', claim], ['/agent/refusal', { code: 'test_run_again', message, remedy }], @@ -400,9 +415,9 @@ test('an unnamed Playwright project claims with the key ".", and its tests name const { tests, out } = await playwright(['agent-says.spec.ts', '-g', 'judged red'], { ...keployEnv(a.url), PW_UNNAMED: '1' }); assert.equal(tests['judged red'].status, 'passed', out); const r = { name: 'judged red', file: 'test/fixtures/agent-says.spec.ts' }; - assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + assert.deepEqual(wire(j), [ ['/agent/instance', { key: '.', dir: fs.realpathSync(PKG) }], - ['/agent/scope/begin', { ...r, canSkip: true }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'passed' }], ]); } finally { await a.close(); } @@ -420,9 +435,9 @@ test('a worker started above keploy\'s project claims with the project\'s folder const { tests, out } = await playwright(['agent-says.spec.ts', '-g', 'judged red'], keployEnv(a.url), path.dirname(PKG)); assert.equal(tests['judged red'].status, 'passed', out); const r = ref('agent-says.spec.ts', 'judged red'); - assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + assert.deepEqual(wire(j), [ ['/agent/instance', { key: 'p1', dir: fs.realpathSync(PKG) }], - ['/agent/scope/begin', { ...r, canSkip: true }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'passed' }], ]); } finally { await a.close(); } @@ -436,9 +451,9 @@ test('when keploy names no project, a worker claims with the current folder, whi const { tests, out } = await playwright(['agent-says.spec.ts', '-g', 'judged red'], { ...keployEnv(a.url), KEPLOY_MOCK_PROJECT: undefined }, cwd); assert.equal(tests['judged red'].status, 'passed', out); const r = { name: 'judged red', file: `${path.basename(PKG)}/test/fixtures/agent-says.spec.ts`, project: 'p1' }; - assert.deepEqual(calls(j).map((c) => [c.path, c.body]), [ + assert.deepEqual(wire(j), [ ['/agent/instance', { key: 'p1', dir: fs.realpathSync(cwd) }], - ['/agent/scope/begin', { ...r, canSkip: true }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'passed' }], ]); } finally { await a.close(); } @@ -537,3 +552,687 @@ for (const [title, why, call] of [ } finally { await p.close(); await a.close(); } }); } + +// The seeding rows' page server: /draws draws random numbers in its first +// script, as it loads, and holds /frame from another host (localhost against +// 127.0.0.1: a cross-site frame), which draws the same way. +async function drawsServer() { + const page = (body) => `${body}`; + // /held.js's answer waits for a load of /fh (or 10 s): each load lets one + // /held.js through. + const held = (() => { + let loads = 0; + const waiting = []; + return { + loaded: () => { const w = waiting.shift(); if (w) w(); else loads++; }, + script: (done) => { + if (loads > 0) { loads--; done(); return; } + const t = setTimeout(() => { const i = waiting.indexOf(go); if (i >= 0) waiting.splice(i, 1); done(); }, 10000); + t.unref(); + const go = () => { clearTimeout(t); done(); }; + waiting.push(go); + }, + }; + })(); + const srv = http.createServer((req, res) => { + const u = new URL(req.url, 'http://x'); + // /csp is served with a CSP sandbox: an opaque origin, no session storage. + if (u.pathname === '/csp') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('csp')); return; } + if (u.pathname === '/held.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); held.script(() => res.end('')); return; } + res.writeHead(200, { 'content-type': 'text/html' }); + if (u.pathname === '/draws') { res.end(page(``)); return; } + if (u.pathname === '/frame') { res.end(page('frame')); return; } + // /nest holds /frame of its own origin. + if (u.pathname === '/nest') { res.end(page('')); return; } + // /long/... is a page at an address of any length. + if (u.pathname.startsWith('/long/')) { res.end(page('')); return; } + // places.spec.ts's: /host puts in a (/fa, its own origin), c (/fc) and + // w (/fw, both cross-site) in the order ?order= names; /race holds r0 + // and r1 at /fr, and /fr answers late the one ?slow= names; /nested + // holds x (/fa) and n (/fn), which holds y, z (/fb) and nn (/fa). + const port = srv.address().port; + if (u.pathname === '/host') { + const src = { a: '/fa', c: `http://localhost:${port}/fc`, w: `http://localhost:${port}/fw` }; + const put = (u.searchParams.get('order') || '').split('').filter((n) => src[n]) + .map((n) => `{const f=document.createElement('iframe');f.name='${n}';f.src='${src[n]}';document.body.appendChild(f);}`).join(''); + res.end(page(``)); + return; + } + if (u.pathname === '/race') { + const slow = u.searchParams.get('slow'); + res.end(page(``)); + return; + } + // /before puts in b0 (/fr, answered late), in a div when ?wrap=1, then + // 50 ms later b1 (/fr, answered at once) before it; /set is a frameset + // of f0 and f1 at /fa. + if (u.pathname === '/before') { + const wrap = u.searchParams.get('wrap') === '1'; + res.end(page(``)); + return; + } + if (u.pathname === '/set') { res.end(``); return; } + if (u.pathname === '/fr') { setTimeout(() => res.end(page('r')), u.searchParams.get('n') === u.searchParams.get('slow') ? 800 : 0); return; } + if (u.pathname === '/nested') { res.end(page('')); return; } + if (u.pathname === '/fn') { res.end(page('')); return; } + if (['/fa', '/fb', '/fc', '/fw', '/fm', '/fs'].includes(u.pathname)) { res.end(page(u.pathname)); return; } + // /fsv, /fss and /fso replace the global parent before they draw. + const swap = { '/fsv': 'var parent = 0;', '/fss': 'parent = self;', '/fso': 'window.parent = { length: 1 };' }[u.pathname]; + if (swap !== undefined) { res.end(`${u.pathname}`); return; } + // many.spec.ts's: /many holds k frames m0.. at /fm, in its markup or put + // in by one script (?how=); /sbx holds n sandboxed frames s0.. at /fs, + // with nest=1 at /fsn (each holding a srcdoc frame c, of its origin), + // or with swap=1, n at each of /fsv (v0..), /fss (s0..) and /fso (o0..); + // /cost is a page of n elements, or of that many open shadow roots of + // 10 elements each, beside an empty #box. + if (u.pathname === '/many') { + const k = Number(u.searchParams.get('k')); + res.end(page(u.searchParams.get('how') === 'markup' + ? Array.from({ length: k }, (_, i) => ``).join('') + : ``)); + return; + } + if (u.pathname === '/sbx') { + const n = Number(u.searchParams.get('n')); + const at = u.searchParams.get('swap') ? [['v', '/fsv'], ['s', '/fss'], ['o', '/fso']] : [['s', u.searchParams.get('nest') ? '/fsn' : '/fs']]; + res.end(page(at.flatMap(([k, path]) => Array.from({ length: n }, (_, i) => ``)).join(''))); + return; + } + if (u.pathname === '/fsn') { res.end(page(``)); return; } + // /held holds sandboxed frame h0 at /fh, then /held.js, then h1 and h2 + // at /fh. /held.js is answered only once h0 has loaded (its load sends + // /fh-loaded), so its parent's parser puts h1 and h2 in only after h0 + // has drawn and loaded, every run. + if (u.pathname === '/held') { + const f = (n) => ``; + res.end(page(`${f('h0')}${f('h1')}${f('h2')}`)); + return; + } + if (u.pathname === '/fh') { res.end(page('')); return; } + if (u.pathname === '/fh-loaded') { held.loaded(); res.end(''); return; } + // parse.spec.ts's: two sandboxed frames u0, u1 at /fs, then a script + // (/unparsed) or an image (/unloaded) the server never answers. + if (u.pathname === '/unparsed' || u.pathname === '/unloaded') { + const f = (n) => ``; + res.end(page(`${f('u0')}${f('u1')}${u.pathname === '/unparsed' ? '' : ''}`)); + return; + } + if (u.pathname === '/hold.js' || u.pathname === '/hold.png') return; + if (u.pathname === '/cost') { + const n = Number(u.searchParams.get('n') || 0); + const sh = Number(u.searchParams.get('shadow') || 0); + res.end(page(`

`)); + return; + } + res.end(''); + }); + // Every address: the frame is asked for at localhost, which may be ::1. + await new Promise((r) => srv.listen(0, r)); + return { url: `http://127.0.0.1:${srv.address().port}`, close: () => new Promise((r) => { srv.closeAllConnections(); srv.close(r); }) }; +} + +// One run of seeded.spec.ts (or another spec of the /draws page): what each +// test's pages drew, the run's tests, and what its begins and ends sent. +async function drawRun(j, env, specs = ['seeded.spec.ts'], names = ['one', 'two', 'again', 'three']) { + const d = await drawsServer(); + const from = j.events.length; + try { + const { tests, tmp, out } = await playwright(specs, { ...env, DRAWS_BASE: d.url }); + for (const [k, t] of Object.entries(tests)) assert.equal(t.status, 'passed', `${k}: ${t.error}\n${out}`); + const drew = {}; + for (const n of names) drew[n] = JSON.parse(fs.readFileSync(path.join(tmp, `${n}.json`), 'utf8')); + const lines = Object.values(tests).map((t) => t.stdout).join('').split('\n'); + const said = lines.filter((l) => l === SEEDED_LINE).length; + const sent = j.events.slice(from).filter((e) => e.agent).map((e) => e.agent); + const begins = sent.filter((c) => c.path === '/agent/scope/begin').map((c) => c.body); + const ends = sent.filter((c) => c.path === '/agent/scope/end').map((c) => [c.body.name, c.body.random]); + // The seed each test's begin sent. + const seeds = Object.fromEntries(begins.map((b) => [b.name.replace(/^own › /, ''), b.seed])); + return { drew, said, lines, tests, tmp, begins, ends, seeds }; + } finally { await d.close(); } +} + +// What a document at path draws first under seed, in the page the fixture +// named page (t0: the test's first). +const expected = (seed, path, storage, page = 't0') => documentAt(initScript(seed), path, storage, page).draws(); +// A document's key: what a frame of its origin's page is named from, with +// the frame's place among its frames after it. +const keyOf = (seed, page, path, loads = 0) => `${seed}\n${page}\n${path}\n${loads}\n`; +const NAMES = ['one', 'two', 'again', 'own › three']; + +test('seeded: a record seeds each test\'s pages from the fresh seed its begin sent, from their first script and in a cross-site frame; another test, or the record made again, draws others', async () => { + const j = journal(); + const a = await fakeAgent(j); + try { + const r1 = await drawRun(j, keployEnv(a.url)); + const r2 = await drawRun(j, keployEnv(a.url)); + for (const r of [r1, r2]) { + // The numbers are the seed's: the init script ran before the page's + // first script, in every frame. + for (const n of ['one', 'two']) assert.deepEqual(r.drew[n], { top: expected(r.seeds[n], '/draws'), frame: expected(r.seeds[n], '/frame', undefined, 't0x') }, n); + // A page loaded again draws new numbers; a query is no part of its name. + const tab = new Map(); + const loads = [expected(r.seeds.again, '/draws', tab), expected(r.seeds.again, '/draws', tab), expected(r.seeds.again, '/draws', tab)]; + assert.deepEqual(r.drew.again.map((l) => l.top), loads); + assert.notDeepEqual(loads[0], loads[1]); + assert.notDeepEqual(r.drew.one, r.drew.two); + // Each begin says how its test's browser means to draw; one that + // seeds sends a fresh seed. The option's test sends none. + assert.deepEqual(r.begins.map((b) => [b.name, b.random, typeof b.seed === 'string' && /^[0-9a-f]{32}$/.test(b.seed)]), + [['one', 'seeded', true], ['two', 'seeded', true], ['again', 'seeded', true], ['own › three', 'own', false]]); + // Each end says what its pages drew from. + assert.deepEqual(r.ends, [['one', 'seeded'], ['two', 'seeded'], ['again', 'seeded'], ['own › three', 'own']]); + // Said once a worker: one worker, one line; and no test went unseeded. + assert.equal(r.said, 1); + assert.equal(r.lines.filter((l) => l.includes('with no seed')).length, 0); + // Called with a this that is no Crypto, the seeded functions throw + // as the browser's own do. + assert.deepEqual(JSON.parse(fs.readFileSync(path.join(r.tmp, 'this.json'), 'utf8')), + { grv: 'TypeError: Illegal invocation', uuid: 'TypeError: Illegal invocation', drew: true }); + } + // A record made again draws new numbers: a browser-made ID is not sent + // twice to a real server. + for (const n of ['one', 'two', 'again']) { + assert.notEqual(r2.seeds[n], r1.seeds[n], n); + assert.notDeepEqual(r2.drew[n], r1.drew[n], n); + } + // A test that turned it off with the option draws the browser's own. + assert.notDeepEqual(r2.drew.three, r1.drew.three); + } finally { await a.close(); } +}); + +test('seeded: a replay draws the numbers its recording drew, from the seed keploy answers, not the one its begin sent', async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + // The replay's agent answers each begin with the seed its recording kept. + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + try { + const r = await drawRun(j, keployEnv(rec.url)); + kept = r.seeds; + for (let i = 0; i < 2; i++) { + const p = await drawRun(j, keployEnv(rep.url)); + for (const n of ['one', 'two', 'again']) { + assert.deepEqual(p.drew[n], r.drew[n], n); + assert.notEqual(p.seeds[n], kept[n], n); + } + assert.notDeepEqual(p.drew.three, r.drew.three); + assert.deepEqual(p.ends, [['one', 'seeded'], ['two', 'seeded'], ['again', 'seeded'], ['own › three', 'own']]); + assert.equal(p.said, 1); + } + // The draws follow the seed keploy answers, not the test's name: a test + // whose entry was renamed (two, answered one's seed) draws as it did. + kept = { ...r.seeds, two: r.seeds.one }; + const renamed = await drawRun(j, keployEnv(rep.url)); + assert.deepEqual(renamed.drew.two, r.drew.one); + } finally { await rep.close(); await rec.close(); } +}); + +test('seeded: each page of a test draws its own numbers at one path, a popup placed by its opener too, the same ones at replay; a document keploy cannot place draws its own and is named', async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const names = ['tabs', 'atonce', 'nest', 'popnest', 'noopener']; + // The unplaced lines a test printed, with the run's port as . + const unplaced = (r, n) => r.tests[n].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)).map((l) => l.replace(/:\d+\//, ':/')); + try { + const r = await drawRun(j, keployEnv(rec.url), ['tabs.spec.ts'], names); + const seed = r.seeds.tabs; + const t = r.drew.tabs; + // Two tabs of the test at one path: each its own numbers, the tab's + // place in the order the test opened them naming it, in its cross-site + // frame too. + assert.notEqual(t.second.top.uuid, t.first.top.uuid); + assert.deepEqual(t.first, { top: expected(seed, '/draws'), frame: expected(seed, '/frame', undefined, 't0x') }); + assert.deepEqual(t.second, { top: expected(seed, '/draws', undefined, 't1'), frame: expected(seed, '/frame', undefined, 't1x') }); + // Two popups the first tab opened at one moment: each the window its + // opener's document opened, in that order, from the opener's copied + // session storage (its /draws loaded once). + const opener = `${seed}\nt0\n/draws\n0\n`; + for (const [w, i] of [['one', 0], ['two', 1]]) { + assert.deepEqual(t[w].top, expected(seed, '/draws', new Map([[LOADS_KEY, '{"/draws":1}']]), `${opener}w${i}`), w); + } + const uuids = [t.first, t.second, t.one, t.two].map((d) => d.top.uuid); + assert.equal(new Set(uuids).size, 4, uuids.join(' ')); + // A popup's cross-site frame has no place: the browser's own numbers, + // named once for the test. + assert.notDeepEqual(t.one.frame, t.two.frame); + assert.deepEqual(unplaced(r, 'tabs'), [unplacedLine('tabs', 'http://localhost:/frame')]); + // Asked for at once, the pages keep the order the test asked in, not + // the order they loaded in. + const at = r.drew.atonce; + assert.deepEqual(at.a.top, expected(r.seeds.atonce, '/draws')); + assert.deepEqual(at.b.top, expected(r.seeds.atonce, '/draws', undefined, 't1')); + assert.deepEqual(unplaced(r, 'atonce'), []); + // A frame of the page's own origin, whose document may take over the + // window of its first about:blank, is seeded in its parent document's + // place, at the path its element asks for, the first there. + assert.deepEqual(r.drew.nest, { top: expected(r.seeds.nest, '/nest'), frame: expected(r.seeds.nest, '/frame', undefined, `${keyOf(r.seeds.nest, 't0', '/nest')}f/frame\n0`) }); + assert.deepEqual(unplaced(r, 'nest'), []); + // So is one in a popup, in the popup's place: the popup's document and + // its frame both took over their windows' first about:blank. + const pop = `${r.seeds.popnest}\nt0\n/draws\n0\nw0`; + assert.deepEqual(r.drew.popnest, { + top: expected(r.seeds.popnest, '/nest', new Map([[LOADS_KEY, '{"/draws":1}']]), pop), + frame: expected(r.seeds.popnest, '/frame', undefined, `${keyOf(r.seeds.popnest, pop, '/nest')}f/frame\n0`), + }); + assert.deepEqual(unplaced(r, 'popnest'), []); + // A popup opened with noopener has no opener to place it: its own + // numbers, never its opener's, and both its documents named. + const n = r.drew.noopener; + assert.deepEqual(n.first.top, expected(r.seeds.noopener, '/draws')); + assert.notDeepEqual(n.popup.top, n.first.top); + // (Sorted: the frame's renderer may tell first.) + assert.deepEqual(unplaced(r, 'noopener').sort(), [unplacedLine('noopener', 'http://127.0.0.1:/draws'), unplacedLine('noopener', 'http://localhost:/frame')].sort()); + assert.deepEqual(r.ends, names.map((x) => [x, 'seeded'])); + // A replay answers the recording's seeds: each tab and popup draws what + // it drew in the recording; what had no place draws anew, named again. + kept = r.seeds; + for (let i = 0; i < 2; i++) { + const p = await drawRun(j, keployEnv(rep.url), ['tabs.spec.ts'], names); + for (const w of ['first', 'second']) assert.deepEqual(p.drew.tabs[w], t[w], w); + for (const w of ['one', 'two']) { + assert.deepEqual(p.drew.tabs[w].top, t[w].top, w); + assert.notDeepEqual(p.drew.tabs[w].frame, t[w].frame, w); + } + assert.deepEqual(p.drew.atonce, at); + assert.deepEqual(p.drew.nest, r.drew.nest); + assert.deepEqual(p.drew.popnest, r.drew.popnest); + assert.deepEqual(p.drew.noopener.first, n.first); + assert.notDeepEqual(p.drew.noopener.popup.top, n.popup.top); + for (const x of names) assert.deepEqual(unplaced(p, x).sort(), unplaced(r, x).sort(), x); + assert.deepEqual(p.ends, names.map((x) => [x, 'seeded'])); + } + } finally { await rep.close(); await rec.close(); } +}); + +test('seeded: a page\'s first document, before its first goto and as setContent writes it, draws in the page\'s place, the same at replay; an address keploy cannot place is shown cut to 512 characters', async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const names = ['blank', 'longpath']; + const unplaced = (r, n) => r.tests[n].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)); + try { + const r = await drawRun(j, keployEnv(rec.url), ['first.spec.ts'], names); + // The page fixture's page and one context.newPage() made, each in its + // place, in its one document: the page's draw before the first goto, then + // its own script in what setContent wrote; the other's script in what + // setContent wrote as soon as newPage returned, then a draw. + const draws = (id) => { + const d = documentAt(initScript(r.seeds.blank), 'blank', undefined, id); + return [0, 1].map(() => JSON.parse(JSON.stringify(d.run(BLANK_DRAWS)))); + }; + const [first, set] = draws('t0'); + assert.deepEqual(r.drew.blank.page, { url: 'about:blank', first, set }); + const [oset, after] = draws('t1'); + assert.deepEqual(r.drew.blank.other, { url: 'about:blank', set: oset, after }); + assert.notDeepEqual(first, oset); + assert.deepEqual(unplaced(r, 'blank'), []); + // A noopener popup at an overlong address: named once, its address cut. + const long = r.drew.longpath.url; + assert.ok(long.length > 600, long); + assert.deepEqual(unplaced(r, 'longpath'), [unplacedLine('longpath', long.slice(0, 512))]); + assert.deepEqual(r.ends, names.map((x) => [x, 'seeded'])); + kept = r.seeds; + const p = await drawRun(j, keployEnv(rep.url), ['first.spec.ts'], names); + assert.deepEqual(p.drew.blank, r.drew.blank); + assert.deepEqual(unplaced(p, 'blank'), []); + assert.deepEqual(unplaced(p, 'longpath'), [unplacedLine('longpath', p.drew.longpath.url.slice(0, 512))]); + } finally { await rep.close(); await rec.close(); } +}); + +// The engines the seeding's frame, first-document popup and own-init-script +// rows run in (CI installs all three). +const ENGINES = ['chromium', 'firefox', 'webkit']; + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: frames of a first document each draw their own numbers, never their parent's, numbered at the path their element asks for, one put where a removed frame was, one in an open shadow root and one drawn from after its removal too; one in a closed shadow root has no place; a first document's popup is named in every engine; a top document with no session storage but the page's first draws its own and is named; the test's own context init script is named as the test's`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const names = ['frames', 'popupfromblank', 'ownctxinit', 'shadow', 'nostorage', 'removed']; + const unplaced = (r, n) => r.tests[n].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)).map((l) => l.replace(/:\d+\//, ':/')).sort(); + const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); + try { + const r = await drawRun(j, env(rec.url), ['engines.spec.ts'], names); + const at = (id, where, seed = r.seeds.frames) => JSON.parse(JSON.stringify(documentAt(initScript(seed), where, new Map(), id).run(BLANK_DRAWS))); + // The first document, its srcdoc frames, a frame inside one, its + // about:blank frame, and the frame put in after that one's removal: + // each its own numbers, a frame in its parent document's place at + // the path its element asks for (about:srcdoc, about:blank), by how + // many of the parent's frames asked for that path before it. + const f = r.drew.frames; + // Firefox runs a window's first document's init scripts before that + // document has its URL, so its draws are its own engine's, not the vm + // model's: there each document draws apart, the same at replay. + const model = engine !== 'firefox'; + const top = keyOf(r.seeds.frames, 't0', 'blank'); + if (model) { + assert.deepEqual(f.top, at('t0', 'blank')); + assert.deepEqual(f.n, { url: 'about:srcdoc', draws: at(`${top}fabout:srcdoc\n0`, 'srcdoc') }); + assert.deepEqual(f.nn, { url: 'about:blank', draws: at(`${keyOf(r.seeds.frames, `${top}fabout:srcdoc\n0`, 'srcdoc')}fabout:blank\n0`, 'blank') }); + assert.deepEqual(f.s, { url: 'about:srcdoc', draws: at(`${top}fabout:srcdoc\n1`, 'srcdoc') }); + assert.deepEqual(f.b, { url: 'about:blank', draws: at(`${top}fabout:blank\n0`, 'blank') }); + assert.deepEqual(f.c, at(`${top}fabout:blank\n1`, 'blank')); + } + const docs = [f.top, f.n.draws, f.nn.draws, f.s.draws, f.b.draws, f.c].map((d) => JSON.stringify(d)); + assert.equal(new Set(docs).size, 6, docs.join('\n')); + assert.deepEqual(unplaced(r, 'frames'), []); + // A popup the first document opens with window.open('about:blank') + // draws its own in every engine, and is named: Chromium places it, + // but it has no session storage (its opener's origin is opaque); + // Firefox and WebKit begin its document before window.open returns, + // so it has no place there. + const pop = r.drew.popupfromblank; + assert.equal(pop.url, 'about:blank'); + assert.deepEqual(unplaced(r, 'popupfromblank'), [unplacedLine('popupfromblank', 'about:blank', engine === 'chromium' ? 'nostorage' : 'popup')]); + // Top documents with no session storage: the page's first draws in + // its place; about:blank loaded after it (twice), a srcdoc frame of + // its origin in the first of those (its own reason, at its address), a page served with a CSP sandbox + // (twice) and a data: URL (twice) each draw their own, named once for + // each address, with why. + const ns = r.drew.nostorage; + if (model) assert.deepEqual(ns.first, at('t0', 'blank', r.seeds.nostorage)); + assert.ok(ns.csp.every((d) => d && typeof d.uuid === 'string'), JSON.stringify(ns.csp)); + const nsDocs = [ns.first, ...ns.blank, ns.inner, ...ns.csp, ...ns.data].map((d) => JSON.stringify(d)); + assert.equal(new Set(nsDocs).size, 8, nsDocs.join('\n')); + assert.deepEqual(unplaced(r, 'nostorage'), ['about:blank', 'about:srcdoc', 'data:text/html,d', 'http://127.0.0.1:/csp'].map((a) => unplacedLine('nostorage', a, 'nostorage')).sort()); + // The test's own context init script draws first in each document of + // its page, its first about:blank too: named as the test's, with the + // reason. + // A popup that first document opens is at about:blank too, and has no + // place since its opener has none: named again there, as a page the + // app opened (a line for each address and reason). + assert.deepEqual(unplaced(r, 'ownctxinit'), [...['about:blank', 'http://127.0.0.1:/draws', 'http://localhost:/frame'].map((a) => unplacedLine('ownctxinit', a, 'own')), + unplacedLine('ownctxinit', 'about:blank', 'popup')].sort()); + // A frame in an open shadow root, which no engine lists among its + // parent's frames, is placed by its element; one in a closed shadow + // root, which its parent document cannot reach, has no place, and is + // named with the frame reason. + const sh = r.drew.shadow; + assert.equal(sh.index, -1); + if (model) assert.deepEqual(sh.open, at(`${keyOf(r.seeds.shadow, 't0', 'blank')}fabout:blank\n0`, 'blank', r.seeds.shadow)); + assert.notDeepEqual(sh.closed, sh.open); + assert.deepEqual(unplaced(r, 'shadow'), [unplacedLine('shadow', 'about:blank', 'frame')]); + // A removed frame's Math.random, first drawn from after the removal: + // its document began (as the row took its functions) while the frame + // was in the page, so it draws in that frame's place, never its + // page's first document's numbers. + assert.notDeepEqual(r.drew.removed, at('t0', 'blank', r.seeds.removed).random); + if (model) assert.deepEqual(r.drew.removed, at(`${keyOf(r.seeds.removed, 't0', 'blank')}fabout:blank\n0`, 'blank', r.seeds.removed).random); + assert.deepEqual(unplaced(r, 'removed'), []); + assert.deepEqual(r.ends, names.map((x) => [x, 'seeded'])); + // A replay draws what each placed document drew; what had no place + // draws anew, named again. + kept = r.seeds; + const p = await drawRun(j, env(rep.url), ['engines.spec.ts'], names); + assert.deepEqual(p.drew.frames, f); + assert.notDeepEqual(p.drew.popupfromblank.draws, pop.draws); + assert.deepEqual(p.drew.nostorage.first, ns.first); + assert.notDeepEqual(p.drew.nostorage.inner, ns.inner); + for (const k of ['blank', 'csp', 'data']) for (const i of [0, 1]) assert.notDeepEqual(p.drew.nostorage[k][i], ns[k][i], `${k}${i}`); + assert.notEqual(p.drew.ownctxinit.early, r.drew.ownctxinit.early); + assert.deepEqual({ ...p.drew.shadow, closed: null }, { ...sh, closed: null }); + assert.notDeepEqual(p.drew.shadow.closed, sh.closed); + assert.deepEqual(p.drew.removed, r.drew.removed); + for (const x of names) assert.deepEqual(unplaced(p, x), unplaced(r, x), x); + } finally { await rep.close(); await rec.close(); } + }); + + test(`seeded, in ${engine}: a frame of its parent's origin keeps its place whatever order the frames come or load in: frames put in another order, two at one path whose documents load in the other order, a frame put back after its removal, a nested frame at another index than its parent, and one that loads another document; a cross-site frame is placed by its page and path`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const names = ['order', 'race', 'remount', 'nested', 'navigate', 'before', 'beforewrapped', 'frameset']; + const lines = (r) => r.lines.filter((l) => l.startsWith('keploy: in ')); + const env = (url, order, slow) => ({ ...keployEnv(url), PW_BROWSER: engine, PLACES_ORDER: order, PLACES_SLOW: slow }); + try { + // The record puts a, c, w in; the server answers r1 late. + const r = await drawRun(j, env(rec.url, 'acw', '1'), ['places.spec.ts'], names); + const seed = r.seeds; + const ex = (n, path, page, loads = 0) => expected(seed[n], path, new Map([[LOADS_KEY, JSON.stringify({ [path]: loads })]]), page); + // a in /host's place, the first frame there at /fa; c and w, under + // a parent of another origin, by the page and their own paths. + const host = (n) => keyOf(seed[n], 't0', '/host'); + assert.deepEqual(r.drew.order, { + top: expected(seed.order, '/host'), + a: ex('order', '/fa', `${host('order')}f/fa\n0`), + c: ex('order', '/fc', 't0x'), + w: ex('order', '/fw', 't0x'), + }); + // r0 and r1 numbered in their elements' order, not their documents'. + const race = keyOf(seed.race, 't0', '/race'); + // Put back, they take the next numbers there as they are put back, + // whatever order their documents then load in. + assert.deepEqual(r.drew.race, { + r0: ex('race', '/fr', `${race}f/fr\n0`), r1: ex('race', '/fr', `${race}f/fr\n1`), + again0: ex('race', '/fr', `${race}f/fr\n2`), again1: ex('race', '/fr', `${race}f/fr\n3`), + }); + // The frame put back at /fa takes the next number there: its own + // numbers, never the removed one's. + assert.deepEqual(r.drew.remount, { first: ex('remount', '/fa', `${host('remount')}f/fa\n0`), again: ex('remount', '/fa', `${host('remount')}f/fa\n1`) }); + // nn, n's third frame (n is the top's second), in n's place: never + // x's numbers, though both are the first at /fa in their parents. + const nested = keyOf(seed.nested, 't0', '/nested'); + const n = `${nested}f/fn\n0`; + const inN = keyOf(seed.nested, n, '/fn'); + assert.deepEqual(r.drew.nested, { + x: ex('nested', '/fa', `${nested}f/fa\n0`), + n: ex('nested', '/fn', n), + y: ex('nested', '/fb', `${inN}f/fb\n0`), + z: ex('nested', '/fb', `${inN}f/fb\n1`), + nn: ex('nested', '/fa', `${inN}f/fa\n0`), + }); + assert.notDeepEqual(r.drew.nested.nn, r.drew.nested.x); + // A frame keeps its place in each document it loads: /fb, then /fa + // a second time in that frame. + const a = `${host('navigate')}f/fa\n0`; + assert.deepEqual(r.drew.navigate, [ex('navigate', '/fa', a), ex('navigate', '/fb', a), ex('navigate', '/fa', a, 1)]); + // b0 is numbered as it is put in, before its document loads: b1, put + // in before it later, and whose document loads first, is the second + // at /fr, wrapped or not. + for (const x of ['before', 'beforewrapped']) { + const k = keyOf(seed[x], 't0', '/before'); + assert.deepEqual(r.drew[x], { b0: ex(x, '/fr', `${k}f/fr\n0`), b1: ex(x, '/fr', `${k}f/fr\n1`) }, x); + } + // A frameset's frames are placed as an iframe is. + const set = keyOf(seed.frameset, 't0', '/set'); + assert.deepEqual(r.drew.frameset, { f0: ex('frameset', '/fa', `${set}f/fa\n0`), f1: ex('frameset', '/fa', `${set}f/fa\n1`) }); + assert.deepEqual(lines(r), []); + // Replays: w, a, c put in, and r0 answered late; then the record's + // own order. Each frame draws what it drew in the record, and no + // document is named. + kept = r.seeds; + for (const [order, slow] of [['wac', '0'], ['acw', '1']]) { + const p = await drawRun(j, env(rep.url, order, slow), ['places.spec.ts'], names); + for (const x of names) assert.deepEqual(p.drew[x], r.drew[x], `${order} ${x}`); + assert.deepEqual(lines(p), [], order); + } + } finally { await rep.close(); await rec.close(); } + }); +} + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: 300 frames in a page's markup, or put in by one script, each load and draw their own numbers, the same at replay, with no page error; sandboxed frames are never seeded, each draws its own numbers and is named with how many frames are beside it once its parent's parser has ended: three at one path, a lone one, a new one put in at a removed one's path, ones whose own script replaces the global parent before they draw, one that draws and loads while its parent's parser holds the frames after it, and a srcdoc frame inside one (named with the frames beside the sandboxed one); a page whose DOM changes often is walked only when a frame is put in`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const names = ['markup', 'script', 'sandboxed', 'lone', 'swapped', 'held', 'nested', 'cost']; + const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); + const unplaced = (r, n) => r.tests[n].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)).map((l) => l.replace(/:\d+\//, ':/')); + const line = (n, at, others) => unplacedLine(n, `http://127.0.0.1:${at}`, 'uncounted', others); + const SWAP = [['v', '/fsv'], ['s', '/fss'], ['o', '/fso']]; + const sandboxed = (run) => { + assert.deepEqual(unplaced(run, 'sandboxed'), [line('sandboxed', '/fs', 2)]); + assert.deepEqual(unplaced(run, 'lone'), [line('lone', '/fs', 0)]); + assert.deepEqual(unplaced(run, 'swapped').sort(), SWAP.map(([, at]) => line('swapped', at, 8)).sort()); + assert.deepEqual(unplaced(run, 'held'), [line('held', '/fh', 2)]); + // A srcdoc frame in a sandboxed one counts the frames beside the + // sandboxed one, as the sandboxed one does. + assert.deepEqual(unplaced(run, 'nested').sort(), [line('nested', '/fsn', 1), unplacedLine('nested', 'about:srcdoc', 'uncounted', 1)].sort()); + }; + try { + const r = await drawRun(j, env(rec.url), ['many.spec.ts'], names); + const seed = r.seeds; + // Every frame loads, each the k-th at /fm in /many, and nothing is + // thrown into the page. + for (const how of ['markup', 'script']) { + const { drew, errors } = r.drew[how]; + assert.deepEqual(errors, [], how); + assert.equal(drew.filter((x) => x !== null).length, 300, how); + const k = keyOf(seed[how], 't0', '/many'); + drew.forEach((x, i) => assert.deepEqual(x, expected(seed[how], '/fm', new Map(), `${k}f/fm\n${i}`), `${how} m${i}`)); + assert.deepEqual(unplaced(r, how), [], how); + } + // Sandboxed frames: none draws what seeding would give it (its page, + // marked, and its path); each draws its own, named once for each + // address, with how many other frames its parent holds once its + // parser has ended. Three at one path, then a lone one; a lone one, + // taken out and another put in at its path; nine whose script + // replaces parent before they draw; three where the first draws and + // loads while its parent holds no other (the count is still 2). + const { three, one } = r.drew.sandboxed; + const seeded = (n, at) => expected(seed[n], at, new Map(), 't0x'); + assert.equal(new Set(Object.values(three).map((x) => x.uuid)).size, 3); + for (const x of [...Object.values(three), one]) assert.notDeepEqual(x, seeded('sandboxed', '/fs')); + const { first, again } = r.drew.lone; + for (const x of [first, again]) assert.notDeepEqual(x, seeded('lone', '/fs')); + assert.notEqual(first.uuid, again.uuid); + const swapped = r.drew.swapped; + assert.equal(new Set(Object.values(swapped).map((x) => x.uuid)).size, 9); + for (const [k, at] of SWAP) for (let i = 0; i < 3; i++) assert.notDeepEqual(swapped[`${k}${i}`], seeded('swapped', at), `${k}${i}`); + const held = r.drew.held; + assert.equal(new Set(Object.values(held).map((x) => x.uuid)).size, 3); + for (const x of Object.values(held)) assert.notDeepEqual(x, seeded('held', '/fh')); + const nested = Object.values(r.drew.nested).flatMap((x) => [x.own, x.child]); + assert.equal(new Set(nested.map((x) => x.uuid)).size, 4); + sandboxed(r); + // A page whose DOM changes often: seeding walks the document for no + // change that puts in no frame, and once when one frame is put in. + // The times are printed, not judged: a loaded machine moves them. + console.log(`seeding cost in ${engine}, ms (least of two runs): ${JSON.stringify(Object.fromEntries(Object.entries(r.drew.cost).map(([k, t]) => [k, { seeded: Math.round(t.seeded), own: Math.round(t.own) }])))}`); + for (const [kind, t] of Object.entries(r.drew.cost)) assert.deepEqual({ walks: t.walks, framed: t.framed, own: t.ownWalks }, { walks: [0, 0], framed: [1, 1], own: [0, 0] }, kind); + // A replay draws what the record drew, frame by frame. + kept = r.seeds; + const p = await drawRun(j, env(rep.url), ['many.spec.ts'], names); + for (const how of ['markup', 'script']) { + assert.deepEqual(p.drew[how].errors, [], how); + assert.deepEqual(p.drew[how].drew, r.drew[how].drew, how); + assert.deepEqual(unplaced(p, how), [], how); + } + // Sandboxed frames draw the browser's own again, named as in the + // record. + assert.notDeepEqual(p.drew.sandboxed.one, one); + assert.notDeepEqual(p.drew.sandboxed.three.s0, three.s0); + assert.notDeepEqual(p.drew.lone.first, first); + sandboxed(p); + } finally { await rep.close(); await rec.close(); } + }); +} + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a sandboxed frame's line counts the frames beside it once its parent's parser has ended, not at its parent's load; or at the test's end when the parser has not ended by then`, async () => { + const j = journal(); + const a = await fakeAgent(j); + const d = await drawsServer(); + try { + const { tests, out } = await playwright(['parse.spec.ts'], { ...keployEnv(a.url), PW_BROWSER: engine, DRAWS_BASE: d.url, PW_TIMEOUT: '15000' }); + // unloaded: counted at the parse's end, before the test put u2 in; + // unparsed: at the test's end, u2 included. Each test then waits for + // the call its page never had answered. + for (const [n, others, call] of [['unloaded', 1, '/hold.png'], ['unparsed', 2, '/hold.js']]) { + const t = tests[n]; + assert.equal(t.status, 'timedOut', `${n}: ${t.error}\n${out}`); + const said = t.stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)).map((l) => l.replace(/:\d+\//, ':/')); + assert.deepEqual(said, [unplacedLine(n, 'http://127.0.0.1:/fs', 'uncounted', others)], n); + assert.ok(t.steps.some((s) => s.startsWith('keploy: wait for the test\'s 1 call(s) in flight') && s.endsWith(call)), `${n}: ${t.steps.join('\n')}`); + } + } finally { await d.close(); await a.close(); } + }); +} + +test('seeded: keploy answers no seed (an older recording, or an older keploy): the browser draws its own, each test says so, and its end says own', async () => { + const j = journal(); + const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); + try { + const r1 = await drawRun(j, keployEnv(a.url)); + const r2 = await drawRun(j, keployEnv(a.url)); + for (const n of ['one', 'two', 'again', 'three']) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); + assert.notDeepEqual(r1.drew.one.top, expected(r1.seeds.one, '/draws')); + for (const r of [r1, r2]) { + assert.equal(r.said, 0); + for (const n of ['one', 'two', 'again']) assert.ok(r.tests[n].stdout.split('\n').includes(unseededLine(n)), `${n}: ${r.tests[n].stdout}`); + assert.ok(!r.tests['own › three'].stdout.includes('with no seed'), r.tests['own › three'].stdout); + assert.deepEqual(r.ends, NAMES.map((n) => [n, 'own'])); + } + // A test with no page says own too: it asked for a seed and got none. + const from = j.events.length; + await playwright(['basic.spec.ts'], keployEnv(a.url)); + const ends = j.events.slice(from).filter((e) => e.agent && e.agent.path === '/agent/scope/end').map((e) => e.agent.body.random); + assert.deepEqual(ends, ['own', 'own', 'own', 'own']); + } finally { await a.close(); } +}); + +test('seeded: keploySeedRandom: false for the whole project asks no seed, says nothing, and the browser draws its own', async () => { + const j = journal(); + // An agent that answers a seed even to a begin that asked for none: the + // test that opted out still draws the browser's own numbers. + const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: '0123456789abcdef' } } : undefined)); + try { + const r1 = await drawRun(j, { ...keployEnv(a.url), PW_NOSEED: '1' }); + const r2 = await drawRun(j, { ...keployEnv(a.url), PW_NOSEED: '1' }); + for (const n of ['one', 'two', 'again', 'three']) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); + for (const r of [r1, r2]) { + assert.equal(r.said, 0); + assert.equal(r.lines.filter((l) => l.startsWith('keploy:')).length, 0, r.lines.join('\n')); + assert.deepEqual(r.begins.map((b) => [b.random, b.seed]), Array(4).fill(['own', undefined])); + assert.deepEqual(r.ends, NAMES.map((n) => [n, 'own'])); + } + } finally { await a.close(); } +}); + +test('KEPLOY_SEED_RANDOM=off, or no keploy: the test browser draws its own random numbers, and nothing is said', async () => { + const j = journal(); + const a = await fakeAgent(j); + try { + for (const env of [{ ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'off' }, { KEPLOY_MOCK_AGENT: undefined, KEPLOY_TEST_PID: undefined }]) { + const r1 = await drawRun(j, env); + const r2 = await drawRun(j, env); + for (const n of ['one', 'two', 'again', 'three']) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); + assert.equal(r1.said + r2.said, 0); + assert.equal([...r1.lines, ...r2.lines].filter((l) => l.startsWith('keploy:')).length, 0); + } + // Under keploy with it off, each begin says the browser draws its own + // and sends no seed, and each end says it did. + assert.deepEqual(calls(j).filter((c) => c.path === '/agent/scope/begin').map((c) => [c.body.random, c.body.seed]), Array(8).fill(['own', undefined])); + assert.deepEqual(calls(j).filter((c) => c.path === '/agent/scope/end').map((c) => c.body.random), Array(8).fill('own')); + } finally { await a.close(); } +}); + +test('KEPLOY_SEED_RANDOM neither on nor off fails every test, naming it, and the worker claims nothing', async () => { + const j = journal(); + const a = await fakeAgent(j); + const d = await drawsServer(); + try { + const { tests } = await playwright(['seeded.spec.ts'], { ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'yes', DRAWS_BASE: d.url }); + assert.equal(Object.keys(tests).length, 4); + for (const [k, t] of Object.entries(tests)) { + assert.equal(t.status, 'failed', k); + assert.match(t.error, /keploy: KEPLOY_SEED_RANDOM="yes" is neither on nor off; set it to off to give the test browser its own random numbers, or unset it to seed them per test/, k); + } + assert.deepEqual(calls(j), []); + } finally { await d.close(); await a.close(); } +}); diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 2ead81f..f0e995d 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -45,15 +45,33 @@ test('the client sends the token and the protocol body, and takes v1 answers', a const c = new AgentClient(a.url + '/', 'tok'); await c.claim('chromium', '/proj'); await c.begin({ name: 'g › a', file: 'tests/a.spec.ts', project: 'chromium' }); + await c.begin({ name: 'g › b', file: 'tests/a.spec.ts' }, 'seeded'); + await c.begin({ name: 'g › c', file: 'tests/a.spec.ts' }, 'own'); + await c.begin({ name: 'g › d', file: 'tests/a.spec.ts' }, 'seeded', '0123abcd'); await c.end({ name: 'g › a', file: 'tests/a.spec.ts', project: 'chromium' }, 'passed'); + await c.end({ name: 'g › d', file: 'tests/a.spec.ts' }, 'failed', 'own'); assert.deepEqual(a.calls.map((x) => [x.path, x.auth, x.body]), [ ['/agent/instance', 'Bearer tok', { key: 'chromium', dir: '/proj' }], ['/agent/scope/begin', 'Bearer tok', { name: 'g › a', file: 'tests/a.spec.ts', project: 'chromium', canSkip: true }], + ['/agent/scope/begin', 'Bearer tok', { name: 'g › b', file: 'tests/a.spec.ts', canSkip: true, random: 'seeded' }], + ['/agent/scope/begin', 'Bearer tok', { name: 'g › c', file: 'tests/a.spec.ts', canSkip: true, random: 'own' }], + ['/agent/scope/begin', 'Bearer tok', { name: 'g › d', file: 'tests/a.spec.ts', canSkip: true, random: 'seeded', seed: '0123abcd' }], ['/agent/scope/end', 'Bearer tok', { name: 'g › a', file: 'tests/a.spec.ts', project: 'chromium', outcome: 'passed' }], + ['/agent/scope/end', 'Bearer tok', { name: 'g › d', file: 'tests/a.spec.ts', outcome: 'failed', random: 'own' }], ]); } finally { await a.close(); } }); +test('a begin\'s answer gives the seed keploy answered, or none', async () => { + for (const [body, seed] of [[{ v: 1, status: 'ok', seed: '0123abcd' }, '0123abcd'], [{ v: 1, status: 'ok', seed: '' }, ''], [{ v: 1, status: 'ok' }, undefined]]) { + const a = await agent(() => ({ body })); + try { + const b = await new AgentClient(a.url, '').begin({ name: 'a', file: 'a.spec.ts' }, 'seeded', 'ffffffff'); + assert.equal(b.seed, seed); + } finally { await a.close(); } + } +}); + test('no token: no authorization header', async () => { const a = await agent(() => ({ body: { v: 1, status: 'ok' } })); try { @@ -76,6 +94,12 @@ test('every answer the client cannot trust is an error', async () => { [{ status: 302, body: '' }, /answered 302/], [{ body: { v: 1, status: 'busy' } }, /begin of a with status "busy"/], [{ body: { v: 1, status: 'ok', action: 'fail' } }, /begin of a with the unknown action "fail"/], + // A seed keploy would not keep, or one that is no string. + [{ body: { v: 1, status: 'ok', seed: 'not-hex-at-all' } }, /^keploy: the keploy agent answered the begin of a with the seed "not-hex-at-all", which is no seed keploy keeps \(8 to 64 lower-case hex digits\)$/], + [{ body: { v: 1, status: 'ok', seed: '0123ABCD' } }, /with the seed "0123ABCD", which is no seed/], + [{ body: { v: 1, status: 'ok', seed: '0123abc' } }, /with the seed "0123abc", which is no seed/], + [{ body: { v: 1, status: 'ok', seed: 12345678 } }, /with the seed "12345678", which is no seed/], + [{ body: { v: 1, status: 'ok', seed: null } }, /with the seed "null", which is no seed/], ]; for (const [answer, re] of cases) { const a = await agent(() => answer); From 68a880a7519df85baf343b551865c5a4e3a09657 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 15:25:46 +0530 Subject: [PATCH 04/73] fix(playwright): wait for a call a new document makes before its commit Firefox can report a call the new document makes as it loads (a script or an image its parser asks for) before it reports the frame's commit. Settle dropped every call of the frame at the commit, that one included, so a call still in flight at the test's end was not waited for, and the context's close cut it. Each call now gets a number in the order the browser reports it, and a frame's navigation answer keeps the number of the last call reported before it. The commit drops only that frame's calls numbered up to it: the new document can make no call before its answer. A redirect's answer is not the document's; the final answer's number replaces it. A limit: a call the old document starts after its replacement's answer (an unload handler's) is waited for too, until the test's timeout if the browser never ends it. It fails toward waiting, never toward a cut call. Rows: a unit row feeds settle the order a Firefox run reported, which fails before this change; a browser row, in all three engines, ends a test with its new document's image in flight. The 300-frames row's nested case now expects what every engine counts: the srcdoc frame inside each sandboxed frame counts no frame beside that sandboxed frame. Signed-off-by: slayerjain --- playwright/README.md | 5 ++- playwright/src/settle.ts | 38 +++++++++++----- playwright/test/fixtures/settle.spec.ts | 11 +++++ playwright/test/rows.test.mjs | 30 +++++++++++-- playwright/test/unit.test.mjs | 59 +++++++++++++++++++++++++ 5 files changed, 125 insertions(+), 18 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 98c180c..b8251d4 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -43,7 +43,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. - **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped )`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. -- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. +- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see the limits below). @@ -68,7 +68,8 @@ Any answer from the agent that this package cannot trust fails the test. That co ## Limits of this version - Settle covers the `context` and `page` fixtures. A context the test makes with `browser.newContext()` is the test's own to close. -- Settle's rows run in Chromium. Which calls the browser itself ends (a removed frame's) is proven there only; Firefox and WebKit are not rowed. +- Settle's rows run in Chromium, and its row for a call a new document makes as it loads runs in Firefox and WebKit too. Which calls the browser itself ends (a removed frame's) is proven in Chromium only. +- A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too: settle cannot tell it from a call of the new document. If the browser never reports its end, the test waits until its timeout fails it, naming the call. Settle fails toward waiting, never toward a cut call. - Seeding's rows run in Chromium. Its rows for frames, for a popup that a page's first document opens, for top documents with no session storage, and for the test's own context init script run in Firefox and WebKit too. A popup that a page's first document opens with `window.open('about:blank')` draws the browser's own numbers and is named in every engine: in Chromium it has no session storage (its opener's origin is opaque), and in Firefox and WebKit it has no place, since its document begins before `window.open` returns. - Seeding covers the pages of the `context` and `page` fixtures. These draws keep the browser's own random numbers, or the app's: - the app's server, and the test's own code in Node; diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 8b720f7..bfb4626 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -7,7 +7,9 @@ import type { BrowserContext, Frame, Page, Request } from '@playwright/test'; export class Inflight { - private readonly open = new Map(); + // Each call's number, in the order the browser reported it. + private readonly open = new Map(); + private reported = 0; private waiters: (() => void)[] = []; constructor(context: BrowserContext) { @@ -15,13 +17,16 @@ export class Inflight { let frame: Frame | null = null; // A service worker's call has no frame. try { frame = r.frame(); } catch { frame = null; } - this.open.set(r, { frame, nav: r.isNavigationRequest() }); + this.open.set(r, { frame, nav: r.isNavigationRequest(), n: ++this.reported }); }); context.on('requestfinished', (r) => this.done(r)); context.on('requestfailed', (r) => this.done(r)); // The frames whose navigation request was answered with a document, which - // the frame commits next. - const loading = new Set(); + // the frame commits next, each with the number of the last call reported + // before that answer: the new document can make no call before its + // answer, so every later one may be its own (a redirect's answer is not + // the document's, and the final answer's number replaces it). + const loading = new Map(); context.on('response', (res) => { const r = res.request(); if (!r.isNavigationRequest()) return; @@ -29,18 +34,27 @@ export class Inflight { try { f = r.frame(); } catch { f = null; } // A 204 or 205 answer, or a download, leaves the document in place. const keeps = res.status() === 204 || res.status() === 205 || /attachment/i.test(res.headers()['content-disposition'] ?? ''); - if (f && !keeps) loading.add(f); + if (f && !keeps) loading.set(f, this.reported); }); const watch = (p: Page): void => { // The browser reports no end for a call whose document was replaced // while it ran: a frame that commits a new document has cancelled - // every call the old one still had open. The new document's calls - // start after the commit. A navigation inside the document (the - // history API, a client-side route change) has no navigation request - // and replaces nothing. + // every call the old one still had open. Only the calls reported + // before the new document's answer are surely the old one's: Firefox + // can report a call the new document makes as it loads (a script or + // an image its parser asks for) before it reports the commit, and + // that call is waited for. A call the old document starts after its + // replacement's answer (an unload handler's) is waited for too: the + // browser tells the two apart by nothing settle sees, and waiting + // fails toward the test's timeout, never toward a cut call. A + // navigation inside the document (the history API, a client-side + // route change) has no navigation request and replaces nothing. p.on('framenavigated', (f) => { - if (!loading.delete(f)) return; - this.drop((o) => o.frame === f && !o.nav); + // A commit with no answer (a route change) is numbered 0: it drops + // nothing. + const answered = loading.get(f) ?? 0; + loading.delete(f); + this.drop((o) => o.frame === f && !o.nav && o.n <= answered); }); // Nor for the calls of a page that closes. Only that page's calls: a // popup that closes cancels nothing of its opener. @@ -82,7 +96,7 @@ export class Inflight { if (this.open.delete(r)) this.wake(); } - private drop(gone: (o: { frame: Frame | null; nav: boolean }) => boolean): void { + private drop(gone: (o: { frame: Frame | null; nav: boolean; n: number }) => boolean): void { let n = 0; for (const [r, o] of this.open) { if (gone(o)) { this.open.delete(r); n++; } diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index cec0ce9..8540681 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -23,6 +23,17 @@ test('a call its page cancelled by loading another document', async ({ page }) = await reached(); await page.goto(`${base}/page?next`); }); +test('a call its new document makes as it loads', async ({ page }) => { + // The document's image, which its parser asks for as it loads, is still + // in flight when the body ends. The body ends once the test has seen the + // image's request and the document's end, so the call settle names is the + // image alone. By then the test has seen the frame's commit too (goto + // resolves after it), in whichever order the browser reported the two. + const sent = page.waitForRequest(/\/slow$/); + const doc = await page.goto(`${base}/loads`, { waitUntil: 'commit' }); + await sent; + await doc?.finished(); +}); test('a call in flight across a route change inside the page', async ({ page }) => { await page.goto(`${base}/page`); const sent = page.waitForRequest(/\/slow$/); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index e4ba7a6..e7be4b5 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -60,6 +60,7 @@ async function pageServer(j) { const seen = new Promise((r) => { saw = r; }); const srv = http.createServer((req, res) => { if (req.url === '/page') { res.writeHead(200, { 'content-type': 'text/html' }); res.end('page'); return; } + if (req.url === '/loads') { res.writeHead(200, { 'content-type': 'text/html' }); res.end(''); return; } if (req.url === '/release') { release(); res.end('ok'); return; } if (req.url === '/seen') { void seen.then(() => res.end('ok')); return; } // A service worker that calls /slow as it installs. @@ -384,6 +385,25 @@ test('settle: a call cancelled by its page loading another document is not waite } finally { await p.close(); await a.close(); } }); +// Firefox can report the new document's first call before the frame's +// commit; the commit cancels only the old document's calls, so this one is +// still waited for, in every engine. +for (const engine of ['chromium', 'firefox', 'webkit']) { + test(`settle, in ${engine}: a call the new document makes as it loads is waited for, whichever the browser reports first, the call or its frame's commit`, async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = 'a call its new document makes as it loads'; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000', PW_BROWSER: engine }); + const t = tests[title]; + assert.equal(t.status, 'timedOut', out); + const named = /^keploy: wait for the test's 1 call\(s\) in flight before its browser context closes: GET http:\/\/127\.0\.0\.1:\d+\/slow$/; + assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n')); + } finally { await p.close(); await a.close(); } + }); +} + test('settle: a route change inside the page cancels nothing, so its call in flight is waited for', async () => { const j = journal(); const a = await fakeAgent(j); @@ -1064,7 +1084,7 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: 300 frames in a page's markup, or put in by one script, each load and draw their own numbers, the same at replay, with no page error; sandboxed frames are never seeded, each draws its own numbers and is named with how many frames are beside it once its parent's parser has ended: three at one path, a lone one, a new one put in at a removed one's path, ones whose own script replaces the global parent before they draw, one that draws and loads while its parent's parser holds the frames after it, and a srcdoc frame inside one (named with the frames beside the sandboxed one); a page whose DOM changes often is walked only when a frame is put in`, async () => { + test(`seeded, in ${engine}: 300 frames in a page's markup, or put in by one script, each load and draw their own numbers, the same at replay, with no page error; sandboxed frames are never seeded, each draws its own numbers and is named with how many frames are beside it once its parent's parser has ended: three at one path, a lone one, a new one put in at a removed one's path, ones whose own script replaces the global parent before they draw, one that draws and loads while its parent's parser holds the frames after it, and a srcdoc frame inside one (sandboxed too, alone under its parent); a page whose DOM changes often is walked only when a frame is put in`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1080,9 +1100,11 @@ for (const engine of ENGINES) { assert.deepEqual(unplaced(run, 'lone'), [line('lone', '/fs', 0)]); assert.deepEqual(unplaced(run, 'swapped').sort(), SWAP.map(([, at]) => line('swapped', at, 8)).sort()); assert.deepEqual(unplaced(run, 'held'), [line('held', '/fh', 2)]); - // A srcdoc frame in a sandboxed one counts the frames beside the - // sandboxed one, as the sandboxed one does. - assert.deepEqual(unplaced(run, 'nested').sort(), [line('nested', '/fsn', 1), unplacedLine('nested', 'about:srcdoc', 'uncounted', 1)].sort()); + // A srcdoc frame in a sandboxed one is sandboxed too, so its origin + // is an opaque one of its own, not its parent's: it is itself a frame + // under a parent of another origin with no session storage, alone + // under its parent, and its line counts no other frame (every engine). + assert.deepEqual(unplaced(run, 'nested').sort(), [line('nested', '/fsn', 1), unplacedLine('nested', 'about:srcdoc', 'uncounted', 0)].sort()); }; try { const r = await drawRun(j, env(rec.url), ['many.spec.ts'], names); diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index f0e995d..4c04835 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -337,6 +337,65 @@ test('settle: a frame that is gone drops only its own calls, its own document re assert.equal(settled, true, 'settle is woken'); }); +// A frame's request, its navigation's, and the navigation's answer. +function frameRequest(frame, url, nav = false) { + return { frame: () => frame, isNavigationRequest: () => nav, method: () => 'GET', url: () => `http://localhost${url}` }; +} +function answer(request, status = 200, headers = {}) { + return { request: () => request, status: () => status, headers: () => headers }; +} + +// Firefox can report a call the new document makes as it loads (here an +// image its parser asks for) before it reports the frame's commit: the +// order a Firefox run gave, made exact. The commit drops only the calls +// reported before the new document's answer, the old document's, one +// started while the navigation ran included; the new document's call is +// waited for. A redirect's answer is not the document's: a call reported +// after it, before the final answer, is still the old document's. +test('settle: a frame\'s commit drops only the calls reported before its new document\'s answer (Firefox reports the new document\'s first call before the commit; a redirect\'s answer is not the document\'s)', async () => { + for (const redirect of [false, true]) { + const c = redirect ? 'redirected' : 'answered at once'; + const ctx = fakeContext(); + const inflight = new Inflight(ctx); + const page = new EventEmitter(); + const frame = { page: () => page }; + ctx.emit('page', page); + const old = frameRequest(frame, '/slow'); + ctx.emit('request', old); + const nav = frameRequest(frame, '/next', true); + ctx.emit('request', nav); + // The old document's poller fires while the navigation runs. + const during = frameRequest(frame, '/poll'); + ctx.emit('request', during); + let final = nav; + let late = null; + if (redirect) { + ctx.emit('response', answer(nav, 302, { location: '/moved' })); + ctx.emit('requestfinished', nav); + late = frameRequest(frame, '/poll2'); + ctx.emit('request', late); + final = frameRequest(frame, '/moved', true); + ctx.emit('request', final); + } + ctx.emit('response', answer(final)); + ctx.emit('requestfinished', final); + const mine = frameRequest(frame, '/hold.png'); + ctx.emit('request', mine); + page.emit('framenavigated', frame); + assert.deepEqual(inflight.describe(), ['GET http://localhost/hold.png'], `${c}: only the new document's call is left in flight`); + let settled = false; + inflight.settled().then(() => { settled = true; }); + await ticks(); + assert.equal(settled, false, `${c}: settle waits for the new document's call`); + // A route change inside the new document (no answer) drops nothing. + page.emit('framenavigated', frame); + assert.equal(inflight.size, 1, `${c}: a commit with no answer drops nothing`); + ctx.emit('requestfinished', mine); + await ticks(); + assert.equal(settled, true, `${c}: settle is woken once the new document's call ends`); + } +}); + test('decide: inert, refused, active', () => { const never = () => { throw new Error('runsInside called'); }; assert.deepEqual(decide({}, 10, never), { mode: 'inert' }); From 7b8ec1f6840ed48326fceaaa5f9d1574bf732df7 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 17:01:44 +0530 Subject: [PATCH 05/73] feat(playwright): seed the test process's Math.random per test A suite that makes its test data in the Node worker that runs the test (a lead named with Math.random()) made other data on each replay, while a mocked dependency answered with the recording's, so the test's own assertion failed. Only the test browser was seeded. Under keploy, the test process's Math.random now draws from the seed keploy answered the test's begin with: the one a record keeps with the test's recording and a replay answers back. It has a stream of its own (sfc32 over the SHA-256 of the seed and the stream's name, pinned by a row to a reference made outside the package), so it does not move the browser's. It is seated from the begin to the end: the test's beforeEach and afterEach hooks and its fixtures draw from it too. At the end the worker's own Math.random is back, unless the test put its own stub there, which it keeps. Outside every test (beforeAll, afterAll, a file as it loads) the worker draws its own, as does its crypto. What a worker does once, in whichever test first needs it, is kept out of that test's stream, so a test draws the same numbers whichever tests its worker ran before it: replayed alone, after keploy set an earlier test aside, or in a new worker after a test failed. - Playwright parses a file's source map the first time a test's stack reaches that file, and source-map's sort draws Math.random. While a test is seeded, Error.prepareStackTrace holds a wrapper of the function last set there, and a draw made inside it is the worker's own. At the test's end the property is a plain one again, holding the function last set, unless the test defined it itself. - A worker fixture registered with this package's test.extend or test.use, or those of a test type extend makes from it, draws the worker's own Math.random from its start until it calls use(), or fails before. A test fixture runs as written, and Playwright still names a fixture at the spec's line that called extend or use. A worker fixture registered through @playwright/test's own test or a test type mergeTests returned, and a library that draws on its first use inside a test, are not covered: the README says so. The seeding line is now said at the first seeded begin, not the first seeded context, so a worker whose tests open no page says it too. The line for a begin keploy answered with no seed names the test process. With seeding off (KEPLOY_SEED_RANDOM=off, or keploySeedRandom: false), the worker says once what that costs. The README states that the test process's clock is keploy's run clock, not a seeded one, and that one stream serves all of a test's draws in the test process. Rows: a test's draws in its beforeEach, body and afterEach are its recording's at replay, and another test's, an opted-out test's and code outside tests are not; no seed answered, or seeding off, each saying so; tests in one worker whose helper file calls test.step or reads an Error's stack, replayed alone, with the first set aside, and after a record whose first test failed; a worker fixture set up in the first test that asks for it, through extend and through use, replayed alone and with the first set aside, named at the spec's line; the worker's own Math.random back after a test that throws, one that times out, and one that defined Error.prepareStackTrace itself. Signed-off-by: slayerjain --- playwright/README.md | 23 +- playwright/src/index.ts | 149 +++++++++-- playwright/src/random.ts | 167 ++++++++++-- playwright/test/fixtures/process-ends.spec.ts | 12 + playwright/test/fixtures/process-helper.ts | 13 + .../test/fixtures/process-order.spec.ts | 15 ++ .../test/fixtures/process-worker.spec.ts | 49 ++++ playwright/test/fixtures/process.spec.ts | 30 +++ playwright/test/fixtures/steps-reporter.cjs | 6 +- playwright/test/random.test.mjs | 171 +++++++++++- playwright/test/rows.test.mjs | 253 +++++++++++++++++- 11 files changed, 830 insertions(+), 58 deletions(-) create mode 100644 playwright/test/fixtures/process-ends.spec.ts create mode 100644 playwright/test/fixtures/process-helper.ts create mode 100644 playwright/test/fixtures/process-order.spec.ts create mode 100644 playwright/test/fixtures/process-worker.spec.ts create mode 100644 playwright/test/fixtures/process.spec.ts diff --git a/playwright/README.md b/playwright/README.md index b8251d4..c8a05d5 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -49,7 +49,12 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Seeded random numbers.** Each test's browser draws its random numbers (`Math.random`, `crypto.getRandomValues`, `crypto.randomUUID`) from a seeded stream, so a replay draws the numbers its recording drew. A page that picks what to load at random (an A/B coin, such as stripe.js's at each page load) then makes the same calls in replay as in its recording, which keploy can answer. - **Where the seed comes from.** Each test's begin sends a seed drawn afresh (`seed`, 128 bits as hex). keploy answers the seed to draw from. A record answers the begin's own, and keeps it in the test's mapping entry (`seed:`, beside its `at:`). A replay (and keploy's auto-replay) answers the one the recording kept. So two tests draw different numbers, and recording a test again draws new ones: an ID the browser makes is not sent twice to a real server. The seed is not derived from the test's name, so a test whose entry is renamed keeps its seed with the rest of the entry. - - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser then draws its own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser draws its own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. + - **The test process.** The Node worker that runs the test draws `Math.random` from the same seed, from its own stream, from the test's begin to its end: its `beforeEach` and `afterEach` hooks and its fixtures included. So data a suite makes in the test process (a lead named `Lead ${Math.random().toString(36).slice(2, 7)}`) is in replay the data its recording made, and a mocked dependency's answer still matches the test's own assertion. At the test's end the worker's own `Math.random` is back, unless the test put its own stub there, which it keeps. + - A draw made while a stack trace is prepared is the worker's own, not the test's. Playwright maps stack frames through source maps, and the first time a test's stack reaches a file (a helper or page-object file's `test.step`, an `expect`, a read of an Error's stack), it parses that file's map with a sort that draws `Math.random`. A worker parses each file once, in whichever test reaches it first. So those draws do not depend on which tests its worker ran before it. + - To tell those draws apart, every stack trace is prepared through a wrapper while a test is seeded: `Error.prepareStackTrace` reads back a wrapper of the function last set there (the same one on each read), and setting that wrapper sets its function. At the test's end `Error.prepareStackTrace` is again a plain property holding the function last set, unless the test defined that property itself, which it keeps. + - A worker fixture registered with this package's `test.extend` or `test.use` draws the worker's own `Math.random` while it is set up, until it calls `use()` (or fails before). Playwright sets a worker fixture up in the first test that asks for it, after that test's seeding began, unless it is `auto`. Its draws (a test account made once for the worker's tests) would otherwise come from that test's stream and shift the test's own. A test fixture is the test's, and draws from its stream. Anything else that draws while a worker fixture is set up (a timer or an event handler of the test's) draws the worker's own too. + - With both, a test draws the same numbers whichever tests its worker ran before it: replayed alone, after keploy set an earlier test aside, or in a new worker after a test failed. The limits below name what a worker does once, in whichever test first needs it, that neither covers. + - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser's pages change: not the app, its server or the browser's workers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; @@ -63,6 +68,8 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - for the whole run: `KEPLOY_SEED_RANDOM=off` (or `0`, `false`; unset, `on`, `1` or `true` seed; any other value fails every test, naming the variable); - for a project, a file or a test: the option `keploySeedRandom: false`, in the config's or a project's `use`, or with `test.use({ keploySeedRandom: false })`. Its type is `KeployOptions`: `defineConfig({ use: { keploySeedRandom: false } })`. + The cost, which the first such test of a worker prints: the test process and the browser draw their own numbers each run, so data a test makes from them differs from its recording's while a mocked dependency answers with the recording's. A test that checks one against the other fails, or keploy sets it aside, and a page's random choice may make calls the recording never saw. + Any answer from the agent that this package cannot trust fails the test. That covers an agent it cannot reach, an error answer, an unreadable answer, and an answer that does not speak scope protocol v1. Nothing is retried. ## Limits of this version @@ -72,7 +79,10 @@ Any answer from the agent that this package cannot trust fails the test. That co - A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too: settle cannot tell it from a call of the new document. If the browser never reports its end, the test waits until its timeout fails it, naming the call. Settle fails toward waiting, never toward a cut call. - Seeding's rows run in Chromium. Its rows for frames, for a popup that a page's first document opens, for top documents with no session storage, and for the test's own context init script run in Firefox and WebKit too. A popup that a page's first document opens with `window.open('about:blank')` draws the browser's own numbers and is named in every engine: in Chromium it has no session storage (its opener's origin is opaque), and in Firefox and WebKit it has no place, since its document begins before `window.open` returns. - Seeding covers the pages of the `context` and `page` fixtures. These draws keep the browser's own random numbers, or the app's: - - the app's server, and the test's own code in Node; + - the app's server; + - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: + - a library that seeds itself from them as it loads: `@faker-js/faker`'s default instance seeds its generator from `Math.random` as its module loads, so every value it makes differs from its recording's; + - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - Web Workers, Shared Workers and Service Workers, since an init script does not reach them; - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; - `crypto.subtle`'s keys. @@ -83,6 +93,13 @@ Any answer from the agent that this package cannot trust fails the test. That co - a frame's number at its path is how many frames its parent document had put in at that path before it, so frames at one path that the page or a third-party script puts in, in an order or a number that varies run to run (as their scripts load, in network order; a re-render that makes a varying number of frames), trade numbers, unnamed. Chat and support widgets often put in `about:blank` frames as their scripts load; - frames under a parent of another origin are counted by their origin in the tab, so two of one origin at one path take their numbers in the order their documents begin, which may change at replay; - a frame put into a shadow root already in the page is not seen as it is put in, since the parent document watches only the nodes added to or removed from it outside its shadow roots: the frame is numbered at the document's next numbering (when the app next puts a frame in outside its shadow roots, or when a document of a frame not yet numbered, its own included, begins), so one put there before another frame at its path, at a moment that varies run to run, may take that frame's number. + + In the test process, what a worker does once, in whichever test first needs it, draws from that test's stream and shifts the test's later draws, so a replay where another test is the first to need it (one test alone, or after keploy set an earlier test aside) draws other numbers than the recording. This package keeps two such cases out of the test's stream: source maps parsed while a stack trace is prepared, and the setup of worker fixtures registered with its own `test.extend` or `test.use`. These it does not: + - a worker fixture registered through `@playwright/test`'s own `test`, or through a test type that `mergeTests` returned, whose `extend` and `use` are Playwright's own. The usual case is fixtures built on `@playwright/test` and joined to this package's `test` with `mergeTests`. Register worker fixtures with this package's `test.extend`, or with `extend` on a test type made from it that way; + - a library that draws `Math.random` the first time it is used, inside a test (a pool or a cache it builds on first use). + + In the test process one stream serves all of a test's `Math.random` draws, its own and those of the code it calls (Playwright's multipart form boundary among them), so a draw made a varying number of times (a retry loop's jitter) shifts the draws after it. +- The test process's clock is not seeded: it is keploy's run clock. A replayed test begins at its recorded begin, or later where the replay is behind, and its clock runs from there. So a value the test makes from the time during the test (`Date.now()`, `new Date()`) is off its recording's by as much as the replay ran faster or slower to that read, while a mocked dependency answers with the recording's: a name stamped with the time in milliseconds differs, and a test that checks it against the dependency's answer fails or is set aside. - Seeding cannot place these documents, so they draw the browser's own numbers and are named (above): - a popup opened with noopener, as a link with `target="_blank"` opens one, or opened other than by `window.open` (a form, or a link with `rel="opener"`, that targets a new window); - a popup at another origin than its opener's page, or whose opener's page has loaded another document by the time the popup's begins, or whose opener the seeding could not place (below); @@ -98,7 +115,7 @@ Any answer from the agent that this package cannot trust fails the test. That co - Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. - The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim. Each test is skipped at its begin, after the group's hooks have started; skipping the whole group before its hooks run needs a way to ask keploy for the set-aside tests first, which this version does not have. - `given` needs the agent route `POST /agent/scope/given` (keploy/keploy#5016). keploy v3.6.192 and later serve it; with the enterprise keploy, that is an enterprise release on keploy v3.6.192 or later. An older keploy answers that route with a plain 404, and `given` fails the test with `keploy: this keploy agent answered 404 to POST /agent/scope/given, so it cannot take the given value : given needs keploy v3.6.192 or later (keploy/keploy#5016), or an enterprise release on keploy v3.6.192 or later; an older keploy answers 404 there`. Claim, begin, settle and end do not need that route. -- Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser draws its own random numbers and prints the line that says so (above). Claim, begin, settle and end work with it as before. +- Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser and test process draw their own random numbers and print the line that says so (above). Claim, begin, settle and end work with it as before. ## Releasing diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 7c65a01..4f58b6c 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -43,8 +43,11 @@ // seed keploy answered its begin with, which a record keeps with the // test's recording, so a replay's pages make the calls the recording's // made; each page draws its own, named by its place in the test's -// order; KEPLOY_SEED_RANDOM=off or the option keploySeedRandom: false -// turns it off. +// order; the test process draws Math.random from that seed too, from +// the test's begin to its end, so data the test makes there (a name +// stamped with Math.random()) is the recording's; KEPLOY_SEED_RANDOM=off +// or the option keploySeedRandom: false turns it off, and the worker +// says once what that costs. // Any answer of the agent this package cannot trust fails the test. import { test as base, expect } from '@playwright/test'; @@ -55,7 +58,7 @@ import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError, RANDOM_OWN, RANDOM_ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; -import { SEEDED_LINE, besideOf, freshSeed, initScript, othersNow, pageAddress, placePages, seedingFromEnv, unplacedLine, unseededLine, type Unplaced } from './random'; +import { OFF_LINE, SEEDED_LINE, besideOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, type Unplaced } from './random'; import { Inflight } from './settle'; export { expect, KeployError }; @@ -75,8 +78,10 @@ let projectDir = ''; let open: TestRef | null = null; // Whether the environment leaves the test browser's seeding on. let seedingOn = false; -// Whether this worker has said that it seeds the test browser. +// Whether this worker has said that it seeds each test's random numbers. let saidSeeded = false; +// Whether this worker has said what a test with seeding off costs. +let saidOff = false; // How many contexts this worker seeded: each one's report function has its // own name. let reports = 0; @@ -111,11 +116,11 @@ function randomDone(s: PageSeeding): Random | undefined { /** @keploy/playwright's options, for a config's or a file's use. */ export type KeployOptions = { /** - * Seed the test browser's random numbers per test under keploy (default - * true), so a replay's pages draw what the recording's drew. Set false, - * for record and replay alike, to give a project, a file or a test the - * browser's own random numbers; KEPLOY_SEED_RANDOM=off does so for the - * whole run. + * Seed the test browser's random numbers and the test process's + * Math.random per test under keploy (default true), so a replay draws + * what the recording drew. Set false, for record and replay alike, to give + * a project, a file or a test their own random numbers; + * KEPLOY_SEED_RANDOM=off does so for the whole run. */ keploySeedRandom: boolean; }; @@ -123,7 +128,7 @@ export type KeployOptions = { type WorkerFixtures = { keployWorker: void }; type TestFixtures = { keployTest: void } & KeployOptions; -export const test = base.extend({ +const keployTestType = base.extend({ keployWorker: [async ({}, use, workerInfo) => { const j = joining(); if (j.mode === 'refused') throw new KeployError(j.refusal); @@ -208,12 +213,6 @@ export const test = base.extend({ const placing = placePages(context, mark); restore = placing.restore; opened = placing.opened; - if (!saidSeeded) { - saidSeeded = true; - process.stdout.write(`${SEEDED_LINE}\n`); - } - } else if (s.asked) { - process.stdout.write(`${unseededLine(name)}\n`); } const seeded = s.seed !== ''; const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; @@ -267,9 +266,27 @@ export const test = base.extend({ return; } const s: PageSeeding = { asked, seed: asked ? b.seed ?? '' : '', seededPage: false, ownPage: false }; + if (s.seed !== '') { + if (!saidSeeded) { + saidSeeded = true; + process.stdout.write(`${SEEDED_LINE}\n`); + } + } else if (asked) { + process.stdout.write(`${unseededLine(ref.name)}\n`); + } else if (!saidOff) { + saidOff = true; + process.stdout.write(`${OFF_LINE}\n`); + } open = ref; seeding = s; - await use(); + // The test process draws Math.random from the test's seed from here, its + // beforeEach hooks included, until its afterEach hooks and fixtures end. + const unseat = s.seed !== '' ? seatRandom(processRandom(s.seed)) : null; + try { + await use(); + } finally { + unseat?.(); + } open = null; seeding = null; const e = await c.end(ref, outcomeOf(testInfo), randomDone(s)); @@ -277,6 +294,104 @@ export const test = base.extend({ }, { auto: true }], }); +/** + * The test type specs import. Its extend, and use, give each worker fixture + * they register the worker's own Math.random while it is set up, until it + * calls use(), and so do those of each test type extend makes from it. A + * worker fixture is set up in the first test that asks for it (unless it is + * auto), after that test's seeding began, so its draws would otherwise come + * from that test's stream and shift the test's own: a test would draw other + * numbers when replayed alone, or after keploy set an earlier test aside. + */ +export const test = onWorkerSide(keployTestType); + +type Registering = (fixtures: Record) => unknown; + +function onWorkerSide(t: T): T { + const o = t as unknown as { extend: Registering; use: Registering }; + const extend = o.extend; + const use = o.use; + o.extend = function extend_(fixtures: Record): unknown { + return onWorkerSide(asCaller(extend_, () => extend(workerSideAll(fixtures))) as object); + }; + o.use = function use_(fixtures: Record): unknown { + return asCaller(use_, () => use(workerSideAll(fixtures))); + }; + return t; +} + +// fixtures with each fixture function, alone or first in a [function, +// options] pair, made workerSide; every other value, and a symbol's (a test +// type's, which Playwright refuses to extend with), as given. +function workerSideAll(fixtures: Record): Record { + const entries = Object.entries(fixtures); + const out: Record = { ...fixtures }; + for (const [name, v] of entries) { + if (typeof v === 'function') out[name] = workerSide(v); + else if (Array.isArray(v) && typeof v[0] === 'function' && typeof v[1] === 'object') out[name] = [workerSide(v[0]), ...v.slice(1)]; + } + return out; +} + +/** + * fn, a fixture function, that while it sets up a worker fixture (until it + * calls use(), or fails before) draws the worker's own Math.random. A worker + * fixture is told of its worker (WorkerInfo), a test fixture of its test + * (TestInfo, with its testId): a test fixture is the test's and runs as it + * is. Playwright reads a fixture's parameters from its source, so the + * wrapper reads as fn's. + */ +function workerSide(fn: Function): Function { + const w = { async fixture(this: unknown, args: unknown, use: (v: unknown) => Promise, info: unknown): Promise { + if (typeof (info as { testId?: unknown } | null)?.testId === 'string') return await Reflect.apply(fn, this, [args, use, info]); + const back = setAside(); + try { + return await Reflect.apply(fn, this, [args, async (v: unknown) => { back(); await use(v); }, info]); + } finally { + back(); + } + } }.fixture; + Object.defineProperty(w, 'toString', { value: () => fn.toString(), enumerable: false, writable: true, configurable: true }); + return w; +} + +/** + * Runs f, a call of Playwright's own extend or use, as if made by the code + * that called wrapper. Playwright takes the file and line it names a fixture + * by, in its errors and reports, from its caller's frame: it sets a function + * in Error.prepareStackTrace and reads the second frame of a stack it + * captures, which here is wrapper's, in this package. While f runs, a + * function set there gets that code's frame in wrapper's place. + */ +function asCaller(wrapper: Function, f: () => R): R { + const E = Error as { prepareStackTrace?: unknown; stackTraceLimit?: number }; + const was = Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'); + const limit = E.stackTraceLimit; + try { + // The frame of the code that called wrapper, as V8 gives it (one frame, + // whatever limit the suite set). + Object.defineProperty(E, 'prepareStackTrace', { value: (_e: unknown, frames: unknown[]) => frames, writable: true, enumerable: false, configurable: true }); + E.stackTraceLimit = 1; + const o: { stack?: unknown } = {}; + Error.captureStackTrace(o, wrapper); + const caller = (o.stack as unknown[])[0]; + // Playwright sets its function, reads, and sets back the one it found; + // each is called, if at all, only before this puts the property back. + let now: unknown; + Object.defineProperty(E, 'prepareStackTrace', { + get: () => now, + set: (v: unknown) => { now = (e: unknown, frames: unknown[]) => Reflect.apply(v as Function, E, [e, [frames[0], caller, ...frames.slice(2)]]); }, + enumerable: false, + configurable: true, + }); + return f(); + } finally { + E.stackTraceLimit = limit; + if (was === undefined) delete E.prepareStackTrace; + else Object.defineProperty(E, 'prepareStackTrace', was); + } +} + /** * given tells keploy a value the open test made itself, as an incoming * request's value is for a server test: keploy follows it where the test's diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 66e2aba..cade79b 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1,18 +1,21 @@ -// Seeded randomness for the test browser. A page that draws random numbers -// (stripe.js picks one of two builds from crypto.randomUUID() at each page -// load) makes other calls each run, and no recording can answer a call it -// never saw. So under keploy each test's browser draws from a seeded stream -// instead. Each test's begin sends a seed drawn afresh, and keploy answers -// the seed to draw from: a record keeps the begin's with the test's -// recording, and a replay answers the recording's. So record and replay -// draw the same numbers and pick the same side, two tests draw different -// ones, and a record made again draws new ones (a browser-made ID is not sent -// twice to a real server). Each page of a test draws its own numbers, so two -// tabs do not make one ID. Only the test browser's pages change; the app, -// its server and the browser's workers do not. +// Seeded randomness for the test browser and the test process. A page that +// draws random numbers (stripe.js picks one of two builds from +// crypto.randomUUID() at each page load) makes other calls each run, and no +// recording can answer a call it never saw. A suite that makes its test data +// in the test process (a lead named with Math.random()) sends other data each +// run, and a mocked dependency answers with the recording's. So under keploy +// each test's browser and its test process draw from seeded streams instead. +// Each test's begin sends a seed drawn afresh, and keploy answers the seed to +// draw from: a record keeps the begin's with the test's recording, and a +// replay answers the recording's. So record and replay draw the same numbers, +// pick the same side and make the same data, two tests draw different ones, +// and a record made again draws new ones (a browser-made ID is not sent twice +// to a real server). Each page of a test draws its own numbers, so two tabs +// do not make one ID. Only the test browser's pages and the test process's +// Math.random change; the app, its server and the browser's workers do not. // // On by default; KEPLOY_SEED_RANDOM=off or the fixture option -// keploySeedRandom: false turns it off. +// keploySeedRandom: false turns it off, and the worker says what that costs. import type { BrowserContext, Frame, Page } from '@playwright/test'; import crypto from 'node:crypto'; @@ -32,23 +35,36 @@ export function seedingFromEnv(env: NodeJS.ProcessEnv): Seeding { if (l === 'off' || l === '0' || l === 'false') return { mode: 'off' }; return { mode: 'refused', - refusal: `keploy: ${ENV_SEED_RANDOM}=${JSON.stringify(v)} is neither on nor off; set it to off to give the test browser its own random numbers, ` + + refusal: `keploy: ${ENV_SEED_RANDOM}=${JSON.stringify(v)} is neither on nor off; set it to off to give the test browser and the test process their own random numbers, ` + 'or unset it to seed them per test', }; } -/** The line a worker shows once, at the first test whose browser it seeds. */ -export const SEEDED_LINE = 'keploy: the test browser\'s random numbers (Math.random, crypto.getRandomValues, crypto.randomUUID) are seeded per test ' + - 'from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew; ' + - 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named). ' + +/** The line a worker shows once, at the first test it seeds. */ +export const SEEDED_LINE = 'keploy: each test\'s random numbers are seeded from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew: ' + + 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random, from the test\'s begin to its end ' + + '(its beforeEach and afterEach hooks too). ' + + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + + 'the test process keeps its own crypto, and its own Math.random outside every test (beforeAll, afterAll, a file as it loads) ' + + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + `To turn it off, record and replay with ${ENV_SEED_RANDOM}=off, or set use: { ${OPTION_SEED_RANDOM}: false } in the Playwright config.`; +/** + * The line a worker shows once, at the first test it runs with seeding off: + * what the test's own random numbers cost a replay. + */ +export const OFF_LINE = `keploy: random numbers are not seeded (${ENV_SEED_RANDOM}=off, or ${OPTION_SEED_RANDOM}: false), ` + + 'so the test process and the test browser draw their own each run: data a test makes from them (a name, an email, an ID) differs from its recording\'s ' + + 'while a mocked dependency answers with the recording\'s, so a test that checks one against the other fails or is set aside, ' + + 'and a page script\'s random choice may make calls the recording never saw. ' + + `Unset ${ENV_SEED_RANDOM} and ${OPTION_SEED_RANDOM} to seed them per test.`; + /** * The line a test shows when it asked keploy for a seed and got none, so its - * pages draw the browser's own random numbers. + * pages and its test process draw their own random numbers. */ export function unseededLine(name: string): string { - return `keploy: keploy answered the begin of ${name} with no seed, so its browser draws its own random numbers: ` + + return `keploy: keploy answered the begin of ${name} with no seed, so its browser and its test process draw their own random numbers: ` + 'its recording kept none (it was recorded before keploy kept seeds, or with seeding off: record the test again with seeding on, the default, to seed it), ' + 'or the test was pulled from a Keploy branch, which keeps no seed, or this keploy predates seeds.'; } @@ -137,6 +153,117 @@ export function freshSeed(): string { return crypto.randomBytes(16).toString('hex'); } +/** + * The stream a test's process draws Math.random from under seed: sfc32 over + * the SHA-256 of the seed and the stream's name, 53 bits a draw, as Node's + * own Math.random gives. A replay draws what its recording drew only while + * this stays the same: a recording keeps the seed, not the numbers. + */ +export function processRandom(seed: string): () => number { + const h = crypto.createHash('sha256').update(`${seed}\ntest process\nMath.random`).digest(); + let a = h.readUInt32LE(0), b = h.readUInt32LE(4), c = h.readUInt32LE(8), d = h.readUInt32LE(12); + const next = (): number => { + let t = (a + b) | 0; + a = b ^ (b >>> 9); + b = (c + (c << 3)) | 0; + c = (c << 21) | (c >>> 11); + d = (d + 1) | 0; + t = (t + d) | 0; + c = (c + t) | 0; + return t >>> 0; + }; + for (let i = 0; i < 15; i++) next(); + return () => ((next() >>> 5) * 67108864 + (next() >>> 6)) / 9007199254740992; +} + +/** + * Puts draw in Math.random's place in this process until the returned + * function puts the process's own back. That one puts it back only while + * Math.random is still draw: a test that put its own stub there keeps it. + * + * A draw made while a stack trace is prepared is not the test's, so it is + * the process's own. Playwright maps each stack frame through its file's + * source map (source-map-support's Error.prepareStackTrace, and the one it + * sets for a moment to read where test.step or test was called), and + * source-map parses a file's map the first time one of its frames is + * mapped, sorting it with pivots from Math.random. A worker parses each + * file once, in the first test whose stack reaches it (a helper file's + * test.step, an expect, an Error's stack read), so those draws would make + * a test's numbers depend on which tests its worker ran before it: a replay + * of one test alone, or one after keploy set an earlier test aside, would + * draw other numbers than its recording. Every stack trace is prepared + * through Error.prepareStackTrace, so while seated that property holds the + * function last set there wrapped, and reading it gives the wrapper (one + * for each function, so two reads are the same); setting a wrapper sets + * its function. Putting back gives Error.prepareStackTrace its own + * property again, holding the function last set: a test that defined the + * property itself keeps its definition. + */ +export function seatRandom(draw: () => number): () => void { + const own = Math.random; + // How many stack traces are being prepared now, one inside another. + let preparing = 0; + // What setAside holds for this seat: worker fixtures being set up. + const aside = new Set(); + asideNow = aside; + const seeded = { random(): number { return preparing > 0 || aside.size > 0 ? own() : draw(); } }.random; + const E = Error as { prepareStackTrace?: unknown }; + const was = Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'); + let set: unknown = E.prepareStackTrace; + const wrapperOf = new WeakMap(); + const wrapped = new WeakMap(); + const get = (): unknown => { + if (typeof set !== 'function') return set; + let w = wrapperOf.get(set); + if (w === undefined) { + const f = set; + w = { prepareStackTrace(this: unknown, ...args: unknown[]): unknown { + preparing++; + try { + return Reflect.apply(f, this, args); + } finally { + preparing--; + } + } }.prepareStackTrace; + wrapperOf.set(f, w); + wrapped.set(w as object, f); + } + return w; + }; + const put = (v: unknown): void => { set = typeof v === 'function' && wrapped.has(v) ? wrapped.get(v) : v; }; + Object.defineProperty(E, 'prepareStackTrace', { get, set: put, enumerable: was?.enumerable ?? true, configurable: true }); + Math.random = seeded; + return () => { + if (Math.random === seeded) Math.random = own; + if (Object.getOwnPropertyDescriptor(E, 'prepareStackTrace')?.get !== get) return; + if (was === undefined) delete E.prepareStackTrace; + else Object.defineProperty(E, 'prepareStackTrace', was); + if (E.prepareStackTrace !== set) E.prepareStackTrace = set; + }; +} + +// The set the seat in place now (or the last one) takes setAside's holds +// from: each seat makes its own, so a hold never outlives its seat. +let asideNow = new Set(); + +/** + * Gives the process's own Math.random back, under the seat in place, until + * the returned function is called (once or more): what runs meanwhile is the + * worker's, not the test's. A worker fixture is set up in the first test that + * asks for it, after that test's seat is in place, so its draws (a test + * account made once for the worker's tests) would come from that test's + * stream and shift the test's own draws: a replay in which another test is + * the first to ask for it (one test alone, or after keploy set an earlier + * test aside) would draw other numbers than its recording. With no seat in + * place it holds nothing. + */ +export function setAside(): () => void { + const held = asideNow; + const hold = {}; + held.add(hold); + return () => { held.delete(hold); }; +} + // A key of the page's session storage: how many documents of each path the // tab has loaded from the document's origin, so that a document loaded again // in the same tab (a reload, a second visit) draws new numbers, as it would diff --git a/playwright/test/fixtures/process-ends.spec.ts b/playwright/test/fixtures/process-ends.spec.ts new file mode 100644 index 0000000..c3558f7 --- /dev/null +++ b/playwright/test/fixtures/process-ends.spec.ts @@ -0,0 +1,12 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +// A test that throws and one that times out; each fails, so Playwright runs +// each in a worker of its own, and each worker's afterAll says whether its +// own Math.random is back. +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +const own = Math.random; +let last = ''; +test('throws', () => { last = 'throws'; Math.random(); throw new Error('boom'); }); +test('times out', async () => { last = 'times out'; test.setTimeout(1000); Math.random(); await new Promise(() => {}); }); +test.afterAll(() => keep(`after ${last}`, { own: Math.random === own })); diff --git a/playwright/test/fixtures/process-helper.ts b/playwright/test/fixtures/process-helper.ts new file mode 100644 index 0000000..9754cea --- /dev/null +++ b/playwright/test/fixtures/process-helper.ts @@ -0,0 +1,13 @@ +import { test } from '../../dist'; + +// A suite's helper in a file of its own, as a login or a page object is: +// the first time a test's stack reaches this file, Playwright parses its +// source map. HELPER=step calls test.step, whose location Playwright reads +// from a stack; HELPER=stack reads an Error's stack, as a logger does. +export async function login(): Promise { + if (process.env.HELPER === 'stack') { + void new Error('where').stack; + return; + } + await test.step('login', async () => {}); +} diff --git a/playwright/test/fixtures/process-order.spec.ts b/playwright/test/fixtures/process-order.spec.ts new file mode 100644 index 0000000..d879d46 --- /dev/null +++ b/playwright/test/fixtures/process-order.spec.ts @@ -0,0 +1,15 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; +import { login } from './process-helper'; + +// Each test logs in through a helper in another file, then makes its data +// from Math.random. FAIL_FIRST=1 makes the first test fail, so Playwright +// runs the rest in a new worker. +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +test.beforeEach(async () => { await login(); }); +for (const n of ['first', 'second', 'third']) { + test(n, () => { + if (process.env.FAIL_FIRST && n === 'first') throw new Error('the first test fails in this run'); + keep(n, [Math.random(), Math.random()]); + }); +} diff --git a/playwright/test/fixtures/process-worker.spec.ts b/playwright/test/fixtures/process-worker.spec.ts new file mode 100644 index 0000000..bfc09fc --- /dev/null +++ b/playwright/test/fixtures/process-worker.spec.ts @@ -0,0 +1,49 @@ +import fs from 'node:fs'; +import { test as keployTest } from '../../dist'; + +// A worker fixture (not auto) that makes a test account from Math.random, as +// a suite makes one account for its worker's tests: the first test that asks +// for it sets it up. A second (team) that only the third test asks for. A +// test fixture (lead), an option and a plain value next to them. Each test +// makes its own data from Math.random after. USE_ACCOUNT=1 gives account's +// function through test.use instead, with no formatter in +// Error.prepareStackTrace and a stack trace limit of 0 while it registers. +// BROKEN=1 adds a test whose worker fixture fails before use(). +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +const sms = Object.getOwnPropertyDescriptor(Error, 'prepareStackTrace'); +const limit = Error.stackTraceLimit; +const test = keployTest.extend<{ lead: string; handlers: Array<() => string>; label: string }, { account: string; team: string; broken: string }>({ + account: [async ({}, use) => { await use(`acct-${Math.random()}`); }, { scope: 'worker' }], + team: [async ({}, use) => { const t = `team-${Math.random()}`; await use(t); }, { scope: 'worker' }], + broken: [async ({}, use) => { if (Math.random() < 2) throw new Error('no account today'); await use(''); }, { scope: 'worker' }], + lead: async ({}, use) => { await use(`lead-${Math.random()}`); }, + handlers: [() => 'a', () => 'b'], + label: ['x', { option: true }], +}); +const after = Object.getOwnPropertyDescriptor(Error, 'prepareStackTrace'); +const pst = { extend: { same: after?.value === sms?.value && after?.get === undefined, limit: Error.stackTraceLimit === limit } }; +if (process.env.USE_ACCOUNT) { + delete (Error as { prepareStackTrace?: unknown }).prepareStackTrace; + Error.stackTraceLimit = 0; + test.use({ account: async ({}, use) => { await use(`used-${Math.random()}`); } }); + Object.assign(pst, { use: { absent: Object.getOwnPropertyDescriptor(Error, 'prepareStackTrace') === undefined, limit: Error.stackTraceLimit === 0 } }); + Error.stackTraceLimit = limit; + if (sms) Object.defineProperty(Error, 'prepareStackTrace', sms); +} +test('first', ({ account, lead }) => { + void account; + keep('first', [lead, Math.random(), Math.random()]); + keep('pst', pst); +}); +test('second', ({ account }) => { + void account; + keep('second', [Math.random(), Math.random()]); +}); +test('third', ({ account, team, handlers, label }) => { + void account; void team; + keep('third', [Math.random(), Math.random(), handlers[0](), label]); +}); +if (process.env.BROKEN) { + test.afterEach(({}, testInfo) => { if (testInfo.title === 'broken') keep('broken', [Math.random()]); }); + test('broken', ({ broken }) => { void broken; }); +} diff --git a/playwright/test/fixtures/process.spec.ts b/playwright/test/fixtures/process.spec.ts new file mode 100644 index 0000000..22d6b91 --- /dev/null +++ b/playwright/test/fixtures/process.spec.ts @@ -0,0 +1,30 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +// A suite that makes its test data in the test process, as many do: a name +// stamped with Math.random(), drawn in the Node worker that runs the test, +// not in its browser. Each test keeps what it drew in its beforeEach, its +// body and its afterEach; no test opens a page. +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +// The worker's own Math.random, as this file loads, before any test. +const own = Math.random; +const drew = new Map(); +const draw = (): void => { + const t = test.info().title; + drew.set(t, [...(drew.get(t) ?? []), Math.random()]); +}; + +test.beforeEach(() => draw()); +test.afterEach(() => { + draw(); + keep(test.info().title, drew.get(test.info().title)); +}); + +test('made', () => { draw(); draw(); }); +test('other', () => { draw(); draw(); }); +test.describe('own', () => { + test.use({ keploySeedRandom: false }); + test('mine', () => { draw(); draw(); }); +}); +// Outside every test the worker draws its own numbers again. +test.afterAll(() => keep('outside', { own: Math.random === own, draws: [Math.random(), Math.random()] })); diff --git a/playwright/test/fixtures/steps-reporter.cjs b/playwright/test/fixtures/steps-reporter.cjs index 0d8555d..55fa240 100644 --- a/playwright/test/fixtures/steps-reporter.cjs +++ b/playwright/test/fixtures/steps-reporter.cjs @@ -1,11 +1,13 @@ // The rows' reporter of every step a test ran, hooks' and fixtures' included // (Playwright's JSON report keeps only steps at the test's top level): one -// JSON line {test, step} per step begun, appended to STEPS_FILE. +// JSON line {test, step, at} per step begun, appended to STEPS_FILE; at is +// the step's file name and line, '' for none. const fs = require('node:fs'); +const path = require('node:path'); class StepsReporter { onStepBegin(test, _result, step) { - if (process.env.STEPS_FILE) fs.appendFileSync(process.env.STEPS_FILE, `${JSON.stringify({ test: test.title, step: step.title })}\n`); + if (process.env.STEPS_FILE) fs.appendFileSync(process.env.STEPS_FILE, `${JSON.stringify({ test: test.title, step: step.title, at: step.location ? `${path.basename(step.location.file)}:${step.location.line}` : '' })}\n`); } } module.exports = StepsReporter; diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 9f0b020..731caad 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1,8 +1,9 @@ -// Unit rows of the test browser's seeding: the env rule, the seed, and the -// init script run in a Node vm context in place of a page. No browser. +// Unit rows of the seeding: the env rule, the seed, the test process's +// stream, and the browser's init script run in a Node vm context in place of +// a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, LOADS_KEY, PAGE_KEY, SEEDED_LINE, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, seedingFromEnv, unplacedLine, unseededLine } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine } from '../dist/random.js'; import { SEED } from '../dist/agent.js'; import { OTHER, documentAt } from './draws.mjs'; @@ -22,16 +23,24 @@ test('seeding is on unless KEPLOY_SEED_RANDOM turns it off; another value is ref for (const v of ['yes', 'no', 'seeded', '2']) { const s = seedingFromEnv({ [ENV_SEED_RANDOM]: v }); assert.equal(s.mode, 'refused', v); - assert.equal(s.refusal, `keploy: KEPLOY_SEED_RANDOM=${JSON.stringify(v)} is neither on nor off; set it to off to give the test browser its own random numbers, or unset it to seed them per test`); + assert.equal(s.refusal, `keploy: KEPLOY_SEED_RANDOM=${JSON.stringify(v)} is neither on nor off; set it to off to give the test browser and the test process their own random numbers, or unset it to seed them per test`); } }); test('the lines say what is seeded, what is not, where the seed lives, and how to turn it off or on', () => { - assert.equal(SEEDED_LINE, 'keploy: the test browser\'s random numbers (Math.random, crypto.getRandomValues, crypto.randomUUID) are seeded per test ' + - 'from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew; ' + - 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named). ' + + assert.equal(SEEDED_LINE, 'keploy: each test\'s random numbers are seeded from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew: ' + + 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random, from the test\'s begin to its end ' + + '(its beforeEach and afterEach hooks too). ' + + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + + 'the test process keeps its own crypto, and its own Math.random outside every test (beforeAll, afterAll, a file as it loads) ' + + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + 'To turn it off, record and replay with KEPLOY_SEED_RANDOM=off, or set use: { keploySeedRandom: false } in the Playwright config.'); - assert.equal(unseededLine('g › t'), 'keploy: keploy answered the begin of g › t with no seed, so its browser draws its own random numbers: ' + + assert.equal(OFF_LINE, 'keploy: random numbers are not seeded (KEPLOY_SEED_RANDOM=off, or keploySeedRandom: false), ' + + 'so the test process and the test browser draw their own each run: data a test makes from them (a name, an email, an ID) differs from its recording\'s ' + + 'while a mocked dependency answers with the recording\'s, so a test that checks one against the other fails or is set aside, ' + + 'and a page script\'s random choice may make calls the recording never saw. ' + + 'Unset KEPLOY_SEED_RANDOM and keploySeedRandom to seed them per test.'); + assert.equal(unseededLine('g › t'), 'keploy: keploy answered the begin of g › t with no seed, so its browser and its test process draw their own random numbers: ' + 'its recording kept none (it was recorded before keploy kept seeds, or with seeding off: record the test again with seeding on, the default, to seed it), ' + 'or the test was pulled from a Keploy branch, which keeps no seed, or this keploy predates seeds.'); assert.equal(unplacedLine('g › t', 'http://h/p'), 'keploy: in g › t, the document at http://h/p drew the browser\'s own random numbers, so its draws may differ at replay: ' + @@ -125,6 +134,152 @@ test('a begin\'s seed is drawn afresh each time: 128 bits of hex, a seed keploy for (const s of ['', '0123abc', 'f'.repeat(65), '0123ABCD', '0123abcg', ' 0123abcd']) assert.doesNotMatch(s, SEED, s); }); +test('the test process\'s stream: the same numbers for a seed on every run and every version, others for another seed, 53 bits in [0, 1)', () => { + // Pinned to a reference made outside this package (sfc32 over the SHA-256 + // of the seed and the stream's name): a recording keeps the seed, not the + // numbers, so a change here would make every recording's data anew. + const draws = (seed, n) => { const r = processRandom(seed); return Array.from({ length: n }, () => r()); }; + assert.deepEqual(draws(S1, 3), [0.6791662241098093, 0.8127873727939283, 0.45502303021506096]); + assert.deepEqual(draws(S2, 3), [0.6007115277024065, 0.36062226677768605, 0.8529252058074182]); + const many = draws(S1, 4096); + assert.deepEqual(many.slice(0, 3), draws(S1, 3)); + for (const x of many) assert.ok(x >= 0 && x < 1 && Number.isInteger(x * 2 ** 53), String(x)); + assert.equal(new Set(many).size, many.length); + // Not the browser's Math.random stream for the same seed. + assert.notDeepEqual(draws(S1, 2), documentAt(initScript(S1), '/p').draws().random); +}); + +test('seatRandom puts the stream in Math.random\'s place and the process\'s own back; a stub the test put there stays', () => { + const own = Math.random; + const r = processRandom(S1); + const unseat = seatRandom(processRandom(S1)); + try { + assert.notEqual(Math.random, own); + assert.deepEqual([Math.random(), Math.random()], [r(), r()]); + } finally { unseat(); } + assert.equal(Math.random, own); + // A test that stubbed Math.random over the seeded one keeps its stub. + const stub = () => 0.25; + const unseat2 = seatRandom(processRandom(S1)); + Math.random = stub; + unseat2(); + assert.equal(Math.random, stub); + Math.random = own; +}); + +test('while seated, a draw made as a stack trace is prepared is the process\'s own and moves no draw of the stream; Error.prepareStackTrace is put back as it was', () => { + const E = Error; + const before = Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'); + const realRandom = Math.random; + const back = () => { + Math.random = realRandom; + if (before === undefined) delete E.prepareStackTrace; else Object.defineProperty(E, 'prepareStackTrace', before); + }; + try { + // The process's own Math.random, and a formatter set before seating + // that draws, as source-map's sort does while it parses a file's map. + Math.random = () => 0.5; + const inside = []; + const fmt = (e, s) => { inside.push(Math.random()); return `F:${s.length}`; }; + Object.defineProperty(E, 'prepareStackTrace', { value: fmt, writable: true, enumerable: false, configurable: true }); + const r = processRandom(S1); + const unseat = seatRandom(processRandom(S1)); + try { + const a = Math.random(); + assert.match(new Error('x').stack, /^F:\d+$/); + // Read twice, the same function; not enumerable, as it was. + const old = E.prepareStackTrace; + assert.equal(E.prepareStackTrace, old); + assert.equal(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace').enumerable, false); + // A formatter set for a moment, as Playwright's location read does, + // and one that calls the formatter it found before drawing itself. + E.prepareStackTrace = (e, s) => { inside.push(Math.random()); return s; }; + const o = {}; + Error.captureStackTrace(o); + assert.ok(Array.isArray(o.stack)); + E.prepareStackTrace = (e, s) => { const t = old(e, s); inside.push(Math.random()); return t; }; + assert.match(new Error('y').stack, /^F:\d+$/); + E.prepareStackTrace = old; + const b = Math.random(); + assert.deepEqual([a, b], [r(), r()]); + assert.deepEqual(inside, [0.5, 0.5, 0.5, 0.5]); + } finally { unseat(); } + // The formatter last set, unwrapped, as its own property was. + assert.equal(E.prepareStackTrace, fmt); + assert.deepEqual(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'), { value: fmt, writable: true, enumerable: false, configurable: true }); + assert.equal(Math.random(), 0.5); + + // No formatter: none while seated, a stack still made, and none after. + delete E.prepareStackTrace; + const unseat2 = seatRandom(processRandom(S1)); + try { + assert.equal(E.prepareStackTrace, undefined); + assert.equal(typeof new Error('z').stack, 'string'); + } finally { unseat2(); } + assert.equal(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'), undefined); + // One set while seated stays set after, as an assignment leaves it. + const unseat3 = seatRandom(processRandom(S1)); + try { E.prepareStackTrace = fmt; } finally { unseat3(); } + assert.deepEqual(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'), { value: fmt, writable: true, enumerable: true, configurable: true }); + + // A test that defines the property itself keeps its definition. + const mine = (e, s) => s.length; + const unseat4 = seatRandom(processRandom(S1)); + try { + Object.defineProperty(E, 'prepareStackTrace', { value: mine, writable: false, enumerable: false, configurable: true }); + } finally { unseat4(); } + assert.deepEqual(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'), { value: mine, writable: false, enumerable: false, configurable: true }); + // And the process's own Math.random is back after it. + assert.equal(Math.random(), 0.5); + + // A formatter that throws (as source-map-support's does on a map it + // cannot read) leaves the stream's draws to the test after it. + delete E.prepareStackTrace; + const r5 = processRandom(S2); + const unseat5 = seatRandom(processRandom(S2)); + try { + E.prepareStackTrace = () => { throw new Error('unreadable map'); }; + assert.throws(() => new Error('w').stack, /unreadable map/); + E.prepareStackTrace = undefined; + assert.equal(Math.random(), r5()); + } finally { unseat5(); } + } finally { back(); } +}); + +test('setAside gives the process\'s own Math.random back under the seat until each hold is let go, once or more; a hold taken before the seat, or under an earlier one, holds nothing', () => { + const realRandom = Math.random; + // The process's own. + Math.random = () => 0.5; + try { + const r = processRandom(S1); + const early = setAside(); + const unseat = seatRandom(processRandom(S1)); + try { + assert.equal(Math.random(), r()); + const a = setAside(); + const b = setAside(); + assert.equal(Math.random(), 0.5); + a(); a(); + assert.equal(Math.random(), 0.5); + b(); + early(); + // No draw of the stream was taken while held. + assert.equal(Math.random(), r()); + } finally { unseat(); } + const unseat1 = seatRandom(processRandom(S1)); + const late = setAside(); + unseat1(); + const r2 = processRandom(S2); + const unseat2 = seatRandom(processRandom(S2)); + try { + assert.equal(Math.random(), r2()); + late(); + assert.equal(Math.random(), r2()); + } finally { unseat2(); } + assert.equal(Math.random(), 0.5); + } finally { Math.random = realRandom; } +}); + test('a seeded document draws the same numbers for the same seed and path, others for another seed or path', () => { const s1 = initScript(S1); const a = documentAt(s1, '/p').draws(); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index e7be4b5..8b5bb4d 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -10,7 +10,7 @@ import http from 'node:http'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; -import { LOADS_KEY, SEEDED_LINE, initScript, unplacedLine, unseededLine } from '../dist/random.js'; +import { LOADS_KEY, OFF_LINE, SEEDED_LINE, initScript, processRandom, unplacedLine, unseededLine } from '../dist/random.js'; import { BLANK_DRAWS, DRAWS, documentAt } from './draws.mjs'; const PKG = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); @@ -136,7 +136,10 @@ async function playwright(specs, env, cwd = PKG) { }); const key = [...titles, spec.title].join(' › '); if (tests[key]) tests[key].runs.push(...runs); - else tests[key] = { ...runs[0], runs, steps: steps.filter((x) => x.test === spec.title).map((x) => x.step) }; + else { + const mine = steps.filter((x) => x.test === spec.title); + tests[key] = { ...runs[0], runs, steps: mine.map((x) => x.step), at: Object.fromEntries(mine.map((x) => [x.step, x.at])) }; + } } }; for (const s of JSON.parse(fs.readFileSync(report, 'utf8')).suites) walk(s, []); @@ -1208,7 +1211,7 @@ test('seeded: keploy answers no seed (an older recording, or an older keploy): t } finally { await a.close(); } }); -test('seeded: keploySeedRandom: false for the whole project asks no seed, says nothing, and the browser draws its own', async () => { +test('seeded: keploySeedRandom: false for the whole project asks no seed, says once what that costs, and the browser draws its own', async () => { const j = journal(); // An agent that answers a seed even to a begin that asked for none: the // test that opted out still draws the browser's own numbers. @@ -1219,23 +1222,23 @@ test('seeded: keploySeedRandom: false for the whole project asks no seed, says n for (const n of ['one', 'two', 'again', 'three']) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); for (const r of [r1, r2]) { assert.equal(r.said, 0); - assert.equal(r.lines.filter((l) => l.startsWith('keploy:')).length, 0, r.lines.join('\n')); + assert.deepEqual(r.lines.filter((l) => l.startsWith('keploy:')), [OFF_LINE], r.lines.join('\n')); assert.deepEqual(r.begins.map((b) => [b.random, b.seed]), Array(4).fill(['own', undefined])); assert.deepEqual(r.ends, NAMES.map((n) => [n, 'own'])); } } finally { await a.close(); } }); -test('KEPLOY_SEED_RANDOM=off, or no keploy: the test browser draws its own random numbers, and nothing is said', async () => { +test('KEPLOY_SEED_RANDOM=off, or no keploy: the test browser draws its own random numbers; under keploy the worker says once what that costs, without it nothing is said', async () => { const j = journal(); const a = await fakeAgent(j); try { - for (const env of [{ ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'off' }, { KEPLOY_MOCK_AGENT: undefined, KEPLOY_TEST_PID: undefined }]) { + for (const [env, said] of [[{ ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'off' }, [OFF_LINE]], [{ KEPLOY_MOCK_AGENT: undefined, KEPLOY_TEST_PID: undefined }, []]]) { const r1 = await drawRun(j, env); const r2 = await drawRun(j, env); for (const n of ['one', 'two', 'again', 'three']) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); assert.equal(r1.said + r2.said, 0); - assert.equal([...r1.lines, ...r2.lines].filter((l) => l.startsWith('keploy:')).length, 0); + for (const r of [r1, r2]) assert.deepEqual(r.lines.filter((l) => l.startsWith('keploy:')), said, r.lines.join('\n')); } // Under keploy with it off, each begin says the browser draws its own // and sends no seed, and each end says it did. @@ -1253,8 +1256,242 @@ test('KEPLOY_SEED_RANDOM neither on nor off fails every test, naming it, and the assert.equal(Object.keys(tests).length, 4); for (const [k, t] of Object.entries(tests)) { assert.equal(t.status, 'failed', k); - assert.match(t.error, /keploy: KEPLOY_SEED_RANDOM="yes" is neither on nor off; set it to off to give the test browser its own random numbers, or unset it to seed them per test/, k); + assert.match(t.error, /keploy: KEPLOY_SEED_RANDOM="yes" is neither on nor off; set it to off to give the test browser and the test process their own random numbers, or unset it to seed them per test/, k); } assert.deepEqual(calls(j), []); } finally { await d.close(); await a.close(); } }); + +// One run of process.spec.ts: what each test drew in the test process, what +// code outside every test drew, what the run said, and the seed each begin +// sent. +async function processRun(j, env) { + const from = j.events.length; + const { tests, tmp, out } = await playwright(['process.spec.ts'], env); + for (const [k, t] of Object.entries(tests)) assert.equal(t.status, 'passed', `${k}: ${t.error}\n${out}`); + const drew = {}; + for (const n of ['made', 'other', 'mine', 'outside']) drew[n] = JSON.parse(fs.readFileSync(path.join(tmp, `${n}.json`), 'utf8')); + // Every line the run's tests said, in no order: the report lists a + // describe group's tests before the file's own. + const lines = Object.values(tests).map((t) => t.stdout).join('').split('\n').sort(); + const begins = calls({ events: j.events.slice(from) }).filter((c) => c.path === '/agent/scope/begin').map((c) => c.body); + const seeds = Object.fromEntries(begins.map((b) => [b.name.replace(/^own › /, ''), b.seed])); + return { drew, lines, seeds, out }; +} + +test('seeded: the test process draws Math.random from its test\'s seed in its beforeEach, body and afterEach, so a replay makes the data its recording made; another test, an opted-out one and code outside every test draw others', async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + try { + const r = await processRun(j, keployEnv(rec.url)); + kept = r.seeds; + // Four draws a test: its beforeEach, two in its body, its afterEach, + // each from the stream of the seed keploy answered its begin with. + for (const n of ['made', 'other']) { + const next = processRandom(r.seeds[n]); + assert.deepEqual(r.drew[n], [next(), next(), next(), next()], n); + } + assert.equal(r.drew.mine.length, 4); + assert.equal(r.seeds.mine, undefined); + assert.notDeepEqual(r.drew.made, r.drew.other); + // Said once each: the seeding at the first seeded test, the cost at the + // opted-out one. + assert.deepEqual(r.lines.filter((l) => l.startsWith('keploy:')), [SEEDED_LINE, OFF_LINE].sort(), r.out); + const p = await processRun(j, keployEnv(rep.url)); + // The replay's begins sent fresh seeds; keploy answered the recording's. + for (const n of ['made', 'other']) { + assert.notEqual(p.seeds[n], kept[n], n); + assert.deepEqual(p.drew[n], r.drew[n], n); + } + // The opted-out test and the code after the last test draw the worker's own. + assert.notDeepEqual(p.drew.mine, r.drew.mine); + assert.notDeepEqual(p.drew.outside.draws, r.drew.outside.draws); + for (const x of [r, p]) assert.equal(x.drew.outside.own, true); + // A record made again draws new numbers. + const r2 = await processRun(j, keployEnv(rec.url)); + assert.notDeepEqual(r2.drew.made, r.drew.made); + } finally { await rep.close(); await rec.close(); } +}); + +test('the test process draws its own Math.random when keploy answers no seed, each test saying so, or with KEPLOY_SEED_RANDOM=off, the worker saying once what that costs', async () => { + const j = journal(); + const none = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); + const rec = await fakeAgent(j); + try { + for (const [env, said] of [ + [keployEnv(none.url), [unseededLine('made'), unseededLine('other'), OFF_LINE]], + [{ ...keployEnv(rec.url), KEPLOY_SEED_RANDOM: 'off' }, [OFF_LINE]], + ]) { + const r1 = await processRun(j, env); + const r2 = await processRun(j, env); + for (const n of ['made', 'other', 'mine']) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); + for (const r of [r1, r2]) { + assert.equal(r.drew.outside.own, true); + assert.deepEqual(r.lines.filter((l) => l.startsWith('keploy:')), said.sort(), r.out); + } + } + } finally { await rec.close(); await none.close(); } +}); + +// One run of process-order.spec.ts (spec args after the file, as --grep): +// what each test drew, each test's status, the lines its tests said, and the +// seed each begin sent. +const ORDER = ['first', 'second', 'third']; +async function orderRun(j, env, args = []) { + const from = j.events.length; + const { tests, tmp, out } = await playwright(['process-order.spec.ts', ...args], env); + const drew = {}; + for (const n of ORDER) { + const f = path.join(tmp, `${n}.json`); + drew[n] = fs.existsSync(f) ? JSON.parse(fs.readFileSync(f, 'utf8')) : null; + } + const status = Object.fromEntries(Object.entries(tests).map(([k, t]) => [k, t.status])); + const lines = Object.values(tests).map((t) => t.stdout).join('').split('\n').filter((l) => l.startsWith('keploy:')).sort(); + const seeds = Object.fromEntries(calls({ events: j.events.slice(from) }).filter((c) => c.path === '/agent/scope/begin').map((c) => [c.body.name, c.body.seed])); + return { drew, status, lines, seeds, out }; +} + +for (const helper of ['step', 'stack']) { + test(`seeded: a test's draws in the test process do not depend on which tests its worker ran before it, when its helper file's source map is parsed in it (${helper === 'step' ? 'test.step' : 'an Error\'s stack'}): replayed alone, after keploy set an earlier test aside, or after its record's first test failed, it draws what its recording drew`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + // A replay answers each begin the recording's seed, and sets aside the + // tests named in skip, as keploy does a quarantined one. + let skip = []; + const rep = await fakeAgent(j, (c) => { + if (c.path !== '/agent/scope/begin') return undefined; + if (skip.includes(c.body.name)) return { body: { v: 1, status: 'ok', action: 'skip', reason: 'flaky/request_changed: the body changed' } }; + return { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } }; + }); + const env = (url, x = {}) => ({ ...keployEnv(url), HELPER: helper, ...x }); + const same = (p, r, names) => Object.fromEntries(names.map((n) => [n, JSON.stringify(p.drew[n]) === JSON.stringify(r.drew[n]) ? 'same' : 'other'])); + try { + const r = await orderRun(j, env(rec.url)); + assert.deepEqual(r.status, { first: 'passed', second: 'passed', third: 'passed' }, r.out); + // Each test's two draws are its seed's stream's first two: the + // first test, which parsed the helper's map, too. + for (const n of ORDER) { + const next = processRandom(r.seeds[n]); + assert.deepEqual(r.drew[n], [next(), next()], n); + } + // Seeded, with no test opting out: the seeding line once, nothing else. + assert.deepEqual(r.lines, [SEEDED_LINE], r.out); + kept = r.seeds; + const alone = await orderRun(j, env(rep.url), ['--grep', 'third']); + assert.deepEqual(alone.status, { third: 'passed' }, alone.out); + assert.deepEqual(same(alone, r, ['third']), { third: 'same' }); + skip = ['first']; + const aside = await orderRun(j, env(rep.url)); + assert.deepEqual(aside.status, { first: 'skipped', second: 'passed', third: 'passed' }, aside.out); + assert.deepEqual(same(aside, r, ['second', 'third']), { second: 'same', third: 'same' }); + skip = []; + // A record whose first test fails runs the rest in a new worker; its + // replay, where the first passes, runs all three in one. + const rf = await orderRun(j, env(rec.url, { FAIL_FIRST: '1' })); + assert.deepEqual(rf.status, { first: 'failed', second: 'passed', third: 'passed' }, rf.out); + kept = rf.seeds; + const pf = await orderRun(j, env(rep.url)); + assert.deepEqual(pf.status, { first: 'passed', second: 'passed', third: 'passed' }, pf.out); + assert.deepEqual(same(pf, rf, ['second', 'third']), { second: 'same', third: 'same' }); + } finally { await rep.close(); await rec.close(); } + }); +} + +// One run of process-worker.spec.ts, as orderRun runs process-order's. +async function workerRun(j, env, args = []) { + const from = j.events.length; + const { tests, tmp, out } = await playwright(['process-worker.spec.ts', ...args], env); + const drew = {}; + for (const n of [...ORDER, 'broken', 'pst']) { + const f = path.join(tmp, `${n}.json`); + drew[n] = fs.existsSync(f) ? JSON.parse(fs.readFileSync(f, 'utf8')) : null; + } + const status = Object.fromEntries(Object.entries(tests).map(([k, t]) => [k, t.status])); + const seeds = Object.fromEntries(calls({ events: j.events.slice(from) }).filter((c) => c.path === '/agent/scope/begin').map((c) => [c.body.name, c.body.seed])); + return { drew, status, seeds, tests, out }; +} + +test('seeded: a worker fixture set up in the first test that asks for it draws the worker\'s own Math.random, so a test draws what its recording drew when replayed alone or after keploy set an earlier test aside; a test fixture draws from its test\'s stream', async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + let skip = []; + const rep = await fakeAgent(j, (c) => { + if (c.path !== '/agent/scope/begin') return undefined; + if (skip.includes(c.body.name)) return { body: { v: 1, status: 'ok', action: 'skip', reason: 'flaky/request_changed: the body changed' } }; + return { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } }; + }); + const same = (p, r, names) => Object.fromEntries(names.map((n) => [n, JSON.stringify(p.drew[n]) === JSON.stringify(r.drew[n]) ? 'same' : 'other'])); + try { + for (const how of ['extend', 'use']) { + const env = (url) => ({ ...keployEnv(url), USE_ACCOUNT: how === 'use' ? '1' : undefined }); + const r = await workerRun(j, env(rec.url)); + assert.deepEqual(r.status, { first: 'passed', second: 'passed', third: 'passed' }, r.out); + // Each test's draws are its stream's first ones: the first test's + // lead (a test fixture) first, then its body's; account (set up in + // the first test) and team (in the third) take none of them. + const firsts = (n, k) => { const next = processRandom(r.seeds[n]); return Array.from({ length: k }, () => next()); }; + const [l, a, b] = firsts('first', 3); + assert.deepEqual(r.drew.first, [`lead-${l}`, a, b], how); + assert.deepEqual(r.drew.second, firsts('second', 2), how); + // A plain value and an option are given as they were written. + assert.deepEqual(r.drew.third, [...firsts('third', 2), 'a', 'x'], how); + // Playwright names each fixture at the line that registered it, in the + // spec, not in this package. + // So it is through use with no formatter set and a stack trace limit + // of 0; extend and use leave Error.prepareStackTrace and the limit as + // they found them. + const line = how === 'use' ? 28 : 15; + assert.equal(r.tests.first.at['Fixture "account"'], `process-worker.spec.ts:${line}`, how); + assert.equal(r.tests.third.at['Fixture "team"'], 'process-worker.spec.ts:15', how); + assert.deepEqual(r.drew.pst, { extend: { same: true, limit: true }, ...(how === 'use' ? { use: { absent: true, limit: true } } : {}) }, how); + if (how === 'use') continue; + kept = r.seeds; + for (const n of ['second', 'third']) { + const alone = await workerRun(j, env(rep.url), ['--grep', n]); + assert.deepEqual(alone.status, { [n]: 'passed' }, alone.out); + assert.deepEqual(same(alone, r, [n]), { [n]: 'same' }); + } + skip = ['first']; + const aside = await workerRun(j, env(rep.url)); + assert.deepEqual(aside.status, { first: 'skipped', second: 'passed', third: 'passed' }, aside.out); + assert.deepEqual(same(aside, r, ['second', 'third']), { second: 'same', third: 'same' }); + skip = []; + } + // A worker fixture that fails before use() leaves the rest of its test + // (its afterEach) drawing the test's stream. + const bk = await workerRun(j, { ...keployEnv(rec.url), BROKEN: '1' }, ['--grep', 'broken']); + assert.deepEqual(bk.status, { broken: 'failed' }, bk.out); + assert.deepEqual(bk.drew.broken, [processRandom(bk.seeds.broken)()]); + } finally { await rep.close(); await rec.close(); } +}); + +test('the test process draws its own Math.random when keploy answers no seed and no test opts out: each test says so, and nothing else is said', async () => { + const j = journal(); + const none = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); + try { + const r1 = await orderRun(j, { ...keployEnv(none.url), HELPER: 'step' }); + const r2 = await orderRun(j, { ...keployEnv(none.url), HELPER: 'step' }); + for (const n of ORDER) assert.notDeepEqual(r2.drew[n], r1.drew[n], n); + for (const r of [r1, r2]) assert.deepEqual(r.lines, ORDER.map((n) => unseededLine(n)).sort(), r.out); + } finally { await none.close(); } +}); + +test('seeded: after a test that throws, and after one that times out, the worker draws its own Math.random again', async () => { + const j = journal(); + const rec = await fakeAgent(j); + try { + const { tests, tmp, out } = await playwright(['process-ends.spec.ts'], keployEnv(rec.url)); + assert.deepEqual(Object.fromEntries(Object.entries(tests).map(([k, t]) => [k, t.status])), { throws: 'failed', 'times out': 'timedOut' }, out); + for (const n of ['after throws', 'after times out']) { + assert.deepEqual(JSON.parse(fs.readFileSync(path.join(tmp, `${n}.json`), 'utf8')), { own: true }, n); + } + // Both were seeded: each began with a seed. + const begins = calls(j).filter((c) => c.path === '/agent/scope/begin').map((c) => c.body); + assert.deepEqual(begins.map((b) => [b.name, b.random, /^[0-9a-f]{32}$/.test(b.seed)]), [['throws', 'seeded', true], ['times out', 'seeded', true]]); + } finally { await rec.close(); } +}); From 63c61e7cc4b2d011ac312ce5ba87c918d8cc7867 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 17:18:25 +0530 Subject: [PATCH 06/73] refactor(playwright): drop an unreachable branch from a document's why A document's kind is top, frame or other; whyOf returns for top and other before its last test, so `s.kind !== 'frame'` was never true there. The reason a document gives is unchanged. Signed-off-by: slayerjain --- playwright/src/random.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/playwright/src/random.ts b/playwright/src/random.ts index cade79b..f567698 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -726,7 +726,6 @@ export function initScript(seed: string, report = '', mark = ''): string { const s = now(); if (s.kind === 'top') return s.uncounted ? 'nostorage' : ''; if (s.kind === 'other') return s.uncounted ? 'uncounted' : ''; - if (s.kind !== 'frame') return ''; return s.frame === null ? 'frame' : s.up.why(); }; const aboveFor = () => { From 43f3045101de43196972ade934f5ea81267ec0b7 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 17:18:25 +0530 Subject: [PATCH 07/73] test(playwright): count an uncounted frame's peers from one inside it No row reached a frame of its parent's origin inside a frame that cannot count its loads, so the fixture's count of the frames beside the uncounted one could have counted the inner frame's instead and every row passed. many.spec.ts's new test: a cross-site frame fills its origin's session storage in the tab, then the page puts in a second frame of that origin, which cannot count its load, holding a frame of its own origin. Both lines must name one other frame (the first, beside the uncounted one under the page), not zero (the inner frame is alone in its parent), in every engine, at record and at replay. Signed-off-by: slayerjain --- playwright/test/fixtures/many.spec.ts | 19 +++++++++++++++++- playwright/test/rows.test.mjs | 28 +++++++++++++++++++++++++-- 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/playwright/test/fixtures/many.spec.ts b/playwright/test/fixtures/many.spec.ts index 4857c95..327eac4 100644 --- a/playwright/test/fixtures/many.spec.ts +++ b/playwright/test/fixtures/many.spec.ts @@ -5,7 +5,8 @@ import { test } from '../../dist'; // Run in Chromium, Firefox and WebKit (PW_BROWSER). A page that holds many // frames of its origin, in its markup or put in by one script; sandboxed // frames, which have no session storage (some whose own script replaces the -// global parent, some holding a srcdoc frame); and what seeding costs a page +// global parent, some holding a srcdoc frame); a cross-site frame whose +// origin's session storage is full, holding a frame; and what seeding costs a page // whose DOM changes often. Each test keeps what it saw. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); @@ -110,6 +111,22 @@ test('nested', async ({ page }) => { keep('nested', drew); }); +test('filled', async ({ page }) => { + // A cross-site frame a fills its origin's session storage in the tab; + // then b, of that origin, cannot count its load, and holds y of its own + // origin. + await page.goto(`${base}/full`); + const b = await loaded(page, 'b', '/ffx'); + let y: unknown; + for (let i = 0; i < 400 && y === undefined; i++) { + const c = b.childFrames()[0]; + if (c) { try { y = await c.evaluate(read); } catch { /* not yet */ } } + if (y === undefined) await page.waitForTimeout(25); + } + if (y === undefined) throw new Error('no frame y drew in b'); + keep('filled', { b: await b.evaluate(read), y }); +}); + // A workload in the page: 'grow' adds 20000 rows in 500 tasks; 'churn' // adds and removes one span in each of 300 tasks, on a page of 20000 // elements; 'shadow' does the same on a page of 2000 open shadow roots of diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 8b5bb4d..74f3f9a 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -663,6 +663,23 @@ async function drawsServer() { return; } if (u.pathname === '/fsn') { res.end(page(``)); return; } + // /full holds a (/ffill, cross-site), which fills its origin's session + // storage in the tab and then has /full put in b (/ffx, cross-site too), + // which holds y (/ffy, of b's origin). b cannot count its load, so it is + // a frame under a parent of another origin with no session storage, and + // y a frame inside one: a is beside b, y is alone in b. + if (u.pathname === '/full') { + res.end(page(``)); + return; + } + if (u.pathname === '/ffill') { + res.end(page(``)); + return; + } + if (u.pathname === '/ffx') { res.end(page('')); return; } + if (u.pathname === '/ffy') { res.end(page('')); return; } // /held holds sandboxed frame h0 at /fh, then /held.js, then h1 and h2 // at /fh. /held.js is answered only once h0 has loaded (its load sends // /fh-loaded), so its parent's parser puts h1 and h2 in only after h0 @@ -1087,13 +1104,13 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: 300 frames in a page's markup, or put in by one script, each load and draw their own numbers, the same at replay, with no page error; sandboxed frames are never seeded, each draws its own numbers and is named with how many frames are beside it once its parent's parser has ended: three at one path, a lone one, a new one put in at a removed one's path, ones whose own script replaces the global parent before they draw, one that draws and loads while its parent's parser holds the frames after it, and a srcdoc frame inside one (sandboxed too, alone under its parent); a page whose DOM changes often is walked only when a frame is put in`, async () => { + test(`seeded, in ${engine}: 300 frames in a page's markup, or put in by one script, each load and draw their own numbers, the same at replay, with no page error; sandboxed frames are never seeded, each draws its own numbers and is named with how many frames are beside it once its parent's parser has ended: three at one path, a lone one, a new one put in at a removed one's path, ones whose own script replaces the global parent before they draw, one that draws and loads while its parent's parser holds the frames after it, and a srcdoc frame inside one (sandboxed too, alone under its parent); a cross-site frame that cannot count its load (its origin's session storage full) and a frame of its origin inside it are named with the frames beside the first; a page whose DOM changes often is walked only when a frame is put in`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); - const names = ['markup', 'script', 'sandboxed', 'lone', 'swapped', 'held', 'nested', 'cost']; + const names = ['markup', 'script', 'sandboxed', 'lone', 'swapped', 'held', 'nested', 'filled', 'cost']; const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); const unplaced = (r, n) => r.tests[n].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)).map((l) => l.replace(/:\d+\//, ':/')); const line = (n, at, others) => unplacedLine(n, `http://127.0.0.1:${at}`, 'uncounted', others); @@ -1108,6 +1125,12 @@ for (const engine of ENGINES) { // under a parent of another origin with no session storage, alone // under its parent, and its line counts no other frame (every engine). assert.deepEqual(unplaced(run, 'nested').sort(), [line('nested', '/fsn', 1), unplacedLine('nested', 'about:srcdoc', 'uncounted', 0)].sort()); + // A cross-site frame b that cannot count its load (a frame of its + // origin filled that origin's session storage in the tab), and y, of + // b's origin, inside it: both lines count the frames beside b under + // its parent (a), not those beside y in b (none). + const full = (at) => unplacedLine('filled', `http://localhost:${at}`, 'uncounted', 1); + assert.deepEqual(unplaced(run, 'filled').sort(), [full('/ffx'), full('/ffy')].sort()); }; try { const r = await drawRun(j, env(rec.url), ['many.spec.ts'], names); @@ -1144,6 +1167,7 @@ for (const engine of ENGINES) { for (const x of Object.values(held)) assert.notDeepEqual(x, seeded('held', '/fh')); const nested = Object.values(r.drew.nested).flatMap((x) => [x.own, x.child]); assert.equal(new Set(nested.map((x) => x.uuid)).size, 4); + assert.notEqual(r.drew.filled.b.uuid, r.drew.filled.y.uuid); sandboxed(r); // A page whose DOM changes often: seeding walks the document for no // change that puts in no frame, and once when one frame is put in. From 5e939057a8a8362d6a61253ccd69afe84e4fa748 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 17:44:14 +0530 Subject: [PATCH 08/73] test(playwright): pin a document's stream and replace a junk load count A document's Math.random stream is now pinned to a reference made outside the package (sfc32 over a 128-bit hash of the seed, page, path and load, 15 draws dropped, 53 bits a draw), as the test process's stream already was: a recording keeps the seed, not the numbers, so a change to the stream would make every recording's data anew. The rows before it compared the browser's draws only with the same code's. The row for junk in a tab's load count now also checks the junk is replaced by a count (and adds a bare number), so the tab's next document at the path draws as its second load. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 731caad..32f22e1 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -149,6 +149,15 @@ test('the test process\'s stream: the same numbers for a seed on every run and e assert.notDeepEqual(draws(S1, 2), documentAt(initScript(S1), '/p').draws().random); }); +test('a document\'s Math.random stream: the same numbers for a seed, page, path and load on every run and every version', () => { + // Pinned to a reference made outside this package (sfc32 over a 128-bit + // hash of "seed\npage\npath\nloads\nMath.random", 15 draws dropped, 53 bits + // a draw): a recording keeps the seed, not the numbers, so a change here + // would make every recording's data anew. + assert.deepEqual(documentAt(initScript(S1), '/p').draws().random, [0.7259937449314731, 0.7408923870185994]); + assert.deepEqual(documentAt(initScript(S1), '/p', new Map([[LOADS_KEY, '{"/p":1}']])).draws().random, [0.2929767053506992, 0.11019693928372987]); +}); + test('seatRandom puts the stream in Math.random\'s place and the process\'s own back; a stub the test put there stays', () => { const own = Math.random; const r = processRandom(S1); @@ -432,8 +441,12 @@ test('a top document with no session storage draws the browser\'s own numbers an const named = []; assert.notDeepEqual(documentAt(r, 'blank', null, 't0', { report: 'rep', told: named, origin: 'null', href: 'about:blank', doc: { querySelectorAll: () => [] } }).draws(), a); assert.deepEqual(named, [['about:blank', 'nostorage']]); - for (const junk of ['not json', '[1,2]', 'null', '{"/p":"x"}']) { - assert.deepEqual(documentAt(s, '/p', new Map([[LOADS_KEY, junk]])).draws(), documentAt(s, '/p').draws(), junk); + // Junk in the tab's count is read as no loads and replaced by a count, so + // the tab's next document at the path draws as its second load. + for (const junk of ['not json', '[1,2]', '5', 'null', '{"/p":"x"}']) { + const tab = new Map([[LOADS_KEY, junk]]); + assert.deepEqual(documentAt(s, '/p', tab).draws(), documentAt(s, '/p').draws(), junk); + assert.equal(tab.get(LOADS_KEY), '{"/p":1}', junk); } }); From 98e83138a15450de09df9b4effb2b36ee4119962 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 18:27:13 +0530 Subject: [PATCH 09/73] test(playwright): row for a page open before keploy's context fixture No row gave keploy's context fixture a context that already held a page, so dropping the check that counts such a page left every row passing, while a test that drove only that page would end with no word on what its browser drew. opened.spec.ts merges a suite fixture that opens a page under keploy's and drives that page alone: its end must say seeded, and own with KEPLOY_SEED_RANDOM=off. Signed-off-by: slayerjain --- playwright/test/fixtures/opened.spec.ts | 18 ++++++++++++++++++ playwright/test/rows.test.mjs | 15 +++++++++++++++ 2 files changed, 33 insertions(+) create mode 100644 playwright/test/fixtures/opened.spec.ts diff --git a/playwright/test/fixtures/opened.spec.ts b/playwright/test/fixtures/opened.spec.ts new file mode 100644 index 0000000..10cb360 --- /dev/null +++ b/playwright/test/fixtures/opened.spec.ts @@ -0,0 +1,18 @@ +import { test as base, mergeTests } from '@playwright/test'; +import { test as keploy } from '../../dist'; + +// A context that already holds a page when keploy's context fixture gets it: +// a fixture of the suite's, merged under keploy's, opens it. The test drives +// that page and opens none, so only the page already there tells keploy's +// fixture that the test's browser drew. +const held = base.extend({ + context: async ({ context }, use) => { + await context.newPage(); + await use(context); + }, +}); +const test = mergeTests(held, keploy); + +test('held page', async ({ context }) => { + await context.pages()[0].goto(`${process.env.DRAWS_BASE ?? ''}/draws`); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 74f3f9a..e09183d 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1235,6 +1235,21 @@ test('seeded: keploy answers no seed (an older recording, or an older keploy): t } finally { await a.close(); } }); +test('a page the test\'s context already holds when keploy\'s fixture gets it is the test\'s: its end says seeded, or own with seeding off', async () => { + const j = journal(); + const a = await fakeAgent(j); + const d = await drawsServer(); + try { + for (const [env, want] of [[{}, 'seeded'], [{ KEPLOY_SEED_RANDOM: 'off' }, 'own']]) { + const from = j.events.length; + const { tests, out } = await playwright(['opened.spec.ts'], { ...keployEnv(a.url), DRAWS_BASE: d.url, ...env }); + assert.equal(tests['held page'].status, 'passed', out); + const ends = j.events.slice(from).filter((e) => e.agent && e.agent.path === '/agent/scope/end').map((e) => e.agent.body.random); + assert.deepEqual(ends, [want], `${want}: ${out}`); + } + } finally { await d.close(); await a.close(); } +}); + test('seeded: keploySeedRandom: false for the whole project asks no seed, says once what that costs, and the browser draws its own', async () => { const j = journal(); // An agent that answers a seed even to a begin that asked for none: the From 5fa42e0b582fe0a0d587c44cad2249e29c05c56c Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 18:33:20 +0530 Subject: [PATCH 10/73] test(playwright): rows for when a sandboxed frame's line is said A sandboxed frame's line is said when its parent's parser ends, when the parent is removed, or at the test's end. No row told those apart from a line said only once the context closed: dropping the flush at the test's end, or letting the parent's removal skip the line, left every row passing. The rows' reporter now also logs each write to a test's stdout among its steps, in order. The unparsed and unloaded rows check that the line comes before the test's wait for its calls in flight. A new test, removed, puts u0 and u1 in a frame whose parser never ends and removes that frame: its line must name no other frame and come before the test's next step. Signed-off-by: slayerjain --- playwright/test/fixtures/parse.spec.ts | 17 ++++++++++- playwright/test/fixtures/steps-reporter.cjs | 8 +++++- playwright/test/rows.test.mjs | 32 +++++++++++++++++++-- 3 files changed, 53 insertions(+), 4 deletions(-) diff --git a/playwright/test/fixtures/parse.spec.ts b/playwright/test/fixtures/parse.spec.ts index 2001e4c..b3029aa 100644 --- a/playwright/test/fixtures/parse.spec.ts +++ b/playwright/test/fixtures/parse.spec.ts @@ -7,7 +7,10 @@ import { test } from '../../dist'; // never answers: unloaded), and a third the test puts in once both have // drawn. The test then ends, and its context waits for that call until the // test times out. A 1 ms navigation timeout, so that the fixture's wait for -// the parse's end holds only by its own timeout of 0 (none). +// the parse's end holds only by its own timeout of 0 (none). removed: frame p +// at /rparsed holds u0 and u1 and a script the server answers only once the +// test asks (a browser may not end a removed frame's calls); the test removes +// p once both have drawn, and its line must come before its next step. const base = process.env.DRAWS_BASE ?? ''; test.use({ navigationTimeout: 1 }); @@ -36,3 +39,15 @@ for (const [name, waitUntil] of [['unparsed', 'commit'], ['unloaded', 'domconten await drawn(page, 'u2'); }); } + +test('removed', async ({ page }) => { + await page.goto(`${base}/removed`, { waitUntil: 'commit', timeout: 0 }); + for (const n of ['u0', 'u1']) await drawn(page, n); + const gone = page.waitForEvent('framedetached', { predicate: (f) => f.name() === 'p', timeout: 0 }); + await page.evaluate(() => document.querySelector('iframe[name="p"]')!.remove()); + await gone; + // A call to the browser and back: every promise p's removal settled has run. + await page.evaluate(() => 0); + await test.step('p removed', async () => {}); + await page.evaluate(async () => { await fetch('/rrelease'); }); +}); diff --git a/playwright/test/fixtures/steps-reporter.cjs b/playwright/test/fixtures/steps-reporter.cjs index 55fa240..a53076f 100644 --- a/playwright/test/fixtures/steps-reporter.cjs +++ b/playwright/test/fixtures/steps-reporter.cjs @@ -1,7 +1,9 @@ // The rows' reporter of every step a test ran, hooks' and fixtures' included // (Playwright's JSON report keeps only steps at the test's top level): one // JSON line {test, step, at} per step begun, appended to STEPS_FILE; at is -// the step's file name and line, '' for none. +// the step's file name and line, '' for none. And one line {test, out} for +// each write to a test's stdout, in the same file, so that a row can tell +// whether a test's line came before or after one of its steps. const fs = require('node:fs'); const path = require('node:path'); @@ -9,5 +11,9 @@ class StepsReporter { onStepBegin(test, _result, step) { if (process.env.STEPS_FILE) fs.appendFileSync(process.env.STEPS_FILE, `${JSON.stringify({ test: test.title, step: step.title, at: step.location ? `${path.basename(step.location.file)}:${step.location.line}` : '' })}\n`); } + + onStdOut(chunk, test) { + if (process.env.STEPS_FILE && test) fs.appendFileSync(process.env.STEPS_FILE, `${JSON.stringify({ test: test.title, out: String(chunk) })}\n`); + } } module.exports = StepsReporter; diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index e09183d..1685988 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -138,7 +138,10 @@ async function playwright(specs, env, cwd = PKG) { if (tests[key]) tests[key].runs.push(...runs); else { const mine = steps.filter((x) => x.test === spec.title); - tests[key] = { ...runs[0], runs, steps: mine.map((x) => x.step), at: Object.fromEntries(mine.map((x) => [x.step, x.at])) }; + const begun = mine.filter((x) => x.step !== undefined); + // order: the test's steps begun and its stdout writes, as they came. + const order = mine.map((x) => (x.step !== undefined ? { step: x.step } : { out: x.out })); + tests[key] = { ...runs[0], runs, steps: begun.map((x) => x.step), at: Object.fromEntries(begun.map((x) => [x.step, x.at])), order }; } } }; @@ -597,11 +600,15 @@ async function drawsServer() { }, }; })(); + const rheld = []; const srv = http.createServer((req, res) => { const u = new URL(req.url, 'http://x'); // /csp is served with a CSP sandbox: an opaque origin, no session storage. if (u.pathname === '/csp') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('csp')); return; } if (u.pathname === '/held.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); held.script(() => res.end('')); return; } + // parse.spec.ts's removed: /rhold.js is answered at /rrelease. + if (u.pathname === '/rhold.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); rheld.push(res); return; } + if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } res.writeHead(200, { 'content-type': 'text/html' }); if (u.pathname === '/draws') { res.end(page(``)); return; } if (u.pathname === '/frame') { res.end(page('frame')); return; } @@ -699,6 +706,14 @@ async function drawsServer() { return; } if (u.pathname === '/hold.js' || u.pathname === '/hold.png') return; + // parse.spec.ts's removed: frame p at /rparsed, which holds u0 and u1 at + // /fs, then /rhold.js. + if (u.pathname === '/removed') { res.end(page('')); return; } + if (u.pathname === '/rparsed') { + const f = (n) => ``; + res.end(page(`${f('u0')}${f('u1')}`)); + return; + } if (u.pathname === '/cost') { const n = Number(u.searchParams.get('n') || 0); const sh = Number(u.searchParams.get('shadow') || 0); @@ -1193,7 +1208,7 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a sandboxed frame's line counts the frames beside it once its parent's parser has ended, not at its parent's load; or at the test's end when the parser has not ended by then`, async () => { + test(`seeded, in ${engine}: a sandboxed frame's line counts the frames beside it once its parent's parser has ended, not at its parent's load; or at the test's end, before its wait, when the parser has not ended by then; or at once, naming none, when its parent is removed`, async () => { const j = journal(); const a = await fakeAgent(j); const d = await drawsServer(); @@ -1208,7 +1223,20 @@ for (const engine of ENGINES) { const said = t.stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${n}, `)).map((l) => l.replace(/:\d+\//, ':/')); assert.deepEqual(said, [unplacedLine(n, 'http://127.0.0.1:/fs', 'uncounted', others)], n); assert.ok(t.steps.some((s) => s.startsWith('keploy: wait for the test\'s 1 call(s) in flight') && s.endsWith(call)), `${n}: ${t.steps.join('\n')}`); + // The line comes before that wait, not when the context closes. + const line = t.order.findIndex((o) => o.out?.startsWith(`keploy: in ${n}, `)); + const wait = t.order.findIndex((o) => o.step?.startsWith('keploy: wait for the test\'s 1 call(s) in flight')); + assert.ok(line >= 0 && line < wait, `${n}: ${JSON.stringify(t.order)}`); } + // removed: the line comes once p is removed, before the test's next + // step, and names no other frame: theirs are gone with p. + const t = tests.removed; + assert.equal(t.status, 'passed', `removed: ${t.error}\n${out}`); + const said = t.stdout.split('\n').filter((l) => l.startsWith('keploy: in removed, ')).map((l) => l.replace(/:\d+\//, ':/')); + assert.deepEqual(said, [unplacedLine('removed', 'http://127.0.0.1:/fs', 'uncounted', 0)]); + const line = t.order.findIndex((o) => o.out?.startsWith('keploy: in removed, ')); + const next = t.order.findIndex((o) => o.step === 'p removed'); + assert.ok(line >= 0 && line < next, `removed: ${JSON.stringify(t.order)}`); } finally { await d.close(); await a.close(); } }); } From b136e63d4d0f01e9cb841af7cb67dab50a86d27a Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 18:35:01 +0530 Subject: [PATCH 11/73] test(playwright): row for Error's keys while the test process is seated With no Error.prepareStackTrace before the seat, the seat's accessor is enumerable, as an assignment would make the property. No row read that, so making it non-enumerable left every row passing, and an app that sets the formatter and lists Error's keys would miss it during a test. The prepareStackTrace row now checks that the formatter a test sets while seated is among Error's own keys. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 32f22e1..6079735 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -228,7 +228,11 @@ test('while seated, a draw made as a stack trace is prepared is the process\'s o assert.equal(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'), undefined); // One set while seated stays set after, as an assignment leaves it. const unseat3 = seatRandom(processRandom(S1)); - try { E.prepareStackTrace = fmt; } finally { unseat3(); } + try { + E.prepareStackTrace = fmt; + // Among Error's own keys while seated too, as an assignment lists it. + assert.ok(Object.keys(E).includes('prepareStackTrace')); + } finally { unseat3(); } assert.deepEqual(Object.getOwnPropertyDescriptor(E, 'prepareStackTrace'), { value: fmt, writable: true, enumerable: true, configurable: true }); // A test that defines the property itself keeps its definition. From 47e1bee6d01990398f485f892903615530078b79 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 18:39:07 +0530 Subject: [PATCH 12/73] test(playwright): rows for a pageless test and a document's one source Two rules no row read: - A test that takes the browser context and opens no page drew nothing in its browser, so its end says neither seeded nor own. Counting a page for every context left every row passing. opened.spec.ts gains a test that takes the context and opens no page; its end must carry no random, seeding on or off. - A document that drew before its page was named (the test's own context init script draws first) draws the browser's own numbers and is named once as drawing them; it keeps drawing its own. Working out the page again at each draw left every row passing, while the document's later draws turned seeded. The ownctxinit test now also draws after the page's scripts, and a replay must draw anew there. Signed-off-by: slayerjain --- playwright/test/fixtures/engines.spec.ts | 4 +++- playwright/test/fixtures/opened.spec.ts | 5 +++++ playwright/test/rows.test.mjs | 11 +++++++---- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/playwright/test/fixtures/engines.spec.ts b/playwright/test/fixtures/engines.spec.ts index 4dfab0d..2e770d7 100644 --- a/playwright/test/fixtures/engines.spec.ts +++ b/playwright/test/fixtures/engines.spec.ts @@ -54,7 +54,9 @@ test('ownctxinit', async ({ context }) => { const [q] = await Promise.all([context.waitForEvent('page'), p.evaluate(() => { window.open('about:blank'); })]); await q.waitForLoadState('load'); await p.goto(`${base}/draws`); - keep('ownctxinit', { early, url: p.url() }); + // A draw after the page's own scripts: the document drew its own first. + const late = await p.evaluate(() => Math.random()); + keep('ownctxinit', { early, late, url: p.url() }); }); test('shadow', async ({ page }) => { diff --git a/playwright/test/fixtures/opened.spec.ts b/playwright/test/fixtures/opened.spec.ts index 10cb360..de916c6 100644 --- a/playwright/test/fixtures/opened.spec.ts +++ b/playwright/test/fixtures/opened.spec.ts @@ -16,3 +16,8 @@ const test = mergeTests(held, keploy); test('held page', async ({ context }) => { await context.pages()[0].goto(`${process.env.DRAWS_BASE ?? ''}/draws`); }); + +// A test that takes the context and opens no page: its browser drew nothing. +keploy('no page', async ({ context }) => { + await context.cookies(); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 1685988..78c2d42 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1036,6 +1036,9 @@ for (const engine of ENGINES) { assert.notDeepEqual(p.drew.nostorage.inner, ns.inner); for (const k of ['blank', 'csp', 'data']) for (const i of [0, 1]) assert.notDeepEqual(p.drew.nostorage[k][i], ns[k][i], `${k}${i}`); assert.notEqual(p.drew.ownctxinit.early, r.drew.ownctxinit.early); + // A document that drew its own numbers before its page was named + // keeps drawing its own: it is named once, as drawing them. + assert.notEqual(p.drew.ownctxinit.late, r.drew.ownctxinit.late); assert.deepEqual({ ...p.drew.shadow, closed: null }, { ...sh, closed: null }); assert.notDeepEqual(p.drew.shadow.closed, sh.closed); assert.deepEqual(p.drew.removed, r.drew.removed); @@ -1263,7 +1266,7 @@ test('seeded: keploy answers no seed (an older recording, or an older keploy): t } finally { await a.close(); } }); -test('a page the test\'s context already holds when keploy\'s fixture gets it is the test\'s: its end says seeded, or own with seeding off', async () => { +test('a page the test\'s context already holds when keploy\'s fixture gets it is the test\'s: its end says seeded, or own with seeding off; a test that takes the context and opens no page says neither', async () => { const j = journal(); const a = await fakeAgent(j); const d = await drawsServer(); @@ -1271,9 +1274,9 @@ test('a page the test\'s context already holds when keploy\'s fixture gets it is for (const [env, want] of [[{}, 'seeded'], [{ KEPLOY_SEED_RANDOM: 'off' }, 'own']]) { const from = j.events.length; const { tests, out } = await playwright(['opened.spec.ts'], { ...keployEnv(a.url), DRAWS_BASE: d.url, ...env }); - assert.equal(tests['held page'].status, 'passed', out); - const ends = j.events.slice(from).filter((e) => e.agent && e.agent.path === '/agent/scope/end').map((e) => e.agent.body.random); - assert.deepEqual(ends, [want], `${want}: ${out}`); + for (const n of ['held page', 'no page']) assert.equal(tests[n].status, 'passed', `${n}: ${out}`); + const ends = j.events.slice(from).filter((e) => e.agent && e.agent.path === '/agent/scope/end').map((e) => [e.agent.body.name, e.agent.body.random]); + assert.deepEqual(ends, [['held page', want], ['no page', undefined]], `${want}: ${out}`); } } finally { await d.close(); await a.close(); } }); From 970b7566cd9eb34948d2618f594ec1e6645749fa Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 21:07:37 +0530 Subject: [PATCH 13/73] test(playwright): row for newPage waiting on its first document's name placePages promises a page's first document is named, and marked as the page's first, before newPage returns. The rows before used a page whose evaluate ran at once, so a newPage that did not wait for that script passed them. The new row holds the first document's script until it says, and checks newPage has not come; then that a first document that cannot be named (its page closed) fails newPage instead of handing the test a page whose numbers are not its own. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 6079735..06b6446 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -536,6 +536,33 @@ test('a page the test asked for is the test\'s own when its first document tells assert.equal(inherited.newPage, original); }); +test('newPage comes only once its page\'s first document is named, and a first document that cannot be named fails newPage', async () => { + // A page whose first document runs the script only when the row says: the + // test's next call to the page may not wait on the browser's own order. + const fakePage = () => { + const page = { scripts: [], ran: null, addInitScript: async (s) => { page.scripts.push(s.content); } }; + page.evaluate = (s) => new Promise((resolve, reject) => { page.ran = { resolve: () => { page.scripts.push(s); resolve(); }, reject }; }); + return page; + }; + const named = fakePage(), lost = fakePage(); + const pages = [named, lost]; + const context = { newPage: async () => pages.shift() }; + const placing = placePages(context, 'MK'); + let came = false; + const a = context.newPage().then((p) => { came = true; return p; }); + while (!named.ran) await new Promise((r) => setImmediate(r)); + for (let i = 0; i < 5; i++) await new Promise((r) => setImmediate(r)); + assert.equal(came, false, 'newPage came before its first document was named'); + named.ran.resolve(); + assert.equal(await a, named); + assert.deepEqual(named.scripts, [pageScript('t0'), firstScript('t0', 'MK')]); + const b = context.newPage(); + while (!lost.ran) await new Promise((r) => setImmediate(r)); + lost.ran.reject(new Error('Target page, context or browser has been closed')); + await assert.rejects(b, /has been closed/); + placing.restore(); +}); + // A frame element of a document the row makes: its tag, its src (null for // none), whether it has srcdoc, the shadow root it hosts, the frame // elements under it, and whether it has element children (inner; it has From 3ea8a2932d5a1c4f19168707341b6fde1cc656a7 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 21:17:33 +0530 Subject: [PATCH 14/73] chore(playwright): build with TypeScript 7, lint with type-aware oxlint The build moves to TypeScript 7.0, and a lint step runs oxlint with its type-aware rules (tsgolint, TypeScript 7's checker) on src in CI, on every Node leg. The rows run on Playwright 1.65.0; peerDependencies stay at @playwright/test >=1.40.0, and @types/node is Node 20's, the oldest Node the package supports. TypeScript 7 no longer loads @types packages unless tsconfig names them (types: ["node"]). Its declaration of test named playwright, the package that @playwright/test re-exports, which pnpm does not let a project see. The type of test is now written out from @playwright/test, and with it TypeScript 7's output, JavaScript and declarations, is the same as 5.9's, byte for byte. A new row fails when a declaration names a package that is not one of the peerDependencies. Lint: two copies of a collection made only to iterate it are dropped. Two lines carry a reason to keep a flagged form: a regex that replaces control characters, and an agent's version printed as it sent it. .oxlintrc.json turns off three rules that Playwright's own idioms trip: an empty fixture pattern, void on a fixture asked for its order, and methods kept to call later with their receiver. Signed-off-by: slayerjain --- .github/workflows/playwright-sdk.yml | 5 +- playwright/.oxlintrc.json | 15 + playwright/package-lock.json | 885 +++++++++++++++++++++++++- playwright/package.json | 9 +- playwright/src/agent.ts | 1 + playwright/src/index.ts | 12 +- playwright/src/random.ts | 3 +- playwright/test/declarations.test.mjs | 39 ++ playwright/tsconfig.json | 1 + 9 files changed, 929 insertions(+), 41 deletions(-) create mode 100644 playwright/.oxlintrc.json create mode 100644 playwright/test/declarations.test.mjs diff --git a/.github/workflows/playwright-sdk.yml b/.github/workflows/playwright-sdk.yml index 4b86de7..0853b1b 100644 --- a/.github/workflows/playwright-sdk.yml +++ b/.github/workflows/playwright-sdk.yml @@ -24,7 +24,7 @@ jobs: # result is its own. fail-fast: false matrix: - # Playwright 1.62 (the rows' version) needs Node.js 20 or later, and + # Playwright 1.65 (the rows' version) needs Node.js 20 or later, and # so does this package (package.json engines). node: [20, 22, 24] defaults: @@ -36,6 +36,9 @@ jobs: with: node-version: ${{ matrix.node }} - run: npm ci + # oxlint with its type-aware rules (TypeScript 7's checker), on the + # package's source. + - run: npm run lint # The rows run in Chromium, and the seeding's engine rows in Firefox and # WebKit too. - run: npx playwright install --with-deps chromium firefox webkit diff --git a/playwright/.oxlintrc.json b/playwright/.oxlintrc.json new file mode 100644 index 0000000..58fe3c5 --- /dev/null +++ b/playwright/.oxlintrc.json @@ -0,0 +1,15 @@ +{ + "$schema": "./node_modules/oxlint/configuration_schema.json", + "rules": { + // Playwright reads a fixture's first parameter as an object pattern: {} + // is how a fixture asks for no other fixture. + "no-empty-pattern": "off", + // `void keployTest` asks for a fixture for its order (it runs first), not + // for its value. + "typescript/no-meaningless-void-operator": "off", + // The package keeps the methods it wraps (newPage, querySelectorAll) and + // calls them with their own receiver (Reflect.apply), and names its + // wrappers with object-literal methods that never read this. + "typescript/unbound-method": "off" + } +} diff --git a/playwright/package-lock.json b/playwright/package-lock.json index 1259dd4..f9c2c03 100644 --- a/playwright/package-lock.json +++ b/playwright/package-lock.json @@ -9,9 +9,11 @@ "version": "0.1.2", "license": "Apache-2.0", "devDependencies": { - "@playwright/test": "1.62.1", - "@types/node": "22.19.1", - "typescript": "5.9.3" + "@playwright/test": "1.65.0", + "@types/node": "20.19.43", + "oxlint": "^1.93.0", + "oxlint-tsgolint": "^7.0.2007", + "typescript": "^7.0.2" }, "engines": { "node": ">=20" @@ -20,14 +22,421 @@ "@playwright/test": ">=1.40.0" } }, + "node_modules/@oxlint-tsgolint/darwin-arm64": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/darwin-arm64/-/darwin-arm64-7.0.2007.tgz", + "integrity": "sha512-bjNxPFyu1wNK74epHzXrtVLrj7oEuB8jQvJZDSia2d5EZ/0YgnyhI37E9U530IUZlNCqlTWjko1iT3CKAYF9+A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@oxlint-tsgolint/darwin-x64": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/darwin-x64/-/darwin-x64-7.0.2007.tgz", + "integrity": "sha512-dCd7mzk2FeTaU46LLeXy5mQyEFICaPqlew/LpehNzEuL86D0jTYlwZHbibRqdRrvkSNgq1szz/7014FIHgyV1g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@oxlint-tsgolint/linux-arm64": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/linux-arm64/-/linux-arm64-7.0.2007.tgz", + "integrity": "sha512-qb0XUQXuTmQ8H++zsx/tQitG0/fU3BE16KGp1qXExeDgRzEc7WIlCuN1i+rmA40lty9gQs+x3a73BFODVhTZqA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@oxlint-tsgolint/linux-x64": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/linux-x64/-/linux-x64-7.0.2007.tgz", + "integrity": "sha512-75vx0067HSgxq/HEZzlHwqhcujn3bshg43fVLTx604jZdZZMQfPVCcxTh2RTEShP7ACT1PRty6vboZlWh5RtEA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@oxlint-tsgolint/win32-arm64": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/win32-arm64/-/win32-arm64-7.0.2007.tgz", + "integrity": "sha512-6zl7I+BvScfOPtnPtSEP5JA+z8NkDPGd47YnQChL5lLF/Dzs7Rasr1G+q0zGlQcP8OEgW43hW4QlUTXxPfD13Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@oxlint-tsgolint/win32-x64": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/win32-x64/-/win32-x64-7.0.2007.tgz", + "integrity": "sha512-VptFHYhBm0SIWCUkx89hmTrKVOQsDg0iB1YE3EBlvRRtCHGUd3pU1kMxPBg1wOd83ZMmIn7zB6maBROtpmrP5g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@oxlint/binding-android-arm-eabi": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-android-arm-eabi/-/binding-android-arm-eabi-1.93.0.tgz", + "integrity": "sha512-KKw70pbD5Pvq3G2lJ+WQ71XPMjCmg37rkSe2+mDqCRP3PoTSdDMuEi4UOnE8JBtyNM7bek61EC2s0Jt7S7u+Sg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-android-arm64": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-android-arm64/-/binding-android-arm64-1.93.0.tgz", + "integrity": "sha512-fvr1fTUZGelwRxKDt64pf28aL1WaWgviwz3X+E/FcRYvrlEahH0NECRz/c1SY+K+KWmZk5uAf6h1kQnyBkp/2A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-darwin-arm64": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-darwin-arm64/-/binding-darwin-arm64-1.93.0.tgz", + "integrity": "sha512-VdvSN9JRf4Es6ZzoAdvEeh8tgKdw8L4H82WmA8MMPVfPdZGCwE/nI27Z1HrRHYQhPxOh4kmnxQEioSUFrOvDaw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-darwin-x64": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-darwin-x64/-/binding-darwin-x64-1.93.0.tgz", + "integrity": "sha512-QR1V890Ml0YGRG0F/iV0K9igG+pOhe+VWd+aZnxY849rRHZf3F2gEoQks5+GVHf105X9Mh/6HdShTP5juekG2g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-freebsd-x64": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-freebsd-x64/-/binding-freebsd-x64-1.93.0.tgz", + "integrity": "sha512-jDfIbi30zZaKi+rTvig3sge+Xhi8pFFSgHxb0jzovc4qh8ETm/irKV9c7N/JsuwnYc5sbvf4IC8w1vQQ0IL+BA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-arm-gnueabihf": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.93.0.tgz", + "integrity": "sha512-qtxZ714ND/koFgVH00QgxlzZiIbRPmguaPhnHRU5DFFtrtEdOrSnZD4U/H2AzXrXkZ59eQbMBnJwbum5XohrIQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-arm-musleabihf": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm-musleabihf/-/binding-linux-arm-musleabihf-1.93.0.tgz", + "integrity": "sha512-5pJ9NjMvH7WMUqXB6bTTOSemBm39w2cod27Hu3+Yom/GhbTeNXlYcpDx1o4CgBP6fhp0YHG66TNdVaUm6sHjsA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-arm64-gnu": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.93.0.tgz", + "integrity": "sha512-pRf+7jtbt2fHE23SX7qBxRgnBZicUgsoCK1PVIjnOk+t3a6UBkpt/GrKI99WlxsKUxftz/5nv/kfKvScmYywkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-arm64-musl": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.93.0.tgz", + "integrity": "sha512-zM8Fd/K8sqGvsBcJBZ3oVZ3ctGWbszA08IaGt5gFFF5RcunppAI/ZnAXbouABvnWeJ//S44KKlfPZnDDbeiphg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-ppc64-gnu": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.93.0.tgz", + "integrity": "sha512-7qKCnvhZKDXJG2pdbLazd2hw4FxjJHbZOFrrbzyy5PdHfPLH9vnNUa1le01A8uItZuYKPBPncH+HCoEiFT4SgA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-riscv64-gnu": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-riscv64-gnu/-/binding-linux-riscv64-gnu-1.93.0.tgz", + "integrity": "sha512-2pgulxC6G8V5wE750NJE/fRkf9aO6i6yoj10wyaR2EdiNtan4OemsOtsQ0ZTXGdY8Q3IoTmlND3GYvjJsvikYg==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-riscv64-musl": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-riscv64-musl/-/binding-linux-riscv64-musl-1.93.0.tgz", + "integrity": "sha512-yWsn6Gg2PnGG5jlyDz5+pHGiMTMiy4adaa4WRMzDgzB8efaShk1JMmTHIBSVLQMh0tYXRs4CN5uJCO79r4Mi4g==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-s390x-gnu": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.93.0.tgz", + "integrity": "sha512-WCn5plwp/ezS2RDPQKp0/EXEVRr9/dDexUdRKmsMFqzNM5xBSZ1wbsi3PuMk1VfkMRnI5BuazCsziCWMXRDOlw==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-x64-gnu": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.93.0.tgz", + "integrity": "sha512-F2CGjb9LAkzH+IYTZSzd1klQ1vk5z6Utjax64WeLByCkYXbVDbsun+Oy2hLfW2MDSKCvW+jwy/9ld0EPgdeu+A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-linux-x64-musl": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-linux-x64-musl/-/binding-linux-x64-musl-1.93.0.tgz", + "integrity": "sha512-plf5DE2objcG8Tf7mXRuRa8tT8rzDkpjqXQtKoGsKHWkpgsialbON1+cCtoSEUTVbQUjXvuAm8vV9+3rR0yCBw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-openharmony-arm64": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-openharmony-arm64/-/binding-openharmony-arm64-1.93.0.tgz", + "integrity": "sha512-Kp2yJrKvoTe9FfKdO/xwLNUbs2bYTMKx0+aoQD1s0h/Tx0kSgtHSdGmrk8tTKdluosjFbo88qYRIaNIX43AfFw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-win32-arm64-msvc": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.93.0.tgz", + "integrity": "sha512-vigjcHCFsR6KjnWTEVfaCxcMYapEuaK9FZa1MpewwQ1S+Xud3v0okfkj1RySsD6RF1NRHIjSlQSalTnKIaj3nA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-win32-ia32-msvc": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-win32-ia32-msvc/-/binding-win32-ia32-msvc-1.93.0.tgz", + "integrity": "sha512-0Ot3BEe/AAr2/uQhe55r8CmPlperYk2mQmxdutOru7oU0kir6IiEVHb1TG3GORRlfra97lPHVDLhgmihiKzLrg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@oxlint/binding-win32-x64-msvc": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/@oxlint/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.93.0.tgz", + "integrity": "sha512-ZzzIIVs/Qkm8trUWYKF0OYub5ALlv50/3gsEZbKAvWgz+yNJX6uRmQ8ySPXU2Q0/Y9m4JCOjhpmC8XUYbPHX5Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, "node_modules/@playwright/test": { - "version": "1.62.1", - "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.1.tgz", - "integrity": "sha512-DTcUc8qii+cpHvtOwggMtBRMjKZHXYWdw8syRYu2vtzuq4Wxphqq4NfCs5Zt44L6mA8rfDfj+PHnxFc/FeK6mQ==", + "version": "1.65.0", + "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.65.0.tgz", + "integrity": "sha512-iGURLGuKDLakSEl7Pgfv8h7f+XLT+V861Wfhz0zlEl5nyFuKUopdkFBPKqTllTUp06aOwPuH8iuUEb3yp3MpbQ==", "dev": true, "license": "Apache-2.0", "dependencies": { - "playwright": "1.62.1" + "playwright": "1.65.0" }, "bin": { "playwright": "cli.js" @@ -37,53 +446,442 @@ } }, "node_modules/@types/node": { - "version": "22.19.1", - "resolved": "https://registry.npmjs.org/@types/node/-/node-22.19.1.tgz", - "integrity": "sha512-LCCV0HdSZZZb34qifBsyWlUmok6W7ouER+oQIGBScS8EsZsQbrtFTUrDX4hOl+CS6p7cnNC4td+qrSVGSCTUfQ==", + "version": "20.19.43", + "resolved": "https://registry.npmjs.org/@types/node/-/node-20.19.43.tgz", + "integrity": "sha512-6oYBAi5ikg4Pl+kGsoYtawUMBT2zZMCvPNF7pVLnHZfd1zf38DRiWn/gT01RYCdUqkv7Fhr+C9ot4/tb+2sVvA==", "dev": true, "license": "MIT", "dependencies": { "undici-types": "~6.21.0" } }, - "node_modules/fsevents": { - "version": "2.3.2", - "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", - "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", + "node_modules/@typescript/typescript-aix-ppc64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-aix-ppc64/-/typescript-aix-ppc64-7.0.2.tgz", + "integrity": "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==", + "cpu": [ + "ppc64" + ], "dev": true, - "hasInstallScript": true, - "license": "MIT", + "license": "Apache-2.0", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-darwin-arm64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-arm64/-/typescript-darwin-arm64-7.0.2.tgz", + "integrity": "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-darwin-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-x64/-/typescript-darwin-x64-7.0.2.tgz", + "integrity": "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", "optional": true, "os": [ "darwin" ], "engines": { - "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-freebsd-arm64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-arm64/-/typescript-freebsd-arm64-7.0.2.tgz", + "integrity": "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-freebsd-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-x64/-/typescript-freebsd-x64-7.0.2.tgz", + "integrity": "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-arm": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm/-/typescript-linux-arm-7.0.2.tgz", + "integrity": "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-arm64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm64/-/typescript-linux-arm64-7.0.2.tgz", + "integrity": "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-loong64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-loong64/-/typescript-linux-loong64-7.0.2.tgz", + "integrity": "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-mips64el": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-mips64el/-/typescript-linux-mips64el-7.0.2.tgz", + "integrity": "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-ppc64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-ppc64/-/typescript-linux-ppc64-7.0.2.tgz", + "integrity": "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-riscv64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-riscv64/-/typescript-linux-riscv64-7.0.2.tgz", + "integrity": "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-s390x": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-s390x/-/typescript-linux-s390x-7.0.2.tgz", + "integrity": "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-linux-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-linux-x64/-/typescript-linux-x64-7.0.2.tgz", + "integrity": "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-netbsd-arm64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-arm64/-/typescript-netbsd-arm64-7.0.2.tgz", + "integrity": "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-netbsd-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-x64/-/typescript-netbsd-x64-7.0.2.tgz", + "integrity": "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-openbsd-arm64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-arm64/-/typescript-openbsd-arm64-7.0.2.tgz", + "integrity": "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-openbsd-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-x64/-/typescript-openbsd-x64-7.0.2.tgz", + "integrity": "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-sunos-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-sunos-x64/-/typescript-sunos-x64-7.0.2.tgz", + "integrity": "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-win32-arm64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-win32-arm64/-/typescript-win32-arm64-7.0.2.tgz", + "integrity": "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/@typescript/typescript-win32-x64": { + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/@typescript/typescript-win32-x64/-/typescript-win32-x64-7.0.2.tgz", + "integrity": "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=16.20.0" + } + }, + "node_modules/oxlint": { + "version": "1.93.0", + "resolved": "https://registry.npmjs.org/oxlint/-/oxlint-1.93.0.tgz", + "integrity": "sha512-19JMsQFX+YvEbcUTFQxjTwhvs0Ibg0D6CFneSLgpwnACWcY/6LjhpGpkPodh0T5aIT1pjMTLlYaeBFzaVgugTA==", + "dev": true, + "license": "MIT", + "bin": { + "oxlint": "bin/oxlint" + }, + "engines": { + "node": "^20.19.0 || >=22.12.0" + }, + "funding": { + "url": "https://github.com/sponsors/oxc-project" + }, + "optionalDependencies": { + "@oxlint/binding-android-arm-eabi": "1.93.0", + "@oxlint/binding-android-arm64": "1.93.0", + "@oxlint/binding-darwin-arm64": "1.93.0", + "@oxlint/binding-darwin-x64": "1.93.0", + "@oxlint/binding-freebsd-x64": "1.93.0", + "@oxlint/binding-linux-arm-gnueabihf": "1.93.0", + "@oxlint/binding-linux-arm-musleabihf": "1.93.0", + "@oxlint/binding-linux-arm64-gnu": "1.93.0", + "@oxlint/binding-linux-arm64-musl": "1.93.0", + "@oxlint/binding-linux-ppc64-gnu": "1.93.0", + "@oxlint/binding-linux-riscv64-gnu": "1.93.0", + "@oxlint/binding-linux-riscv64-musl": "1.93.0", + "@oxlint/binding-linux-s390x-gnu": "1.93.0", + "@oxlint/binding-linux-x64-gnu": "1.93.0", + "@oxlint/binding-linux-x64-musl": "1.93.0", + "@oxlint/binding-openharmony-arm64": "1.93.0", + "@oxlint/binding-win32-arm64-msvc": "1.93.0", + "@oxlint/binding-win32-ia32-msvc": "1.93.0", + "@oxlint/binding-win32-x64-msvc": "1.93.0" + }, + "peerDependencies": { + "oxlint-tsgolint": ">=7.0.2005", + "vite-plus": "*" + }, + "peerDependenciesMeta": { + "oxlint-tsgolint": { + "optional": true + }, + "vite-plus": { + "optional": true + } + } + }, + "node_modules/oxlint-tsgolint": { + "version": "7.0.2007", + "resolved": "https://registry.npmjs.org/oxlint-tsgolint/-/oxlint-tsgolint-7.0.2007.tgz", + "integrity": "sha512-i8X2qf6VVqFzoNVP34joMKuZpO6T2uXMOgKJaSPFc6p+E3FTnbIGPRbI8K4ZApmtDTMu2CIeGfpzeK1zzGwfnA==", + "dev": true, + "license": "MIT", + "bin": { + "tsgolint": "bin/tsgolint.js" + }, + "optionalDependencies": { + "@oxlint-tsgolint/darwin-arm64": "7.0.2007", + "@oxlint-tsgolint/darwin-x64": "7.0.2007", + "@oxlint-tsgolint/linux-arm64": "7.0.2007", + "@oxlint-tsgolint/linux-x64": "7.0.2007", + "@oxlint-tsgolint/win32-arm64": "7.0.2007", + "@oxlint-tsgolint/win32-x64": "7.0.2007" } }, "node_modules/playwright": { - "version": "1.62.1", - "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz", - "integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==", + "version": "1.65.0", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.65.0.tgz", + "integrity": "sha512-F3AMWcR44qMBt4kq+6Kb6mX7bjGXshktKFOPULd7reED6ZJWevjbNRqysP7q+AATE9kdPEZBibex1BYsvpVoeA==", "dev": true, "license": "Apache-2.0", "dependencies": { - "playwright-core": "1.62.1" + "playwright-core": "1.65.0" }, "bin": { "playwright": "cli.js" }, "engines": { "node": ">=20" - }, - "optionalDependencies": { - "fsevents": "2.3.2" } }, "node_modules/playwright-core": { - "version": "1.62.1", - "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz", - "integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==", + "version": "1.65.0", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.65.0.tgz", + "integrity": "sha512-vNSKgH6YqtN7cdT/L334zo0a+YwkrpicyrHJlWWPIm/Q+Nn2SWInil01oT0mbu96U4GgoRtCYQD9CzJQgju+lA==", "dev": true, "license": "Apache-2.0", "bin": { @@ -94,17 +892,38 @@ } }, "node_modules/typescript": { - "version": "5.9.3", - "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", - "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "version": "7.0.2", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz", + "integrity": "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==", "dev": true, "license": "Apache-2.0", "bin": { - "tsc": "bin/tsc", - "tsserver": "bin/tsserver" + "tsc": "bin/tsc" }, "engines": { - "node": ">=14.17" + "node": ">=16.20.0" + }, + "optionalDependencies": { + "@typescript/typescript-aix-ppc64": "7.0.2", + "@typescript/typescript-darwin-arm64": "7.0.2", + "@typescript/typescript-darwin-x64": "7.0.2", + "@typescript/typescript-freebsd-arm64": "7.0.2", + "@typescript/typescript-freebsd-x64": "7.0.2", + "@typescript/typescript-linux-arm": "7.0.2", + "@typescript/typescript-linux-arm64": "7.0.2", + "@typescript/typescript-linux-loong64": "7.0.2", + "@typescript/typescript-linux-mips64el": "7.0.2", + "@typescript/typescript-linux-ppc64": "7.0.2", + "@typescript/typescript-linux-riscv64": "7.0.2", + "@typescript/typescript-linux-s390x": "7.0.2", + "@typescript/typescript-linux-x64": "7.0.2", + "@typescript/typescript-netbsd-arm64": "7.0.2", + "@typescript/typescript-netbsd-x64": "7.0.2", + "@typescript/typescript-openbsd-arm64": "7.0.2", + "@typescript/typescript-openbsd-x64": "7.0.2", + "@typescript/typescript-sunos-x64": "7.0.2", + "@typescript/typescript-win32-arm64": "7.0.2", + "@typescript/typescript-win32-x64": "7.0.2" } }, "node_modules/undici-types": { diff --git a/playwright/package.json b/playwright/package.json index 9556839..9782467 100644 --- a/playwright/package.json +++ b/playwright/package.json @@ -16,6 +16,7 @@ ], "scripts": { "build": "tsc -p tsconfig.json", + "lint": "oxlint --type-aware --deny-warnings src", "prepublishOnly": "npm run build", "test": "npm run build && node --test test/*.test.mjs" }, @@ -23,9 +24,11 @@ "@playwright/test": ">=1.40.0" }, "devDependencies": { - "@playwright/test": "1.62.1", - "@types/node": "22.19.1", - "typescript": "5.9.3" + "@playwright/test": "1.65.0", + "@types/node": "20.19.43", + "oxlint": "^1.93.0", + "oxlint-tsgolint": "^7.0.2007", + "typescript": "^7.0.2" }, "engines": { "node": ">=20" diff --git a/playwright/src/agent.ts b/playwright/src/agent.ts index 1b81fe5..17b6d2f 100644 --- a/playwright/src/agent.ts +++ b/playwright/src/agent.ts @@ -102,6 +102,7 @@ export class AgentClient { `this @keploy/playwright needs a keploy agent that speaks scope protocol v${PROTOCOL_VERSION}: upgrade keploy`, res.status); } if (v !== PROTOCOL_VERSION) { + // oxlint-disable-next-line typescript/no-base-to-string -- the version as the agent sent it; its whole answer follows throw new KeployError(`keploy: the keploy agent answered ${path} in scope protocol v${String(v)} (${quoted(data)}); ` + `this @keploy/playwright speaks scope protocol v${PROTOCOL_VERSION}: use a keploy and a @keploy/playwright of the same release`, res.status); } diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 4f58b6c..a60a248 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -51,7 +51,7 @@ // Any answer of the agent this package cannot trust fails the test. import { test as base, expect } from '@playwright/test'; -import type { Page, TestInfo } from '@playwright/test'; +import type { Page, PlaywrightTestArgs, PlaywrightTestOptions, PlaywrightWorkerArgs, PlaywrightWorkerOptions, TestInfo, TestType } from '@playwright/test'; import fs from 'node:fs'; import path from 'node:path'; import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError, RANDOM_OWN, RANDOM_SEEDED } from './agent'; @@ -223,7 +223,7 @@ const keployTestType = base.extend({ await use(context); } finally { restore(); - for (const say of [...waiting]) say(); + for (const say of waiting) say(); } if (inflight.size > 0) { const calls = inflight.describe(); @@ -302,8 +302,14 @@ const keployTestType = base.extend({ * auto), after that test's seeding began, so its draws would otherwise come * from that test's stream and shift the test's own: a test would draw other * numbers when replayed alone, or after keploy set an earlier test aside. + * + * Its type is written out so the declaration names @playwright/test, the + * package a project installs: TypeScript 7 would otherwise name playwright, + * the package @playwright/test re-exports, which a strict installer such as + * pnpm does not let the project see. */ -export const test = onWorkerSide(keployTestType); +export const test: TestType = + onWorkerSide(keployTestType); type Registering = (fixtures: Record) => unknown; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index f567698..6f4b217 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -74,6 +74,7 @@ export function unseededLine(name: string): string { * control character a ?), of at most 512 characters. */ export function pageAddress(at: unknown): string { + // oxlint-disable-next-line no-control-regex -- control characters are what it replaces return String(at).replace(/[\u0000-\u001f\u007f]/g, '?').slice(0, 512); } @@ -349,7 +350,7 @@ export function placePages(context: BrowserContext, mark: string): { restore: () }, // A page's first document may tell before its newPage has come. opened: async (page) => { - const came = await Promise.allSettled([...coming]); + const came = await Promise.allSettled(coming); return pages.has(page as Page) || came.some((c) => c.status === 'fulfilled' && c.value === page); }, }; diff --git a/playwright/test/declarations.test.mjs b/playwright/test/declarations.test.mjs new file mode 100644 index 0000000..e291dfa --- /dev/null +++ b/playwright/test/declarations.test.mjs @@ -0,0 +1,39 @@ +// Rows for the published declarations (dist/*.d.ts, built by npm test). +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { readdirSync, readFileSync } from 'node:fs'; +import { builtinModules } from 'node:module'; + +const dist = new URL('../dist/', import.meta.url); +const { peerDependencies } = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8')); + +// The module each `from '...'`, `import '...'` and `import("...")` names. +function specifiers(text) { + const found = []; + for (const m of text.matchAll(/\bfrom\s+(['"])([^'"]+)\1|\bimport\s+(['"])([^'"]+)\3|\bimport\(\s*(['"])([^'"]+)\5\s*\)/g)) { + found.push(m[2] ?? m[4] ?? m[6]); + } + return found; +} + +test('the reader finds each kind of module a declaration names', () => { + assert.deepEqual(specifiers(`import { a } from './a';\nimport type { B } from "b/c";\nimport 'd';\nexport declare const t: import("e").T;`), ['./a', 'b/c', 'd', 'e', './f']); +}); + +test('a declaration names only its own files, node and the packages a project installs (peerDependencies)', () => { + const files = readdirSync(dist).filter((f) => f.endsWith('.d.ts')); + assert.ok(files.includes('index.d.ts'), `dist holds no index.d.ts: ${files.join(', ')}`); + const wrong = []; + for (const f of files) { + for (const s of specifiers(readFileSync(new URL(f, dist), 'utf8'))) { + if (s.startsWith('./') || s.startsWith('../')) continue; + if (s.startsWith('node:') || builtinModules.includes(s)) continue; + if (Object.hasOwn(peerDependencies, s)) continue; + wrong.push(`${f}: ${s}`); + } + } + // A package a project does not install itself (playwright, which + // @playwright/test re-exports) is one a strict installer such as pnpm + // hides from it: the type would not resolve there. + assert.deepEqual(wrong, [], `declarations name packages outside peerDependencies (${Object.keys(peerDependencies).join(', ')})`); +}); diff --git a/playwright/tsconfig.json b/playwright/tsconfig.json index d9e9d79..d775cb4 100644 --- a/playwright/tsconfig.json +++ b/playwright/tsconfig.json @@ -3,6 +3,7 @@ "target": "ES2020", "module": "commonjs", "lib": ["ES2022", "DOM"], + "types": ["node"], "declaration": true, "outDir": "dist", "rootDir": "src", From 30e811884302a849809c0d87d358f720eda5ca16 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Fri, 13 Nov 2026 22:01:49 +0530 Subject: [PATCH 15/73] test(playwright): rows for the page script wait and a fixed page key Two promises of the seeding had no row that failed without them. newPage returns only once the init script that names the page's later documents is in place: the test's next goto must not race it, and a page that refuses the script (it was closed) fails newPage instead of handing the test a page whose later documents draw the browser's own numbers. The new row holds addInitScript until it says and checks newPage has not come, then rejects it and checks newPage fails. A document's page is decided once, when first asked. A page's first document that a test's context.addInitScript made draw before keploy named it draws the browser's own numbers; its key, which its frames and the popups it opens are placed by, stays none after the name and the mark. The new row checks that key, and that a first document named before it drew keeps the same key each time it is asked. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 56 +++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 06b6446..aa62901 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -454,6 +454,34 @@ test('a top document with no session storage draws the browser\'s own numbers an } }); +test('a page\'s first document that drew before keploy named it keeps the place it drew in, none: the frames and popups that ask it after are not given the name it got later', () => { + // A script the test added to its context draws as the first document + // begins, before newPage returns and keploy names the page: it draws the + // browser's own and says so once. Its key, which its frames and the + // popups it opens are placed by, stays none after the name and the mark. + const api = `globalThis[${JSON.stringify(DOC_KEY)}]`; + const first = (drawn) => { + const told = []; + const d = documentAt(initScript(S1, 'rep', 'MK'), 'blank', null, null, { report: 'rep', told, origin: 'null', href: 'about:blank', doc: { querySelectorAll: () => [] } }); + if (drawn) d.draws(); + d.run(firstScript('t0', 'MK')); + return { d, told }; + }; + const early = first(true); + assert.equal(early.d.run(`${api}.key()`), null); + assert.equal(early.d.run(`${api}.key()`), null); + assert.equal(early.d.run(`${api}.why()`), ''); + assert.notDeepEqual(early.d.draws(), early.d.draws()); + assert.deepEqual(early.told, ['about:blank']); + // Named before it drew: placed by the name, each time it is asked. + const named = first(false); + const k = `${S1}\nt0\nblank\n0\n`; + assert.equal(named.d.run(`${api}.key()`), k); + assert.deepEqual(named.d.draws(), documentAt(initScript(S1), 'blank', new Map(), 't0').draws()); + assert.equal(named.d.run(`${api}.key()`), k); + assert.deepEqual(named.told, []); +}); + test('each function has its own stream: extra draws of one do not move another\'s', () => { const s = initScript(S1); const d = documentAt(s); @@ -563,6 +591,34 @@ test('newPage comes only once its page\'s first document is named, and a first d placing.restore(); }); +test('newPage comes only once the script that names its page\'s later documents is in place, and one the page refuses fails newPage', async () => { + // A page that takes the script for its later documents only when the row + // says: the test's next goto may not wait on the browser's own order. + const fakePage = () => { + const page = { scripts: [], added: null, evaluate: async (s) => { page.scripts.push(s); } }; + page.addInitScript = (s) => new Promise((resolve, reject) => { page.added = { resolve: () => { page.scripts.push(s.content); resolve(); }, reject }; }); + return page; + }; + const named = fakePage(), refused = fakePage(); + const pages = [named, refused]; + const context = { newPage: async () => pages.shift() }; + const placing = placePages(context, 'MK'); + let came = false; + const a = context.newPage().then((p) => { came = true; return p; }); + while (!named.added) await new Promise((r) => setImmediate(r)); + for (let i = 0; i < 5; i++) await new Promise((r) => setImmediate(r)); + assert.equal(came, false, 'newPage came before its later documents\' script was in place'); + named.added.resolve(); + assert.equal(await a, named); + assert.deepEqual(named.scripts, [pageScript('t0'), firstScript('t0', 'MK')]); + const b = context.newPage(); + while (!refused.added) await new Promise((r) => setImmediate(r)); + refused.added.reject(new Error('Target page, context or browser has been closed')); + await assert.rejects(b, /has been closed/); + assert.deepEqual(refused.scripts, []); + placing.restore(); +}); + // A frame element of a document the row makes: its tag, its src (null for // none), whether it has srcdoc, the shadow root it hosts, the frame // elements under it, and whether it has element children (inner; it has From dd9f362ef08403873ea08833538aa1658594e372 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 02:17:15 +0530 Subject: [PATCH 16/73] test(playwright): rows for the seeded functions' property flags The seeding puts its own Math.random, getRandomValues and randomUUID in place with the flags the original had. Nothing checked that: a seeded page whose code lists Math's keys (Object.keys, a for-in, a spread) would see random there if the seeding made it enumerable. The first row reads each function's flags in a document with no seeding and in a seeded one, checks the seeded document has its own functions with the same flags, and that Object.keys(Math) is empty. The second row covers the two other cases the seeding handles: a Math.random that a script before it made read-only stays read-only, and a method that Crypto.prototype only inherits gets an own one, writable and not enumerable. Both draw the same seeded numbers as a plain seeded document. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 41 +++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index aa62901..849681b 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -309,6 +309,47 @@ test('a seeded document draws the same numbers for the same seed and path, other assert.notDeepEqual(documentAt('', '/p').draws(), documentAt('', '/p').draws()); }); +// Each seeded function's own property flags and source, as a document sees them. +const FLAGS = `JSON.stringify([[Math, 'random'], [Crypto.prototype, 'getRandomValues'], [Crypto.prototype, 'randomUUID']].map(([o, k]) => { + const p = Object.getOwnPropertyDescriptor(o, k); + return p ? [k, p.writable, p.enumerable, p.configurable, String(p.value)] : [k]; +}))`; + +test('a seeded document\'s Math.random, getRandomValues and randomUUID keep the browser\'s property flags: a page that lists Math\'s keys sees none', () => { + const own = JSON.parse(documentAt(null, '/p').run(FLAGS)); + const seeded = JSON.parse(documentAt(initScript(S1), '/p').run(FLAGS)); + // The seeding put its own functions in place, with the flags the browser's own had. + for (const [i, f] of seeded.entries()) assert.notEqual(f[4], own[i][4], f[0]); + assert.deepEqual(seeded.map((f) => f.slice(0, 4)), own.map((f) => f.slice(0, 4))); + assert.deepEqual(own.map((f) => f.slice(0, 4)), [['random', true, false, true], ['getRandomValues', true, false, true], ['randomUUID', true, false, true]]); + assert.deepEqual(JSON.parse(documentAt(initScript(S1), '/p').run('JSON.stringify(Object.keys(Math))')), []); +}); + +test('the seeding keeps a Math.random a script before it made read-only read-only, and gives a method Crypto.prototype only inherits an own one, writable and not enumerable; both draw the seeded numbers', () => { + const seeded = documentAt(initScript(S1), '/p').draws(); + const FROZEN = "Object.defineProperty(Math, 'random', { writable: false });\n"; + const frozenOwn = JSON.parse(documentAt(FROZEN, '/p').run(FLAGS)); + assert.deepEqual(frozenOwn[0].slice(0, 4), ['random', false, false, true]); + const frozen = documentAt(FROZEN + initScript(S1), '/p'); + const frozenFlags = JSON.parse(frozen.run(FLAGS)); + assert.deepEqual(frozenFlags[0].slice(0, 4), frozenOwn[0].slice(0, 4)); + assert.notEqual(frozenFlags[0][4], frozenOwn[0][4]); + assert.deepEqual(frozen.draws(), seeded); + // As a browser whose Crypto.prototype inherits both methods from a base. + const INHERITED = `{ + const base = Object.create(Object.getPrototypeOf(Crypto.prototype)); + for (const k of ['getRandomValues', 'randomUUID']) { + Object.defineProperty(base, k, Object.getOwnPropertyDescriptor(Crypto.prototype, k)); + delete Crypto.prototype[k]; + } + Object.setPrototypeOf(Crypto.prototype, base); +}\n`; + assert.deepEqual(JSON.parse(documentAt(INHERITED, '/p').run(FLAGS)).slice(1), [['getRandomValues'], ['randomUUID']]); + const inherited = documentAt(INHERITED + initScript(S1), '/p'); + assert.deepEqual(JSON.parse(inherited.run(FLAGS)).slice(1).map((f) => f.slice(0, 4)), [['getRandomValues', true, false, true], ['randomUUID', true, false, true]]); + assert.deepEqual(inherited.draws(), seeded); +}); + test('each page the test opened draws its own numbers at one path, the same ones each run; its place is named in every document after the seeding script', () => { const s = initScript(S1); const first = documentAt(s, '/p', new Map(), 't0').draws(); From f7288048b5d016b881c0a92d10bbbac21bd16426 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 04:45:05 +0530 Subject: [PATCH 17/73] feat(playwright): seed faker's instances in the test process per test @faker-js/faker's instances (faker, the locales' fakerXX, simpleFaker) seed their own generators from Math.random as the package loads, before any test, so seeding Math.random never reached them: every name, email and ID a suite made with faker differed at replay while a mocked dependency answered with the recording's. Each test's seat now also puts each instance's draws (faker 10's fakerCore.randomizer, faker 8 and 9's _randomizer) on a stream of the test's seed named by its export, so a replay makes the data its recording made and one instance's draws move neither another's nor Math.random's. The instances are those of the module the test file's own import loads, resolved from the test file: faker 10 is one ES module for CommonJS and ES module test files alike (require(esm)), and faker 9's ES module test files get its ES module copy. They are found, and the module loaded, before the seat, so faker's own draws as it loads are the worker's. While a worker fixture is set up, and once a test seeds an instance itself, that instance draws its own; at the test's end each instance's own generator is back, unmoved. A faker that is installed where the tests import it but cannot be loaded is named once per worker, with the reason and the next step. The seeded line and the README say so; faker 10.6.0 is a dev dependency for the rows. Signed-off-by: slayerjain --- playwright/README.md | 5 +- playwright/package-lock.json | 18 ++ playwright/package.json | 1 + playwright/src/index.ts | 22 +- playwright/src/random.ts | 257 +++++++++++++++++++- playwright/test/fixtures/faker-esm.spec.mts | 9 + playwright/test/fixtures/faker.spec.ts | 41 ++++ playwright/test/random.test.mjs | 216 +++++++++++++++- playwright/test/rows.test.mjs | 47 ++++ 9 files changed, 598 insertions(+), 18 deletions(-) create mode 100644 playwright/test/fixtures/faker-esm.spec.mts create mode 100644 playwright/test/fixtures/faker.spec.ts diff --git a/playwright/README.md b/playwright/README.md index c8a05d5..7ba770e 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -50,6 +50,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Seeded random numbers.** Each test's browser draws its random numbers (`Math.random`, `crypto.getRandomValues`, `crypto.randomUUID`) from a seeded stream, so a replay draws the numbers its recording drew. A page that picks what to load at random (an A/B coin, such as stripe.js's at each page load) then makes the same calls in replay as in its recording, which keploy can answer. - **Where the seed comes from.** Each test's begin sends a seed drawn afresh (`seed`, 128 bits as hex). keploy answers the seed to draw from. A record answers the begin's own, and keeps it in the test's mapping entry (`seed:`, beside its `at:`). A replay (and keploy's auto-replay) answers the one the recording kept. So two tests draw different numbers, and recording a test again draws new ones: an ID the browser makes is not sent twice to a real server. The seed is not derived from the test's name, so a test whose entry is renamed keeps its seed with the rest of the entry. - **The test process.** The Node worker that runs the test draws `Math.random` from the same seed, from its own stream, from the test's begin to its end: its `beforeEach` and `afterEach` hooks and its fixtures included. So data a suite makes in the test process (a lead named `Lead ${Math.random().toString(36).slice(2, 7)}`) is in replay the data its recording made, and a mocked dependency's answer still matches the test's own assertion. At the test's end the worker's own `Math.random` is back, unless the test put its own stub there, which it keeps. + - **faker in the test process.** Where a test file can import `@faker-js/faker` (8, 9 or 10), the instances the package makes as it loads (`faker`, `fakerDE` and the other locales, `simpleFaker`) draw from the test's seed too, each from its own stream, so the names, emails and IDs a test makes with them are in replay the ones its recording made. keploy takes them from the module the test file's own import loads: faker 10 is one module for CommonJS and ES module test files alike, and for faker 9 an ES module test file's instances are its ES module copy's. A test that seeds an instance itself (`faker.seed(n)`) draws that instance as it seeded it, and at the test's end each instance draws its own again, its own generator not moved by the test's draws. Where faker is installed but keploy cannot load it (an ES module on a Node older than 20.19 or 22.12), the worker prints `keploy: @faker-js/faker is installed where the tests import it, but keploy cannot seed it (), ...` once, with the next step. - A draw made while a stack trace is prepared is the worker's own, not the test's. Playwright maps stack frames through source maps, and the first time a test's stack reaches a file (a helper or page-object file's `test.step`, an `expect`, a read of an Error's stack), it parses that file's map with a sort that draws `Math.random`. A worker parses each file once, in whichever test reaches it first. So those draws do not depend on which tests its worker ran before it. - To tell those draws apart, every stack trace is prepared through a wrapper while a test is seeded: `Error.prepareStackTrace` reads back a wrapper of the function last set there (the same one on each read), and setting that wrapper sets its function. At the test's end `Error.prepareStackTrace` is again a plain property holding the function last set, unless the test defined that property itself, which it keeps. - A worker fixture registered with this package's `test.extend` or `test.use` draws the worker's own `Math.random` while it is set up, until it calls `use()` (or fails before). Playwright sets a worker fixture up in the first test that asks for it, after that test's seeding began, unless it is `auto`. Its draws (a test account made once for the worker's tests) would otherwise come from that test's stream and shift the test's own. A test fixture is the test's, and draws from its stream. Anything else that draws while a worker fixture is set up (a timer or an event handler of the test's) draws the worker's own too. @@ -80,8 +81,8 @@ Any answer from the agent that this package cannot trust fails the test. That co - Seeding's rows run in Chromium. Its rows for frames, for a popup that a page's first document opens, for top documents with no session storage, and for the test's own context init script run in Firefox and WebKit too. A popup that a page's first document opens with `window.open('about:blank')` draws the browser's own numbers and is named in every engine: in Chromium it has no session storage (its opener's origin is opaque), and in Firefox and WebKit it has no place, since its document begins before `window.open` returns. - Seeding covers the pages of the `context` and `page` fixtures. These draws keep the browser's own random numbers, or the app's: - the app's server; - - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: - - a library that seeds itself from them as it loads: `@faker-js/faker`'s default instance seeds its generator from `Math.random` as its module loads, so every value it makes differs from its recording's; + - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: + - a faker instance the suite makes itself with `new Faker(...)` as a test file loads: it seeds its own generator from the worker's `Math.random` then; - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - Web Workers, Shared Workers and Service Workers, since an init script does not reach them; - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; diff --git a/playwright/package-lock.json b/playwright/package-lock.json index f9c2c03..b891092 100644 --- a/playwright/package-lock.json +++ b/playwright/package-lock.json @@ -9,6 +9,7 @@ "version": "0.1.2", "license": "Apache-2.0", "devDependencies": { + "@faker-js/faker": "10.6.0", "@playwright/test": "1.65.0", "@types/node": "20.19.43", "oxlint": "^1.93.0", @@ -22,6 +23,23 @@ "@playwright/test": ">=1.40.0" } }, + "node_modules/@faker-js/faker": { + "version": "10.6.0", + "resolved": "https://registry.npmjs.org/@faker-js/faker/-/faker-10.6.0.tgz", + "integrity": "sha512-3RQHgEtvL1Frl/d1cSreo7qhJ3Gk1OdNUai/CtZ8G+wYeRQnJih3s9xJ9/kgYekPQRdwgh0HXRPqMlzWGwivIQ==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/fakerjs" + } + ], + "license": "MIT", + "engines": { + "node": "^20.19.0 || ^22.13.0 || ^23.5.0 || >=24.0.0", + "npm": ">=10" + } + }, "node_modules/@oxlint-tsgolint/darwin-arm64": { "version": "7.0.2007", "resolved": "https://registry.npmjs.org/@oxlint-tsgolint/darwin-arm64/-/darwin-arm64-7.0.2007.tgz", diff --git a/playwright/package.json b/playwright/package.json index 9782467..2542b89 100644 --- a/playwright/package.json +++ b/playwright/package.json @@ -24,6 +24,7 @@ "@playwright/test": ">=1.40.0" }, "devDependencies": { + "@faker-js/faker": "10.6.0", "@playwright/test": "1.65.0", "@types/node": "20.19.43", "oxlint": "^1.93.0", diff --git a/playwright/src/index.ts b/playwright/src/index.ts index a60a248..0f51880 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -58,7 +58,7 @@ import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError, RANDOM_OWN, RANDOM_ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; -import { OFF_LINE, SEEDED_LINE, besideOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, type Unplaced } from './random'; +import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, type Fakers, type Unplaced } from './random'; import { Inflight } from './settle'; export { expect, KeployError }; @@ -82,6 +82,8 @@ let seedingOn = false; let saidSeeded = false; // Whether this worker has said what a test with seeding off costs. let saidOff = false; +// The reasons this worker has said it cannot seed faker, each said once. +const saidFaker = new Set(); // How many contexts this worker seeded: each one's report function has its // own name. let reports = 0; @@ -279,12 +281,26 @@ const keployTestType = base.extend({ } open = ref; seeding = s; - // The test process draws Math.random from the test's seed from here, its - // beforeEach hooks included, until its afterEach hooks and fixtures end. + // faker's instances, found (and its module loaded, when the test file + // has not loaded it) before the seat, so what faker draws as it loads is + // the worker's own, not the test's. + let fakers: Fakers = { found: [], problem: '' }; + if (s.seed !== '') { + fakers = fakersOf(testInfo.file); + if (fakers.problem !== '' && !saidFaker.has(fakers.problem)) { + saidFaker.add(fakers.problem); + process.stdout.write(`${fakerLine(fakers.problem)}\n`); + } + } + // The test process draws Math.random, and faker's instances draw, from + // the test's seed from here, its beforeEach hooks included, until its + // afterEach hooks and fixtures end. const unseat = s.seed !== '' ? seatRandom(processRandom(s.seed)) : null; + const unseatFaker = s.seed !== '' ? seatFaker(s.seed, fakers.found) : null; try { await use(); } finally { + unseatFaker?.(); unseat?.(); } open = null; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 6f4b217..084b565 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -19,12 +19,18 @@ import type { BrowserContext, Frame, Page } from '@playwright/test'; import crypto from 'node:crypto'; +import fs from 'node:fs'; +import { createRequire } from 'node:module'; +import path from 'node:path'; export const ENV_SEED_RANDOM = 'KEPLOY_SEED_RANDOM'; /** The option that turns seeding off for a project, a file or a test. */ export const OPTION_SEED_RANDOM = 'keploySeedRandom'; +/** The package whose prebuilt instances the test process seeds. */ +export const FAKER = '@faker-js/faker'; + export type Seeding = { mode: 'seeded' } | { mode: 'off' } | { mode: 'refused'; refusal: string }; /** Whether the environment leaves seeding on (unset, on) or turns it off. */ @@ -42,11 +48,13 @@ export function seedingFromEnv(env: NodeJS.ProcessEnv): Seeding { /** The line a worker shows once, at the first test it seeds. */ export const SEEDED_LINE = 'keploy: each test\'s random numbers are seeded from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew: ' + - 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random, from the test\'s begin to its end ' + + 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random and the instances ' + + `${FAKER} makes as it loads (faker, fakerDE, ...), from the test's begin to its end ` + '(its beforeEach and afterEach hooks too). ' + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + - 'the test process keeps its own crypto, and its own Math.random outside every test (beforeAll, afterAll, a file as it loads) ' + - 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up; a faker instance the suite makes itself ' + + '(new Faker(...)) as a file loads draws its own, and one it seeds itself (faker.seed(n)) draws as seeded. ' + `To turn it off, record and replay with ${ENV_SEED_RANDOM}=off, or set use: { ${OPTION_SEED_RANDOM}: false } in the Playwright config.`; /** @@ -155,13 +163,14 @@ export function freshSeed(): string { } /** - * The stream a test's process draws Math.random from under seed: sfc32 over - * the SHA-256 of the seed and the stream's name, 53 bits a draw, as Node's - * own Math.random gives. A replay draws what its recording drew only while - * this stays the same: a recording keeps the seed, not the numbers. + * The stream a test's process draws Math.random from under seed (or, named, + * another of its streams: a faker instance's): sfc32 over the SHA-256 of the + * seed and the stream's name, 53 bits a draw, as Node's own Math.random + * gives. A replay draws what its recording drew only while this stays the + * same: a recording keeps the seed, not the numbers. */ -export function processRandom(seed: string): () => number { - const h = crypto.createHash('sha256').update(`${seed}\ntest process\nMath.random`).digest(); +export function processRandom(seed: string, name = 'Math.random'): () => number { + const h = crypto.createHash('sha256').update(`${seed}\ntest process\n${name}`).digest(); let a = h.readUInt32LE(0), b = h.readUInt32LE(4), c = h.readUInt32LE(8), d = h.readUInt32LE(12); const next = (): number => { let t = (a + b) | 0; @@ -265,6 +274,236 @@ export function setAside(): () => void { return () => { held.delete(hold); }; } +/** + * What a faker instance draws from: each of its draws calls next, for a + * number in [0, 1), and its seed method calls seed. + */ +export type Randomizer = { next: () => number; seed: (...args: unknown[]) => unknown }; + +/** + * The prebuilt faker instances a test file gets, each by its export's name; + * problem: why none could be had where the file can import the package ('' + * when there is none to say). + */ +export type Fakers = { found: ReadonlyArray; problem: string }; + +/** + * Whether file, a test file, runs as an ES module, as Node and Playwright + * decide it: .mjs and .mts files do, .cjs and .cts files do not, and any + * other does when the nearest package.json above it says "type": "module". + */ +export function esmFile(file: string): boolean { + const ext = path.extname(file); + if (ext === '.mjs' || ext === '.mts') return true; + if (ext === '.cjs' || ext === '.cts') return false; + for (let d = path.dirname(file); ; d = path.dirname(d)) { + let text: string | null = null; + try { + text = fs.readFileSync(path.join(d, 'package.json'), 'utf8'); + } catch { + text = null; + } + if (text !== null) { + try { + return (JSON.parse(text) as { type?: unknown } | null)?.type === 'module'; + } catch { + return false; + } + } + if (path.dirname(d) === d) return false; + } +} + +// The conditions Node resolves an import under. +const IMPORT_CONDITIONS: readonly string[] = ['node', 'import', 'module-sync', 'default']; + +/** + * The file a package's exports give its main entry under conditions, as Node + * picks it: in a conditions object, the first key in the object's order that + * is one of them and gives a file; in an array, the first entry that gives + * one. null for none. + */ +export function exportOf(exp: unknown, conditions: readonly string[]): string | null { + if (typeof exp === 'string') return exp; + if (Array.isArray(exp)) { + for (const e of exp) { + const t = exportOf(e, conditions); + if (t !== null) return t; + } + return null; + } + if (exp === null || typeof exp !== 'object') return null; + const o = exp as Record; + const keys = Object.keys(o); + if (keys.some((k) => k.startsWith('.'))) return '.' in o ? exportOf(o['.'], conditions) : null; + for (const k of keys) { + if (!conditions.includes(k)) continue; + const t = exportOf(o[k], conditions); + if (t !== null) return t; + } + return null; +} + +// What a faker instance draws from: faker 10's fakerCore.randomizer, faker 8 +// and 9's _randomizer; null for a value that is no instance. +function randomizerOf(v: unknown): Randomizer | null { + if (v === null || typeof v !== 'object') return null; + let r: unknown; + try { + const o = v as { fakerCore?: { randomizer?: unknown } | null; _randomizer?: unknown }; + r = o.fakerCore?.randomizer ?? o._randomizer; + } catch { + return null; + } + if (r === null || typeof r !== 'object') return null; + const x = r as Partial; + return typeof x.next === 'function' && typeof x.seed === 'function' ? x as Randomizer : null; +} + +/** + * The prebuilt instances a faker module exports, each once, by its export's + * name, in the order of the names: the instances allFakers holds are those + * its fakerXX exports name, so those names come first. + */ +export function fakersIn(mod: unknown): Array<[string, Randomizer]> { + const out: Array<[string, Randomizer]> = []; + if (mod === null || (typeof mod !== 'object' && typeof mod !== 'function')) return out; + const seen = new Set(); + const add = (name: string, from: Record, k: string): void => { + let v: unknown; + try { + v = from[k]; + } catch { + return; + } + const r = randomizerOf(v); + if (r !== null && !seen.has(r)) { + seen.add(r); + out.push([name, r]); + } + }; + const m = mod as Record; + for (const k of Object.keys(m).sort()) if (k !== 'allFakers') add(k, m, k); + let all: unknown; + try { + all = m.allFakers; + } catch { + all = null; + } + if (all !== null && typeof all === 'object') { + const a = all as Record; + for (const k of Object.keys(a).sort()) add(`allFakers.${k}`, a, k); + } + return out; +} + +// The folder of the faker package that holds entry. +function rootOf(entry: string): string { + for (let d = path.dirname(entry); ; d = path.dirname(d)) { + try { + if ((JSON.parse(fs.readFileSync(path.join(d, 'package.json'), 'utf8')) as { name?: unknown } | null)?.name === FAKER) return d; + } catch { + // No package.json here, or not one keploy can read: look further up. + } + if (path.dirname(d) === d) throw new Error(`no package.json of ${FAKER} holds ${entry}`); + } +} + +// What fakersOf found for each test file. +const fakersFound = new Map(); + +/** + * The prebuilt instances of @faker-js/faker (faker, fakerDE, ...) that file, + * a test file, gets when it imports the package, so that the test process can + * seed them: those of the module the file's own import or require loads, as + * the file resolves the package, loaded here when the file has not loaded it + * yet. faker 10 is an ES module only, so a test file that Playwright runs as + * CommonJS gets it through require, the very instances an ES module test + * file imports; faker 9 has a CommonJS copy as well, so an ES module test + * file's instances are its ES module ones. None, and no problem, when the + * file cannot resolve the package; none, with the problem, when it resolves + * it but its instances cannot be had (an ES module on a Node that cannot + * require one). + */ +export function fakersOf(file: string): Fakers { + const had = fakersFound.get(file); + if (had !== undefined) return had; + const f = findFakers(file); + fakersFound.set(file, f); + return f; +} + +function findFakers(file: string): Fakers { + const req = createRequire(file); + const problem = (err: unknown): Fakers => ({ found: [], problem: err instanceof Error ? err.message : String(err) }); + let main: string; + try { + main = req.resolve(FAKER); + } catch (err) { + return (err as { code?: unknown } | null)?.code === 'MODULE_NOT_FOUND' ? { found: [], problem: '' } : problem(err); + } + try { + let entry = main; + if (esmFile(file)) { + const root = rootOf(main); + const pkg = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8')) as { exports?: unknown; main?: unknown }; + const t = pkg.exports !== undefined ? exportOf(pkg.exports, IMPORT_CONDITIONS) : typeof pkg.main === 'string' ? pkg.main : 'index.js'; + if (t === null) throw new Error(`its package.json exports no file for import`); + entry = path.resolve(root, t); + } + const found = fakersIn(req(entry)); + return { found, problem: found.length > 0 ? '' : 'it exports no instance keploy knows how to seed (faker 8, 9 and 10 do)' }; + } catch (err) { + return problem(err); + } +} + +/** + * The line a worker shows once for each reason it could not seed faker + * where a test file can import it. + */ +export function fakerLine(problem: string): string { + return `keploy: ${FAKER} is installed where the tests import it, but keploy cannot seed it (${pageAddress(problem)}), ` + + 'so what its instances make (a name, an email, an ID) differs from its recording\'s at replay while a mocked dependency answers with the recording\'s. ' + + 'Fix what the message names (an ES module needs Node 20.19, 22.12 or later to be loaded here), or seed it in a beforeEach (faker.seed(n)).'; +} + +/** + * Puts the draws of each faker instance in found on a stream of the test's + * seed, named by the instance's name, until the returned function puts its + * own back: so a replay makes the names, emails and IDs its recording made, + * and one instance's draws move neither another's nor Math.random's. Called + * just after seatRandom: while setAside holds under that seat (a worker + * fixture being set up), an instance draws its own. A test that seeds an + * instance itself (faker.seed(n)) draws that instance's own from then on, + * seeded as it asked. An instance's own generator does not move while + * seated, so outside every test it goes on as if no test had drawn. + */ +export function seatFaker(seed: string, found: Fakers['found']): () => void { + const aside = asideNow; + const puts: Array<() => void> = []; + for (const [name, r] of found) { + const own = r.next; + const ownSeed = r.seed; + const draw = processRandom(seed, `${FAKER} ${name}`); + const seated: Randomizer = { + next(this: unknown): number { return aside.size > 0 ? Reflect.apply(own, this, []) as number : draw(); }, + seed(this: unknown, ...args: unknown[]): unknown { + put(); + return Reflect.apply(ownSeed, this, args); + }, + }; + const put = (): void => { + if (r.next === seated.next) r.next = own; + if (r.seed === seated.seed) r.seed = ownSeed; + }; + r.next = seated.next; + r.seed = seated.seed; + puts.push(put); + } + return () => { for (const p of puts) p(); }; +} + // A key of the page's session storage: how many documents of each path the // tab has loaded from the document's origin, so that a document loaded again // in the same tab (a reload, a second visit) draws new numbers, as it would diff --git a/playwright/test/fixtures/faker-esm.spec.mts b/playwright/test/fixtures/faker-esm.spec.mts new file mode 100644 index 0000000..19a518b --- /dev/null +++ b/playwright/test/fixtures/faker-esm.spec.mts @@ -0,0 +1,9 @@ +import fs from 'node:fs'; +import { faker } from '@faker-js/faker'; +import { test } from '../../dist/index.js'; + +// faker in a test file Playwright runs as an ES module: the instance it +// imports is the one keploy seeds. +test('esm made', () => { + fs.writeFileSync(`${process.env.MARKER_DIR}/faker-esm made.json`, JSON.stringify([faker.string.uuid(), faker.string.uuid()])); +}); diff --git a/playwright/test/fixtures/faker.spec.ts b/playwright/test/fixtures/faker.spec.ts new file mode 100644 index 0000000..487c816 --- /dev/null +++ b/playwright/test/fixtures/faker.spec.ts @@ -0,0 +1,41 @@ +import fs from 'node:fs'; +import { faker, fakerDE, fakerFR } from '@faker-js/faker'; +import { test } from '../../dist'; + +// A suite that makes its test data with faker in the test process, as many +// do: an ID and a name, from faker's default instance and a locale's. Each +// test keeps what it made in its beforeEach, its body and its afterEach; no +// test opens a page. Playwright runs this file as CommonJS, so it gets faker +// 10, an ES module, through require. +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/faker-${name}.json`, JSON.stringify(v)); +// faker's own draw, as this file loads, before any test. +const own = faker.fakerCore.randomizer.next; +const made = new Map(); +const make = (): void => { + const t = test.info().title; + made.set(t, [...(made.get(t) ?? []), faker.string.uuid(), fakerDE.person.firstName()]); +}; + +test.beforeEach(() => make()); +test.afterEach(() => { + make(); + keep(test.info().title, made.get(test.info().title)); +}); + +test('made', () => { make(); }); +test('other', () => { make(); }); +// A test that seeds an instance itself draws it as it seeded it (an +// instance no other test draws: the seed stays with it after the test, as +// it would without keploy). +test('seeds itself', () => { + fakerFR.seed(7); + const a = fakerFR.number.int(); + fakerFR.seed(7); + keep('itself', [a, fakerFR.number.int()]); +}); +test.describe('own', () => { + test.use({ keploySeedRandom: false }); + test('mine', () => { make(); }); +}); +// Outside every test faker draws its own again. +test.afterAll(() => keep('outside', { own: faker.fakerCore.randomizer.next === own, made: [faker.string.uuid()] })); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 849681b..ef74c64 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,11 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine } from '../dist/random.js'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; import { SEED } from '../dist/agent.js'; import { OTHER, documentAt } from './draws.mjs'; @@ -29,11 +33,13 @@ test('seeding is on unless KEPLOY_SEED_RANDOM turns it off; another value is ref test('the lines say what is seeded, what is not, where the seed lives, and how to turn it off or on', () => { assert.equal(SEEDED_LINE, 'keploy: each test\'s random numbers are seeded from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew: ' + - 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random, from the test\'s begin to its end ' + + 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random and the instances ' + + '@faker-js/faker makes as it loads (faker, fakerDE, ...), from the test\'s begin to its end ' + '(its beforeEach and afterEach hooks too). ' + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + - 'the test process keeps its own crypto, and its own Math.random outside every test (beforeAll, afterAll, a file as it loads) ' + - 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up; a faker instance the suite makes itself ' + + '(new Faker(...)) as a file loads draws its own, and one it seeds itself (faker.seed(n)) draws as seeded. ' + 'To turn it off, record and replay with KEPLOY_SEED_RANDOM=off, or set use: { keploySeedRandom: false } in the Playwright config.'); assert.equal(OFF_LINE, 'keploy: random numbers are not seeded (KEPLOY_SEED_RANDOM=off, or keploySeedRandom: false), ' + 'so the test process and the test browser draw their own each run: data a test makes from them (a name, an email, an ID) differs from its recording\'s ' + @@ -293,6 +299,208 @@ test('setAside gives the process\'s own Math.random back under the seat until ea } finally { Math.random = realRandom; } }); +// A faker instance's randomizer as faker makes one: its own next and seed. +const fakeRandomizer = (v) => { + const seeds = []; + return { seeds, r: { next() { return v; }, seed(...a) { seeds.push(a); } } }; +}; + +test('a named stream of the test process: the same numbers for a seed and name on every run and every version, others for another name; Math.random\'s is the unnamed one', () => { + assert.deepEqual([processRandom(S1, 'Math.random')(), processRandom(S1, 'Math.random')()], [processRandom(S1)(), processRandom(S1)()]); + const f = processRandom(S1, `${FAKER} faker`); + // Pinned to a reference made outside this package (sfc32 over the SHA-256 + // of the seed, "test process" and the name): a recording keeps the seed, + // not the numbers. + assert.deepEqual([f(), f()], [0.9618393967615386, 0.5619964902771277]); + assert.notEqual(processRandom(S1, `${FAKER} faker`)(), processRandom(S1)()); + assert.notEqual(processRandom(S1, `${FAKER} faker`)(), processRandom(S1, `${FAKER} fakerDE`)()); + assert.notEqual(processRandom(S1, `${FAKER} faker`)(), processRandom(S2, `${FAKER} faker`)()); +}); + +test('seatFaker puts each instance\'s draws on its named stream of the seed and its own back; setAside, and a test that seeds the instance itself, give it its own; a replacement the test put there stays', () => { + const realRandom = Math.random; + try { + const a = fakeRandomizer(0.25); + const b = fakeRandomizer(0.75); + const ownA = a.r.next; + const ownSeedA = a.r.seed; + const ownB = b.r.next; + const mine = () => 0.5; + const unseat = seatRandom(processRandom(S1)); + const unseatFaker = seatFaker(S1, [['faker', a.r], ['fakerDE', b.r]]); + const fa = processRandom(S1, `${FAKER} faker`); + const fb = processRandom(S1, `${FAKER} fakerDE`); + const m = processRandom(S1); + try { + assert.equal(a.r.next(), fa()); + // One instance's draws move neither another's nor Math.random's. + assert.equal(b.r.next(), fb()); + assert.equal(Math.random(), m()); + assert.equal(a.r.next(), fa()); + // While a worker fixture is set up, its own; no draw of the stream taken. + const back = setAside(); + assert.equal(a.r.next(), 0.25); + assert.equal(b.r.next(), 0.75); + back(); + assert.equal(a.r.next(), fa()); + // The test seeds a itself: a's own seed is called as asked, and a + // draws its own from then on; b stays seated. + assert.equal(a.r.seed(7), undefined); + assert.deepEqual(a.seeds, [[7]]); + assert.equal(a.r.next, ownA); + assert.equal(a.r.seed, ownSeedA); + assert.equal(a.r.next(), 0.25); + assert.equal(b.r.next(), fb()); + // A replacement the test put on b stays. + b.r.next = mine; + } finally { + unseatFaker(); + unseat(); + } + assert.equal(a.r.next, ownA); + assert.equal(b.r.next, mine); + assert.notEqual(b.r.next, ownB); + assert.equal(b.r.next(), 0.5); + // With nothing found, a seat changes nothing. + seatFaker(S1, [])(); + } finally { Math.random = realRandom; } +}); + +test('seatFaker puts back the seed of an instance the test did not seed, and leaves one the test replaced', () => { + const a = fakeRandomizer(0.25); + const ownSeed = a.r.seed; + const unseat = seatRandom(processRandom(S1)); + const un = seatFaker(S1, [['faker', a.r]]); + assert.notEqual(a.r.seed, ownSeed); + un(); + unseat(); + assert.equal(a.r.seed, ownSeed); + const c = fakeRandomizer(0.25); + const unseat2 = seatRandom(processRandom(S1)); + const un2 = seatFaker(S1, [['faker', c.r]]); + const theirs = () => 'theirs'; + c.r.seed = theirs; + un2(); + unseat2(); + assert.equal(c.r.seed, theirs); +}); + +test('esmFile: .mjs and .mts files are ES modules and .cjs and .cts are not; any other is one when its nearest package.json says "type": "module"', () => { + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'kpw-esm-')); + try { + for (const [f, want] of [['a.mjs', true], ['a.mts', true], ['a.cjs', false], ['a.cts', false]]) assert.equal(esmFile(path.join(tmp, f)), want, f); + fs.mkdirSync(path.join(tmp, 'm', 'deep'), { recursive: true }); + fs.writeFileSync(path.join(tmp, 'm', 'package.json'), '{"type":"module"}'); + assert.equal(esmFile(path.join(tmp, 'm', 'deep', 'a.spec.ts')), true); + assert.equal(esmFile(path.join(tmp, 'm', 'deep', 'a.cts')), false); + // The nearest package.json decides, even one with no type. + fs.writeFileSync(path.join(tmp, 'm', 'deep', 'package.json'), '{"name":"x"}'); + assert.equal(esmFile(path.join(tmp, 'm', 'deep', 'a.spec.ts')), false); + fs.writeFileSync(path.join(tmp, 'm', 'deep', 'package.json'), '{"type":"commonjs"}'); + assert.equal(esmFile(path.join(tmp, 'm', 'deep', 'a.spec.js')), false); + // One keploy cannot read as JSON: not an ES module. + fs.writeFileSync(path.join(tmp, 'm', 'deep', 'package.json'), '{'); + assert.equal(esmFile(path.join(tmp, 'm', 'deep', 'a.spec.ts')), false); + fs.writeFileSync(path.join(tmp, 'm', 'deep', 'package.json'), 'null'); + assert.equal(esmFile(path.join(tmp, 'm', 'deep', 'a.spec.ts')), false); + // This package's own test files are CommonJS. + assert.equal(esmFile(fileURLToPath(import.meta.url).replace(/\.mjs$/, '.ts')), false); + } finally { fs.rmSync(tmp, { recursive: true, force: true }); } +}); + +test('exportOf picks a package\'s main entry as Node does: the first condition in the object\'s order, nested, in an array the first that gives a file', () => { + const imp = ['node', 'import', 'module-sync', 'default']; + assert.equal(exportOf('./i.js', imp), './i.js'); + assert.equal(exportOf({ '.': './d.js', './x': './x.js' }, imp), './d.js'); + assert.equal(exportOf({ './x': './x.js' }, imp), null); + // faker 9's: require first, but an import takes default. + const f9 = { '.': { require: { types: './i.d.cts', default: './i.cjs' }, default: { types: './i.d.ts', default: './i.js' } } }; + assert.equal(exportOf(f9, imp), './i.js'); + assert.equal(exportOf(f9, ['require', 'default']), './i.cjs'); + // Key order decides, not the conditions' order. + assert.equal(exportOf({ default: './d.js', import: './i.js' }, imp), './d.js'); + assert.equal(exportOf({ browser: './b.js', import: './i.js' }, imp), './i.js'); + // A condition whose target gives no file passes to the next. + assert.equal(exportOf({ import: { browser: './b.js' }, default: './d.js' }, imp), './d.js'); + assert.equal(exportOf([{ browser: './b.js' }, './a.js'], imp), './a.js'); + assert.equal(exportOf([{ browser: './b.js' }], imp), null); + assert.equal(exportOf({ browser: './b.js' }, imp), null); + for (const v of [null, undefined, 3, true]) assert.equal(exportOf(v, imp), null, String(v)); +}); + +test('fakersIn finds each faker instance a module exports once, by its export\'s name, in the names\' order; allFakers\' are named by their fakerXX export', async () => { + const m = await import('@faker-js/faker'); + const found = fakersIn(m); + const names = found.map(([n]) => n); + assert.ok(names.includes('faker') && names.includes('fakerDE') && names.includes('simpleFaker'), names.join(' ')); + assert.ok(!names.some((n) => n.startsWith('allFakers.')), names.join(' ')); + assert.deepEqual(names, [...names].sort()); + const rs = found.map(([, r]) => r); + assert.equal(new Set(rs).size, rs.length); + assert.equal(found.find(([n]) => n === 'faker')[1], m.faker.fakerCore.randomizer); + for (const v of Object.values(m.allFakers)) assert.ok(rs.includes(v.fakerCore.randomizer)); + // faker 8 and 9 keep theirs in _randomizer; one only in allFakers is named there. + const old = fakeRandomizer(0.5).r; + const only = fakeRandomizer(0.5).r; + assert.deepEqual(fakersIn({ faker: { _randomizer: old }, allFakers: { xx: { _randomizer: only }, de: { _randomizer: old } } }), [['faker', old], ['allFakers.xx', only]]); + // Neither an instance nor a module. + assert.deepEqual(fakersIn({ a: 1, b: { fakerCore: null }, c: { _randomizer: { next() { return 0; } } }, d: { fakerCore: { randomizer: 5 } }, allFakers: 3 }), []); + for (const v of [null, undefined, 'm', 3]) assert.deepEqual(fakersIn(v), []); + const throwing = { get faker() { throw new Error('not yet'); }, other: { get fakerCore() { throw new Error('no'); } }, get allFakers() { throw new Error('no'); } }; + assert.deepEqual(fakersIn(throwing), []); +}); + +test('fakersOf gives a test file the faker instances its own import gets, CommonJS or ES module, the same ones each time; none where it cannot import faker; a problem where it can but keploy cannot load it', async () => { + const here = path.dirname(fileURLToPath(import.meta.url)); + const m = await import('@faker-js/faker'); + for (const f of ['fixtures/faker.spec.ts', 'fixtures/faker-esm.spec.mts']) { + const got = fakersOf(path.join(here, f)); + assert.equal(got.problem, '', f); + assert.equal(got.found.find(([n]) => n === 'faker')[1], m.faker.fakerCore.randomizer, f); + assert.equal(fakersOf(path.join(here, f)), got, f); + } + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'kpw-faker-')); + try { + assert.deepEqual(fakersOf(path.join(tmp, 'a.spec.ts')), { found: [], problem: '' }); + // A faker whose exports give no file for this kind of test file. + const pkg = (dir, json, files) => { + const d = path.join(tmp, dir, 'node_modules', '@faker-js', 'faker'); + fs.mkdirSync(path.join(d, 'dist'), { recursive: true }); + fs.writeFileSync(path.join(d, 'package.json'), JSON.stringify({ name: FAKER, ...json })); + for (const [n, body] of Object.entries(files)) fs.writeFileSync(path.join(d, n), body); + }; + pkg('imp', { exports: { '.': { import: './dist/i.mjs' } } }, { 'dist/i.mjs': 'export const x = 1;\n' }); + const imp = fakersOf(path.join(tmp, 'imp', 'a.spec.ts')); + assert.deepEqual(imp.found, []); + assert.match(imp.problem, /\S/); + // An ES module test file of a faker that exports nothing for import. + pkg('noimp', { exports: { '.': { require: './dist/i.cjs' } } }, { 'dist/i.cjs': 'exports.x = 1;\n' }); + assert.deepEqual(fakersOf(path.join(tmp, 'noimp', 'a.spec.mts')), { found: [], problem: 'its package.json exports no file for import' }); + // One that loads but exports no instance; one with no exports but a main. + pkg('none', { main: 'dist/m.cjs' }, { 'dist/m.cjs': 'exports.x = 1;\n' }); + assert.deepEqual(fakersOf(path.join(tmp, 'none', 'a.spec.ts')), { found: [], problem: 'it exports no instance keploy knows how to seed (faker 8, 9 and 10 do)' }); + assert.deepEqual(fakersOf(path.join(tmp, 'none', 'a.spec.mts')), { found: [], problem: 'it exports no instance keploy knows how to seed (faker 8, 9 and 10 do)' }); + // An ES module test file gets the import's copy, a CommonJS one the require's. + const inst = (v) => `{ fakerCore: { randomizer: { next() { return ${v}; }, seed() {} } } }`; + pkg('dual', { exports: { '.': { require: './dist/i.cjs', default: './dist/i.mjs' } } }, { + 'dist/i.cjs': `exports.faker = ${inst(0.1)};\n`, + 'dist/i.mjs': `export const faker = ${inst(0.2)};\n`, + }); + assert.equal(fakersOf(path.join(tmp, 'dual', 'a.spec.ts')).found[0][1].next(), 0.1); + assert.equal(fakersOf(path.join(tmp, 'dual', 'a.spec.mts')).found[0][1].next(), 0.2); + // A package keploy cannot find the root of from its entry. + pkg('noroot', { name: 'other', main: 'dist/m.cjs' }, { 'dist/m.cjs': 'exports.x = 1;\n' }); + assert.match(fakersOf(path.join(tmp, 'noroot', 'a.spec.mts')).problem, /^no package\.json of @faker-js\/faker holds /); + } finally { fs.rmSync(tmp, { recursive: true, force: true }); } +}); + +test('the line for a faker keploy cannot seed names the reason and the next step', () => { + assert.equal(fakerLine('Cannot find module x'), 'keploy: @faker-js/faker is installed where the tests import it, but keploy cannot seed it (Cannot find module x), ' + + 'so what its instances make (a name, an email, an ID) differs from its recording\'s at replay while a mocked dependency answers with the recording\'s. ' + + 'Fix what the message names (an ES module needs Node 20.19, 22.12 or later to be loaded here), or seed it in a beforeEach (faker.seed(n)).'); + assert.match(fakerLine('a\nb'), /cannot seed it \(a\?b\)/); +}); + test('a seeded document draws the same numbers for the same seed and path, others for another seed or path', () => { const s1 = initScript(S1); const a = documentAt(s1, '/p').draws(); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 78c2d42..0891683 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1386,6 +1386,53 @@ test('seeded: the test process draws Math.random from its test\'s seed in its be } finally { await rep.close(); await rec.close(); } }); +// One run of faker.spec.ts (CommonJS) and faker-esm.spec.mts (an ES +// module): what each test made with faker, and the seed each begin sent. +async function fakerRun(j, env) { + const from = j.events.length; + const { tests, tmp, out } = await playwright(['faker.spec.ts', 'faker-esm.spec.mts'], env); + for (const [k, t] of Object.entries(tests)) assert.equal(t.status, 'passed', `${k}: ${t.error}\n${out}`); + const made = {}; + for (const n of ['made', 'other', 'mine', 'itself', 'outside', 'esm made']) made[n] = JSON.parse(fs.readFileSync(path.join(tmp, `faker-${n}.json`), 'utf8')); + const begins = calls({ events: j.events.slice(from) }).filter((c) => c.path === '/agent/scope/begin').map((c) => c.body); + const seeds = Object.fromEntries(begins.map((b) => [b.name.replace(/^own › /, ''), b.seed])); + return { made, seeds, out }; +} + +test('seeded: faker\'s instances in the test process make from the test\'s seed in its beforeEach, body and afterEach, in a CommonJS test file and an ES module one, so a replay makes the data its recording made; another test, an opted-out one, a test that seeds faker itself and code outside every test make others', async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + try { + const r = await fakerRun(j, keployEnv(rec.url)); + kept = r.seeds; + // An ID and a name in each of the beforeEach, the body and the afterEach. + assert.equal(r.made.made.length, 6); + assert.notDeepEqual(r.made.made, r.made.other); + assert.equal(r.seeds.mine, undefined); + const p = await fakerRun(j, keployEnv(rep.url)); + for (const n of ['made', 'other', 'esm made']) { + assert.notEqual(p.seeds[n], kept[n], n); + assert.deepEqual(p.made[n], r.made[n], n); + } + // The opted-out test and the code after the last test make the worker's own. + assert.notDeepEqual(p.made.mine, r.made.mine); + assert.notDeepEqual(p.made.outside.made, r.made.outside.made); + for (const x of [r, p]) assert.equal(x.made.outside.own, true); + // A test that seeds an instance itself draws what that seed gives. + const { fakerFR } = await import('@faker-js/faker'); + fakerFR.seed(7); + const want = fakerFR.number.int(); + for (const x of [r, p]) assert.deepEqual(x.made.itself, [want, want]); + // A record made again makes new data. + const r2 = await fakerRun(j, keployEnv(rec.url)); + assert.notDeepEqual(r2.made.made, r.made.made); + assert.notDeepEqual(r2.made['esm made'], r.made['esm made']); + } finally { await rep.close(); await rec.close(); } +}); + test('the test process draws its own Math.random when keploy answers no seed, each test saying so, or with KEPLOY_SEED_RANDOM=off, the worker saying once what that costs', async () => { const j = journal(); const none = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); From cfd190684db92fa8809181b278fee926d4c2ab30 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 06:03:25 +0530 Subject: [PATCH 18/73] feat(playwright): seedFaker seeds the suite's own faker instances A faker instance the suite makes itself as a test file loads (new Faker(...), new SimpleFaker(...)) seeds its own generator from the worker's Math.random then, before any test, so it made other names, emails and IDs at replay than in its recording, and nothing said how to fix it. seedFaker(instance, name), exported from @keploy/playwright, seeds such an instance as the prebuilt ones are seeded: from each test's begin to its end, on a stream of the test's seed named by the name, and returns the instance. One given as a file loads, in beforeAll or while a worker fixture is set up is seeded in every test after; one given in a test until that test's end. An instance already seated (a prebuilt one) keeps its stream. It throws, naming the next step, for a value that is no faker 8, 9 or 10 instance or an empty name. The worker's seeded line now says: own Faker instances draw their own, seed them with seedFaker(instance, name). The README shows the call. Rows: the unit rows of seedFaker's streams, lifetimes and refusals, and in a CommonJS and an ES module test file an instance given at load makes the recording's data at replay, one made in a test does too, and one not given makes its own. Signed-off-by: slayerjain --- playwright/README.md | 12 +- playwright/src/index.ts | 6 +- playwright/src/random.ts | 88 ++++++++++-- playwright/test/fixtures/faker-esm.spec.mts | 9 +- playwright/test/fixtures/faker.spec.ts | 14 +- playwright/test/random.test.mjs | 151 +++++++++++++++++++- playwright/test/rows.test.mjs | 12 +- 7 files changed, 264 insertions(+), 28 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 7ba770e..ef65513 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -55,6 +55,16 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - To tell those draws apart, every stack trace is prepared through a wrapper while a test is seeded: `Error.prepareStackTrace` reads back a wrapper of the function last set there (the same one on each read), and setting that wrapper sets its function. At the test's end `Error.prepareStackTrace` is again a plain property holding the function last set, unless the test defined that property itself, which it keeps. - A worker fixture registered with this package's `test.extend` or `test.use` draws the worker's own `Math.random` while it is set up, until it calls `use()` (or fails before). Playwright sets a worker fixture up in the first test that asks for it, after that test's seeding began, unless it is `auto`. Its draws (a test account made once for the worker's tests) would otherwise come from that test's stream and shift the test's own. A test fixture is the test's, and draws from its stream. Anything else that draws while a worker fixture is set up (a timer or an event handler of the test's) draws the worker's own too. - With both, a test draws the same numbers whichever tests its worker ran before it: replayed alone, after keploy set an earlier test aside, or in a new worker after a test failed. The limits below name what a worker does once, in whichever test first needs it, that neither covers. + - **Your own faker instances.** An instance the suite makes itself as a test file loads (`new Faker(...)`, `new SimpleFaker(...)`) seeds its own generator from the worker's `Math.random` then, so it draws other numbers each run. Give it to `seedFaker` and it is seeded as the prebuilt ones are, from its own stream of each test's seed: + + ```ts + import { Faker, de, en } from '@faker-js/faker'; + import { seedFaker, test } from '@keploy/playwright'; + + const fakerMine = seedFaker(new Faker({ locale: [de, en] }), 'mine'); + ``` + + The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser's pages change: not the app, its server or the browser's workers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: @@ -82,7 +92,7 @@ Any answer from the agent that this package cannot trust fails the test. That co - Seeding covers the pages of the `context` and `page` fixtures. These draws keep the browser's own random numbers, or the app's: - the app's server; - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: - - a faker instance the suite makes itself with `new Faker(...)` as a test file loads: it seeds its own generator from the worker's `Math.random` then; + - a faker instance the suite makes itself with `new Faker(...)` as a test file loads and does not give to `seedFaker` (see above): it seeds its own generator from the worker's `Math.random` then; - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - Web Workers, Shared Workers and Service Workers, since an init script does not reach them; - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 0f51880..be4640f 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -44,8 +44,9 @@ // test's recording, so a replay's pages make the calls the recording's // made; each page draws its own, named by its place in the test's // order; the test process draws Math.random from that seed too, from -// the test's begin to its end, so data the test makes there (a name -// stamped with Math.random()) is the recording's; KEPLOY_SEED_RANDOM=off +// the test's begin to its end, and so do @faker-js/faker's prebuilt +// instances and those the suite gives seedFaker, so data the test makes +// there (a name stamped with Math.random()) is the recording's; KEPLOY_SEED_RANDOM=off // or the option keploySeedRandom: false turns it off, and the worker // says once what that costs. // Any answer of the agent this package cannot trust fails the test. @@ -62,6 +63,7 @@ import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initSc import { Inflight } from './settle'; export { expect, KeployError }; +export { seedFaker } from './random'; export type { TestRef, Outcome }; let joined: Joining | null = null; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 084b565..c9079fb 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -53,8 +53,9 @@ export const SEEDED_LINE = 'keploy: each test\'s random numbers are seeded from '(its beforeEach and afterEach hooks too). ' + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + - 'and while a worker fixture registered with this package\'s test.extend or test.use is set up; a faker instance the suite makes itself ' + - '(new Faker(...)) as a file loads draws its own, and one it seeds itself (faker.seed(n)) draws as seeded. ' + + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + + 'Own Faker instances, those the suite makes itself as a file loads (new Faker(...)), draw their own: seed them with seedFaker(instance, name) ' + + 'from this package. An instance the test seeds itself (faker.seed(n)) draws as seeded. ' + `To turn it off, record and replay with ${ENV_SEED_RANDOM}=off, or set use: { ${OPTION_SEED_RANDOM}: false } in the Playwright config.`; /** @@ -468,24 +469,43 @@ export function fakerLine(problem: string): string { 'Fix what the message names (an ES module needs Node 20.19, 22.12 or later to be loaded here), or seed it in a beforeEach (faker.seed(n)).'; } +// The faker instances the suite gave seedFaker outside a test's own code (as +// a file loads, in beforeAll, while a worker fixture is set up), each by the +// name it was given: every seat seats them. +const ownFakers = new Map(); + +// The seat in place now, if any: seedFaker seats an instance on it at once. +let fakerSeatNow: { aside: Set; add: (stream: string, r: Randomizer) => void } | null = null; + +// The stream of a test's seed that an instance given to seedFaker under name +// draws from: never one of a prebuilt instance's, whatever the name. +function ownStream(name: string): string { + return `${FAKER} own ${name}`; +} + /** - * Puts the draws of each faker instance in found on a stream of the test's - * seed, named by the instance's name, until the returned function puts its - * own back: so a replay makes the names, emails and IDs its recording made, - * and one instance's draws move neither another's nor Math.random's. Called - * just after seatRandom: while setAside holds under that seat (a worker - * fixture being set up), an instance draws its own. A test that seeds an - * instance itself (faker.seed(n)) draws that instance's own from then on, - * seeded as it asked. An instance's own generator does not move while - * seated, so outside every test it goes on as if no test had drawn. + * Puts the draws of each faker instance in found, and of each one the suite + * gave seedFaker, on a stream of the test's seed named by the instance's + * name, until the returned function puts its own back: so a replay makes the + * names, emails and IDs its recording made, and one instance's draws move + * neither another's nor Math.random's. An instance is seated once, by the + * first name it has (found's before seedFaker's). Called just after + * seatRandom: while setAside holds under that seat (a worker fixture being + * set up), an instance draws its own. A test that seeds an instance itself + * (faker.seed(n)) draws that instance's own from then on, seeded as it + * asked. An instance's own generator does not move while seated, so outside + * every test it goes on as if no test had drawn. */ export function seatFaker(seed: string, found: Fakers['found']): () => void { const aside = asideNow; const puts: Array<() => void> = []; - for (const [name, r] of found) { + const done = new Set(); + const add = (stream: string, r: Randomizer): void => { + if (done.has(r)) return; + done.add(r); const own = r.next; const ownSeed = r.seed; - const draw = processRandom(seed, `${FAKER} ${name}`); + const draw = processRandom(seed, stream); const seated: Randomizer = { next(this: unknown): number { return aside.size > 0 ? Reflect.apply(own, this, []) as number : draw(); }, seed(this: unknown, ...args: unknown[]): unknown { @@ -500,8 +520,48 @@ export function seatFaker(seed: string, found: Fakers['found']): () => void { r.next = seated.next; r.seed = seated.seed; puts.push(put); + }; + for (const [name, r] of found) add(`${FAKER} ${name}`, r); + for (const [r, name] of ownFakers) add(ownStream(name), r); + const seat = { aside, add }; + fakerSeatNow = seat; + return () => { + if (fakerSeatNow === seat) fakerSeatNow = null; + for (const p of puts) p(); + }; +} + +/** + * Seeds a faker instance the suite makes itself (new Faker(...), + * new SimpleFaker(...)) from each test's seed, as the instances + * @faker-js/faker makes as it loads are seeded: from the test's begin to its + * end, on a stream of the seed named by name, so a replay makes the names, + * emails and IDs its recording made. Returns the instance: + * + * const fakerMine = seedFaker(new Faker({ locale: [de, en] }), 'mine'); + * + * Give each instance its own name: two given one name make the same data. + * One given as a file loads, in beforeAll or while a worker fixture is set + * up is seeded in every test after; one given in a test (its beforeEach, its + * body, a test fixture) from then to that test's end. A test that seeds the + * instance itself (fakerMine.seed(n)) draws it as it seeded it. Throws, + * naming the next step, when instance is no faker instance this package can + * seed or name is empty. + */ +export function seedFaker(instance: T, name: string): T { + const r = randomizerOf(instance); + if (r === null) { + throw new TypeError(`keploy: seedFaker(instance, name) was given no faker instance it can seed: give it an instance of ${FAKER} 8, 9 or 10 ` + + '(new Faker(...) or new SimpleFaker(...))'); + } + if (typeof name !== 'string' || name.trim() === '') { + throw new TypeError('keploy: seedFaker(instance, name) was given no name: give each instance its own name (such as \'mine\'), ' + + 'which names the stream of each test\'s seed it draws from'); } - return () => { for (const p of puts) p(); }; + const seat = fakerSeatNow; + if (seat === null || seat.aside.size > 0) ownFakers.set(r, name); + seat?.add(ownStream(name), r); + return instance; } // A key of the page's session storage: how many documents of each path the diff --git a/playwright/test/fixtures/faker-esm.spec.mts b/playwright/test/fixtures/faker-esm.spec.mts index 19a518b..cb372a3 100644 --- a/playwright/test/fixtures/faker-esm.spec.mts +++ b/playwright/test/fixtures/faker-esm.spec.mts @@ -1,9 +1,10 @@ import fs from 'node:fs'; -import { faker } from '@faker-js/faker'; -import { test } from '../../dist/index.js'; +import { Faker, en, faker } from '@faker-js/faker'; +import { seedFaker, test } from '../../dist/index.js'; // faker in a test file Playwright runs as an ES module: the instance it -// imports is the one keploy seeds. +// imports is the one keploy seeds, and seedFaker is a named import. +const mine = seedFaker(new Faker({ locale: [en] }), 'esm mine'); test('esm made', () => { - fs.writeFileSync(`${process.env.MARKER_DIR}/faker-esm made.json`, JSON.stringify([faker.string.uuid(), faker.string.uuid()])); + fs.writeFileSync(`${process.env.MARKER_DIR}/faker-esm made.json`, JSON.stringify([faker.string.uuid(), faker.string.uuid(), mine.string.uuid()])); }); diff --git a/playwright/test/fixtures/faker.spec.ts b/playwright/test/fixtures/faker.spec.ts index 487c816..be015c6 100644 --- a/playwright/test/fixtures/faker.spec.ts +++ b/playwright/test/fixtures/faker.spec.ts @@ -1,6 +1,6 @@ import fs from 'node:fs'; -import { faker, fakerDE, fakerFR } from '@faker-js/faker'; -import { test } from '../../dist'; +import { Faker, de, faker, fakerDE, fakerFR } from '@faker-js/faker'; +import { seedFaker, test } from '../../dist'; // A suite that makes its test data with faker in the test process, as many // do: an ID and a name, from faker's default instance and a locale's. Each @@ -22,6 +22,16 @@ test.afterEach(() => { keep(test.info().title, made.get(test.info().title)); }); +// The suite's own instances, made as this file loads: one given to +// seedFaker draws from the test's seed, one not given draws its own. One +// made in a test seeds itself from the test's Math.random. +const mine = seedFaker(new Faker({ locale: [de] }), 'mine'); +const notGiven = new Faker({ locale: [de] }); +test('own instances', () => { + const inTest = new Faker({ locale: [de] }); + keep('own', { mine: [mine.string.uuid(), mine.person.firstName()], notGiven: notGiven.string.uuid(), inTest: inTest.string.uuid() }); +}); + test('made', () => { make(); }); test('other', () => { make(); }); // A test that seeds an instance itself draws it as it seeded it (an diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index ef74c64..726440b 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -38,8 +38,9 @@ test('the lines say what is seeded, what is not, where the seed lives, and how t '(its beforeEach and afterEach hooks too). ' + 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + - 'and while a worker fixture registered with this package\'s test.extend or test.use is set up; a faker instance the suite makes itself ' + - '(new Faker(...)) as a file loads draws its own, and one it seeds itself (faker.seed(n)) draws as seeded. ' + + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + + 'Own Faker instances, those the suite makes itself as a file loads (new Faker(...)), draw their own: seed them with seedFaker(instance, name) ' + + 'from this package. An instance the test seeds itself (faker.seed(n)) draws as seeded. ' + 'To turn it off, record and replay with KEPLOY_SEED_RANDOM=off, or set use: { keploySeedRandom: false } in the Playwright config.'); assert.equal(OFF_LINE, 'keploy: random numbers are not seeded (KEPLOY_SEED_RANDOM=off, or keploySeedRandom: false), ' + 'so the test process and the test browser draw their own each run: data a test makes from them (a name, an email, an ID) differs from its recording\'s ' + @@ -385,6 +386,150 @@ test('seatFaker puts back the seed of an instance the test did not seed, and lea assert.equal(c.r.seed, theirs); }); +// An instance as faker 10 makes one (its fakerCore's randomizer) or faker 8 +// and 9 do (its _randomizer). +const instance10 = (v) => { const f = fakeRandomizer(v); return { ...f, i: { fakerCore: { randomizer: f.r } } }; }; +const instance9 = (v) => { const f = fakeRandomizer(v); return { ...f, i: { _randomizer: f.r } }; }; + +test('seedFaker refuses, naming the next step, a value that is no faker instance it can seed, and an empty name', () => { + const noInstance = /^keploy: seedFaker\(instance, name\) was given no faker instance it can seed: give it an instance of @faker-js\/faker 8, 9 or 10 \(new Faker\(\.\.\.\) or new SimpleFaker\(\.\.\.\)\)$/; + for (const v of [null, undefined, 3, 'faker', {}, { fakerCore: null }, { fakerCore: { randomizer: { next() { return 0; } } } }, { _randomizer: 5 }]) { + assert.throws(() => seedFaker(v, 'mine'), (e) => e instanceof TypeError && noInstance.test(e.message), String(v)); + } + const noName = 'keploy: seedFaker(instance, name) was given no name: give each instance its own name (such as \'mine\'), ' + + 'which names the stream of each test\'s seed it draws from'; + for (const n of [undefined, null, 3, '', ' ']) { + const a = instance10(0.25); + assert.throws(() => seedFaker(a.i, n), (e) => e instanceof TypeError && e.message === noName, String(n)); + // A refused instance is not seeded. + const realRandom = Math.random; + const unseat = seatRandom(processRandom(S1)); + const un = seatFaker(S1, []); + try { + assert.equal(a.r.next(), 0.25); + } finally { + un(); + unseat(); + Math.random = realRandom; + } + } +}); + +test('seedFaker: an instance given outside a test is seated in every seat after, on its own stream of the seed named by its name, never a prebuilt instance\'s; it returns the instance; one name gives two instances the same draws', () => { + const realRandom = Math.random; + try { + const a = instance10(0.25); + const b = instance9(0.75); + const c = instance10(0.5); + const ownA = a.r.next; + assert.equal(seedFaker(a.i, 'faker'), a.i); + assert.equal(seedFaker(b.i, 'theirs'), b.i); + assert.equal(seedFaker(c.i, 'theirs'), c.i); + // Given, it still draws its own outside every seat. + assert.equal(a.r.next(), 0.25); + for (const seed of [S1, S2]) { + const unseat = seatRandom(processRandom(seed)); + const un = seatFaker(seed, []); + // Pinned to the stream's name: a recording keeps the seed, not the numbers. + const fa = processRandom(seed, '@faker-js/faker own faker'); + const fb = processRandom(seed, '@faker-js/faker own theirs'); + const fc = processRandom(seed, '@faker-js/faker own theirs'); + const prebuilt = processRandom(seed, '@faker-js/faker faker'); + try { + const x = a.r.next(); + assert.equal(x, fa()); + assert.notEqual(x, prebuilt()); + assert.equal(b.r.next(), fb()); + assert.equal(c.r.next(), fc()); + assert.equal(Math.random(), processRandom(seed)()); + // It draws its own while a worker fixture is set up. + const back = setAside(); + assert.equal(a.r.next(), 0.25); + back(); + } finally { + un(); + unseat(); + } + assert.equal(a.r.next, ownA); + assert.equal(a.r.next(), 0.25); + } + } finally { Math.random = realRandom; } +}); + +test('seedFaker: an instance given in a test is seated at once until that test\'s end, and not in the seats after; one given while a worker fixture is set up is seated in this seat and every one after; one found prebuilt is seated once, by its found name', () => { + const realRandom = Math.random; + try { + const t = instance10(0.25); + const w = instance9(0.75); + const p = instance10(0.5); + const ownT = t.r.next; + const unseat = seatRandom(processRandom(S1)); + const un = seatFaker(S1, [['faker', p.r]]); + try { + assert.equal(t.r.next(), 0.25); + assert.equal(seedFaker(t.i, 'in a test'), t.i); + const ft = processRandom(S1, '@faker-js/faker own in a test'); + assert.equal(t.r.next(), ft()); + assert.equal(t.r.next(), ft()); + // Given again, under another name, it keeps its stream. + seedFaker(t.i, 'other'); + assert.equal(t.r.next(), ft()); + // A worker fixture being set up gives it: its own until set up, then + // the test's stream. + const back = setAside(); + seedFaker(w.i, 'worker'); + assert.equal(w.r.next(), 0.75); + back(); + assert.equal(w.r.next(), processRandom(S1, '@faker-js/faker own worker')()); + // A prebuilt instance the suite gives too keeps its found stream. + const fp = processRandom(S1, '@faker-js/faker faker'); + seedFaker(p.i, 'mine too'); + assert.equal(p.r.next(), fp()); + assert.equal(p.r.next(), fp()); + } finally { + un(); + unseat(); + } + assert.equal(t.r.next, ownT); + assert.equal(t.r.next(), 0.25); + assert.equal(p.r.next(), 0.5); + // The next seat: the test's instance draws its own; the worker's is seated. + const unseat2 = seatRandom(processRandom(S2)); + const un2 = seatFaker(S2, []); + try { + assert.equal(t.r.next(), 0.25); + assert.equal(w.r.next(), processRandom(S2, '@faker-js/faker own worker')()); + assert.equal(p.r.next(), 0.5); + } finally { + un2(); + unseat2(); + } + // Given after the seat ended, an instance waits for the next seat. + const after = instance10(0.125); + seedFaker(after.i, 'after'); + assert.equal(after.r.next(), 0.125); + const unseat3 = seatRandom(processRandom(S1)); + const un3 = seatFaker(S1, []); + try { + assert.equal(after.r.next(), processRandom(S1, '@faker-js/faker own after')()); + } finally { + un3(); + unseat3(); + } + // An older seat put back after a newer one began leaves the newer one in place. + const unseat4 = seatRandom(processRandom(S2)); + const old = seatFaker(S1, []); + const newer = seatFaker(S2, []); + old(); + const late = instance10(0.375); + seedFaker(late.i, 'late'); + assert.equal(late.r.next(), processRandom(S2, '@faker-js/faker own late')()); + newer(); + unseat4(); + assert.equal(late.r.next(), 0.375); + } finally { Math.random = realRandom; } +}); + test('esmFile: .mjs and .mts files are ES modules and .cjs and .cts are not; any other is one when its nearest package.json says "type": "module"', () => { const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'kpw-esm-')); try { diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 0891683..7e56530 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1393,13 +1393,13 @@ async function fakerRun(j, env) { const { tests, tmp, out } = await playwright(['faker.spec.ts', 'faker-esm.spec.mts'], env); for (const [k, t] of Object.entries(tests)) assert.equal(t.status, 'passed', `${k}: ${t.error}\n${out}`); const made = {}; - for (const n of ['made', 'other', 'mine', 'itself', 'outside', 'esm made']) made[n] = JSON.parse(fs.readFileSync(path.join(tmp, `faker-${n}.json`), 'utf8')); + for (const n of ['made', 'other', 'mine', 'itself', 'outside', 'esm made', 'own']) made[n] = JSON.parse(fs.readFileSync(path.join(tmp, `faker-${n}.json`), 'utf8')); const begins = calls({ events: j.events.slice(from) }).filter((c) => c.path === '/agent/scope/begin').map((c) => c.body); const seeds = Object.fromEntries(begins.map((b) => [b.name.replace(/^own › /, ''), b.seed])); return { made, seeds, out }; } -test('seeded: faker\'s instances in the test process make from the test\'s seed in its beforeEach, body and afterEach, in a CommonJS test file and an ES module one, so a replay makes the data its recording made; another test, an opted-out one, a test that seeds faker itself and code outside every test make others', async () => { +test('seeded: faker\'s instances in the test process make from the test\'s seed in its beforeEach, body and afterEach, in a CommonJS test file and an ES module one, so a replay makes the data its recording made, and so do the suite\'s own instances given to seedFaker or made in a test; another test, an opted-out one, a test that seeds faker itself, code outside every test and an own instance not given make others', async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1417,6 +1417,13 @@ test('seeded: faker\'s instances in the test process make from the test\'s seed assert.notEqual(p.seeds[n], kept[n], n); assert.deepEqual(p.made[n], r.made[n], n); } + // The suite's own instances: one given to seedFaker as the file loads and + // one made in the test make the recording's data; one not given, its own. + assert.notEqual(p.seeds['own instances'], kept['own instances']); + assert.equal(r.made['esm made'].length, 3); + assert.deepEqual(p.made.own.mine, r.made.own.mine); + assert.equal(p.made.own.inTest, r.made.own.inTest); + assert.notEqual(p.made.own.notGiven, r.made.own.notGiven); // The opted-out test and the code after the last test make the worker's own. assert.notDeepEqual(p.made.mine, r.made.mine); assert.notDeepEqual(p.made.outside.made, r.made.outside.made); @@ -1430,6 +1437,7 @@ test('seeded: faker\'s instances in the test process make from the test\'s seed const r2 = await fakerRun(j, keployEnv(rec.url)); assert.notDeepEqual(r2.made.made, r.made.made); assert.notDeepEqual(r2.made['esm made'], r.made['esm made']); + assert.notDeepEqual(r2.made.own.mine, r.made.own.mine); } finally { await rep.close(); await rec.close(); } }); From 53b695367a84dcf063e316bb4c3d41ec5516dd9a Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 08:21:34 +0530 Subject: [PATCH 19/73] feat(playwright): seed the Web Workers and Shared Workers a page starts A test's pages were seeded, but a worker they started drew the browser's own random numbers, unnamed, so its draws could differ at replay. The page's Worker and SharedWorker now add a __keploy_worker mark to the script URL's query: a Web Worker's names its document, its script's path and how many workers the document started at that path; a Shared Worker's names its script's path and name only, so the test's pages still share it. A route of the test's context takes the mark off before the request leaves the browser and answers with the worker's seeding, then the app's own script as its server answered it (importScripts for a classic worker, an import for a module worker). In the worker, location shows no mark and scripts beside it load as before. Each start of a Shared Worker is the next instance. A worker keploy cannot mark (a blob: or data: URL, or a URL refused with the mark) and, in Firefox, whose route does not reach it, a Shared Worker, start as the app asked and are named once as the test runs, with the next step. Rows: the mark, the page's wrappers, the worker's seeding and the route, and in Chromium, Firefox and WebKit a classic and a module worker and a Shared Worker drawing their recording's numbers at replay. Signed-off-by: slayerjain --- playwright/README.md | 5 +- playwright/src/index.ts | 11 +- playwright/src/random.ts | 413 ++++++++++++++++++----- playwright/test/draws.mjs | 24 +- playwright/test/fixtures/workers.spec.ts | 16 + playwright/test/random.test.mjs | 173 +++++++++- playwright/test/rows.test.mjs | 67 +++- 7 files changed, 610 insertions(+), 99 deletions(-) create mode 100644 playwright/test/fixtures/workers.spec.ts diff --git a/playwright/README.md b/playwright/README.md index ef65513..b778f7a 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -66,7 +66,8 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser's pages change: not the app, its server or the browser's workers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context takes it off before the request leaves the browser and answers with the worker's seeding, then the app's own script, unchanged, as its server answered it. In the worker, `location` shows no mark, and scripts and modules beside the worker load as before; a module worker's `import.meta.url` and a stack trace show it. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own that matches a worker's script URL whole needs a pattern that allows the mark (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -94,7 +95,7 @@ Any answer from the agent that this package cannot trust fails the test. That co - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: - a faker instance the suite makes itself with `new Faker(...)` as a test file loads and does not give to `seedFaker` (see above): it seeds its own generator from the worker's `Math.random` then; - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - - Web Workers, Shared Workers and Service Workers, since an init script does not reach them; + - Service Workers, and a worker that a worker starts; - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; - `crypto.subtle`'s keys. diff --git a/playwright/src/index.ts b/playwright/src/index.ts index be4640f..026d009 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -59,7 +59,7 @@ import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError, RANDOM_OWN, RANDOM_ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; -import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, type Fakers, type Unplaced } from './random'; +import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; import { Inflight } from './settle'; export { expect, KeployError }; @@ -158,7 +158,7 @@ const keployTestType = base.extend({ keploySeedRandom: [true, { option: true }], // After keployTest: its begin's answer holds the seed. - context: async ({ context, keployTest }, use) => { + context: async ({ context, keployTest, browserName }, use) => { void keployTest; const s = seeding; if (client === null || s === null || open === null) { @@ -178,7 +178,7 @@ const keployTestType = base.extend({ let opened: (page: Page | null | undefined) => Promise = async () => false; await context.exposeBinding(report, async (source, at: unknown, why: unknown, above: unknown) => { const a = pageAddress(at); - const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' ? why : (await opened(source.page)) ? 'own' : 'popup'; + const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' || why === 'worker' || why === 'shared' ? why : (await opened(source.page)) ? 'own' : 'popup'; // Once for each address and reason: two documents at one address // (about:blank) may have no place for two reasons. const k = `${w}\n${a}`; @@ -213,7 +213,10 @@ const keployTestType = base.extend({ // The value by which the page fixture's script marks a page's first // document; the page's own scripts do not know it. const mark = freshSeed(); - await context.addInitScript({ content: initScript(s.seed, report, mark) }); + // The workers' scripts: Firefox's route does not reach a Shared + // Worker's, so there it is not marked, and says so. + await workerRoutes(context); + await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox') }); const placing = placePages(context, mark); restore = placing.restore; opened = placing.opened; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index c9079fb..2db90a4 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -48,10 +48,11 @@ export function seedingFromEnv(env: NodeJS.ProcessEnv): Seeding { /** The line a worker shows once, at the first test it seeds. */ export const SEEDED_LINE = 'keploy: each test\'s random numbers are seeded from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew: ' + - 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random and the instances ' + + 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, in its pages and the Web Workers and Shared Workers they start, ' + + 'and in the test process Math.random and the instances ' + `${FAKER} makes as it loads (faker, fakerDE, ...), from the test's begin to its end ` + '(its beforeEach and afterEach hooks too). ' + - 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + + 'Service Workers keep the browser\'s own, and so do a page the app opens and a worker that keploy cannot place or reach (each one is named); ' + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + 'Own Faker instances, those the suite makes itself as a file loads (new Faker(...)), draw their own: seed them with seedFaker(instance, name) ' + @@ -97,9 +98,17 @@ export function pageAddress(at: unknown): string { * storage to count its documents in (other than a page's first document), * or a frame of its origin inside one. */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared'; -const UNPLACED: Record, string> = { +// A worker keploy cannot reach: its line's reason (unplacedLine). +const UNREACHED: Record<'worker' | 'shared', string> = { + worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', + shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + + 'record and replay the test in Chromium or WebKit to seed it.', +}; + +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -121,6 +130,9 @@ const UNPLACED: Record, string> = { * (othersNow). */ export function unplacedLine(name: string, at: string, why: Unplaced = 'popup', others = 0): string { + if (why === 'worker' || why === 'shared') { + return `keploy: in ${name}, the ${why === 'shared' ? 'Shared Worker' : 'worker'} at ${at} draws the browser's own random numbers, so its draws may differ at replay: ${UNREACHED[why]}`; + } const n = Math.floor(Number(others)); const reason = why === 'uncounted' ? 'it is a frame under a parent of another origin with no session storage (a sandboxed frame without allow-same-origin), or a frame inside one: ' + @@ -578,6 +590,101 @@ export const PAGE_KEY = '__keploy_random_page'; // windows it opens and the frames it holds. export const DOC_KEY = '__keploy_random_doc'; +// The query parameter that marks a worker's script URL with the worker's +// name in the seeding (initScript, workerMark); the fixture's route takes it +// off before the request leaves the browser. +export const WORKER_PARAM = '__keploy_worker'; + +// A browser script's streams: hash, a string's 128 bits (cyrb128), and +// stream, sfc32 over them, 32 random bits a call. A replay draws what its +// recording drew only while these stay the same. +const STREAM_JS = ` const imul = Math.imul; + const hash = (s) => { + let h1 = 1779033703, h2 = 3144134277, h3 = 1013904242, h4 = 2773480762; + for (let i = 0; i < s.length; i++) { + const k = s.charCodeAt(i); + h1 = h2 ^ imul(h1 ^ k, 597399067); + h2 = h3 ^ imul(h2 ^ k, 2869860233); + h3 = h4 ^ imul(h3 ^ k, 951274213); + h4 = h1 ^ imul(h4 ^ k, 2716044179); + } + h1 = imul(h3 ^ (h1 >>> 18), 597399067); + h2 = imul(h4 ^ (h2 >>> 22), 2869860233); + h3 = imul(h1 ^ (h3 >>> 17), 951274213); + h4 = imul(h2 ^ (h4 >>> 19), 2716044179); + h1 ^= h2 ^ h3 ^ h4; h2 ^= h1; h3 ^= h1; h4 ^= h1; + return [h1 >>> 0, h2 >>> 0, h3 >>> 0, h4 >>> 0]; + }; + const stream = (s) => { + let [a, b, c, d] = hash(s); + const next = () => { + let t = (a + b) | 0; + a = b ^ (b >>> 9); + b = (c + (c << 3)) | 0; + c = (c << 21) | (c >>> 11); + d = (d + 1) | 0; + t = (t + d) | 0; + c = (c + t) | 0; + return t >>> 0; + }; + for (let i = 0; i < 15; i++) next(); + return next; + }; +`; + +// A browser script's seeded Math.random, crypto.getRandomValues and +// crypto.randomUUID, each drawing from from(name) ('random', 'grv', +// 'uuid'): a stream, or null for the browser's own; and define, which +// keeps a property's flags. +const DRAW_JS = ` const define = (o, name, value) => { + const d = Object.getOwnPropertyDescriptor(o, name); + Object.defineProperty(o, name, { value, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); + }; + const fill = (next, u) => { + for (let i = 0; i < u.length; i += 4) { + let w = next(); + for (let j = 0; j < 4 && i + j < u.length; j++) { u[i + j] = w & 255; w >>>= 8; } + } + }; + const realRandom = Math.random; + // 53 bits, as the browser's own Math.random gives. + define(Math, 'random', { random() { + const rnd = from('random'); + if (rnd === null) return realRandom(); + return ((rnd() >>> 5) * 67108864 + (rnd() >>> 6)) / 9007199254740992; + } }.random); + const C = typeof Crypto === 'function' ? Crypto.prototype : null; + if (C && typeof C.getRandomValues === 'function') { + const real = C.getRandomValues; + define(C, 'getRandomValues', { getRandomValues(array) { + // The browser's own call first: it refuses what the browser refuses + // (a float array, more than 65536 bytes, a this that is no Crypto). + const out = real.call(this, array); + const grv = from('grv'); + if (grv !== null) fill(grv, new Uint8Array(out.buffer, out.byteOffset, out.byteLength)); + return out; + } }.getRandomValues); + } + if (C && typeof C.randomUUID === 'function') { + const realUUID = C.randomUUID; + define(C, 'randomUUID', { randomUUID() { + const own = realUUID.call(this); + const uuid = from('uuid'); + if (uuid === null) return own; + const b = new Uint8Array(16); + fill(uuid, b); + b[6] = (b[6] & 15) | 64; + b[8] = (b[8] & 63) | 128; + let h = ''; + for (let i = 0; i < 16; i++) { + h += (b[i] + 256).toString(16).slice(1); + if (i === 3 || i === 5 || i === 7 || i === 9) h += '-'; + } + return h; + } }.randomUUID); + } +`; + /** * The init script that names the page a test opened (context.newPage(), the * page fixture's among them) in each of its documents and frames: its place @@ -715,45 +822,13 @@ export function placePages(context: BrowserContext, mark: string): { restore: () * mark: the value by which firstScript marks a page's first document ('' for * none, so no document is marked). */ -export function initScript(seed: string, report = '', mark = ''): string { +export function initScript(seed: string, report = '', mark = '', shared = true): string { return `(() => { const seed = ${JSON.stringify(seed)}; const mark = ${JSON.stringify(mark)}; + const shared = ${shared ? 'true' : 'false'}; const g = globalThis; - const imul = Math.imul; - // cyrb128: a string's 128-bit hash, a stream's four words. - const hash = (s) => { - let h1 = 1779033703, h2 = 3144134277, h3 = 1013904242, h4 = 2773480762; - for (let i = 0; i < s.length; i++) { - const k = s.charCodeAt(i); - h1 = h2 ^ imul(h1 ^ k, 597399067); - h2 = h3 ^ imul(h2 ^ k, 2869860233); - h3 = h4 ^ imul(h3 ^ k, 951274213); - h4 = h1 ^ imul(h4 ^ k, 2716044179); - } - h1 = imul(h3 ^ (h1 >>> 18), 597399067); - h2 = imul(h4 ^ (h2 >>> 22), 2869860233); - h3 = imul(h1 ^ (h3 >>> 17), 951274213); - h4 = imul(h2 ^ (h4 >>> 19), 2716044179); - h1 ^= h2 ^ h3 ^ h4; h2 ^= h1; h3 ^= h1; h4 ^= h1; - return [h1 >>> 0, h2 >>> 0, h3 >>> 0, h4 >>> 0]; - }; - // sfc32 over those words: 32 random bits a call. - const stream = (s) => { - let [a, b, c, d] = hash(s); - const next = () => { - let t = (a + b) | 0; - a = b ^ (b >>> 9); - b = (c + (c << 3)) | 0; - c = (c << 21) | (c >>> 11); - d = (d + 1) | 0; - t = (t + d) | 0; - c = (c + t) | 0; - return t >>> 0; - }; - for (let i = 0; i < 15; i++) next(); - return next; - }; +${STREAM_JS} // The fixture's function that hears of a document with no place, taken // off the window so the page does not see it. let tell = null; @@ -947,7 +1022,7 @@ export function initScript(seed: string, report = '', mark = ''): string { // An opener of another origin. given = null; } - const s = { doc, place, loads, counted, kind, up, frame, given, page: undefined, first: false, uncounted: false, opened: [], els: new WM(), held: 0, bound: new WM(), mo: null, numbers: new Mp(), counts: new Mp(), streams: null }; + const s = { doc, place, loads, counted, kind, up, frame, given, page: undefined, first: false, uncounted: false, opened: [], workers: new Mp(), els: new WM(), held: 0, bound: new WM(), mo: null, numbers: new Mp(), counts: new Mp(), streams: null }; watch(s); return s; }; @@ -1079,16 +1154,7 @@ export function initScript(seed: string, report = '', mark = ''): string { } return s.streams === false ? null : s.streams[name]; }; - const define = (o, name, value) => { - const d = Object.getOwnPropertyDescriptor(o, name); - Object.defineProperty(o, name, { value, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); - }; - const fill = (next, u) => { - for (let i = 0; i < u.length; i += 4) { - let w = next(); - for (let j = 0; j < 4 && i + j < u.length; j++) { u[i + j] = w & 255; w >>>= 8; } - } - }; +${DRAW_JS} const realOpen = g.open; if (typeof realOpen === 'function') { define(g, 'open', { open() { @@ -1097,43 +1163,220 @@ export function initScript(seed: string, report = '', mark = ''): string { return w; } }.open); } - const realRandom = Math.random; - // 53 bits, as the browser's own Math.random gives. - define(Math, 'random', { random() { - const rnd = from('random'); - if (rnd === null) return realRandom(); - return ((rnd() >>> 5) * 67108864 + (rnd() >>> 6)) / 9007199254740992; - } }.random); - const C = typeof Crypto === 'function' ? Crypto.prototype : null; - if (C && typeof C.getRandomValues === 'function') { - const real = C.getRandomValues; - define(C, 'getRandomValues', { getRandomValues(array) { - // The browser's own call first: it refuses what the browser refuses - // (a float array, more than 65536 bytes, a this that is no Crypto). - const out = real.call(this, array); - const grv = from('grv'); - if (grv !== null) fill(grv, new Uint8Array(out.buffer, out.byteOffset, out.byteLength)); - return out; - } }.getRandomValues); - } - if (C && typeof C.randomUUID === 'function') { - const realUUID = C.randomUUID; - define(C, 'randomUUID', { randomUUID() { - const own = realUUID.call(this); - const uuid = from('uuid'); - if (uuid === null) return own; - const b = new Uint8Array(16); - fill(uuid, b); - b[6] = (b[6] & 15) | 64; - b[8] = (b[8] & 63) | 128; - let h = ''; - for (let i = 0; i < 16; i++) { - h += (b[i] + 256).toString(16).slice(1); - if (i === 3 || i === 5 || i === 7 || i === 9) h += '-'; + // Workers. A Web Worker is named by its document's key, its script's + // path and how many workers the document started at that path before + // it; a Shared Worker, which the documents of a context share, by its + // script's path and its name alone. The script URL the worker starts + // with carries that name as a mark (workerMark), which the fixture's + // route takes off before the request leaves the browser: it answers + // with the worker's seeding and then the app's own script, unchanged + // (workerRoutes). A worker keploy cannot mark (a blob: or data: URL, or a + // marked URL the browser refuses), and a Shared Worker where the route + // does not reach its script (shared: false, Firefox), starts as the app + // asked and draws the browser's own numbers, and says so through report. + // A worker of a document with no place starts as the app asked: the + // document says so (its first draw is asked for here). + const construct = Reflect.construct; + const slice = Array.prototype.slice; + const hex = (s) => hash(s).map((w) => (w + 4294967296).toString(16).slice(1)).join(''); + const addressOf = (u) => (u.protocol === 'http:' || u.protocol === 'https:' ? u.origin + u.pathname : u.protocol); + const wrap = (name, tokenOf) => { + const Real = g[name]; + if (typeof Real !== 'function') return; + const W = { [name]: function (url) { + // Called without new, it throws as the browser's own does. + if (!new.target) return Real.apply(this, arguments); + const args = slice.call(arguments); + if (stale()) return construct(Real, args, new.target); + let u = null; + try { + const d = docNow(); + u = new U(String(url), d ? d.baseURI : location.href); + } catch (e) { u = null; } + // A URL the browser cannot read: its own error. + if (u === null) return construct(Real, args, new.target); + const o = args[1]; + const module = o !== null && typeof o === 'object' && o.type === 'module'; + let at = ''; + try { at = addressOf(u); } catch (e) { at = '?'; } + const why = u.protocol !== 'http:' && u.protocol !== 'https:' ? 'worker' : tokenOf(u, o); + if (why !== null && why.length === 32) { + const h = u.href; + const i = h.indexOf('#'); + const b = i < 0 ? h : h.slice(0, i); + const m = b + (b.indexOf('?') < 0 ? '?' : '&') + ${JSON.stringify(WORKER_PARAM)} + '=' + (module ? 'm' : 'w') + '.' + why + (i < 0 ? '' : h.slice(i)); + try { return construct(Real, [m].concat(args.slice(1)), new.target); } catch (e) { + // Refused with the mark (a Trusted Types policy): as the app asked. + } } - return h; - } }.randomUUID); + const w = construct(Real, args, new.target); + if (why !== '' && tell !== null) { + try { const r = tell(at, why === 'shared' ? 'shared' : 'worker', 0); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} + } + return w; + } }[name]; + try { + W.prototype = Real.prototype; + Object.defineProperty(Real.prototype, 'constructor', { value: W, writable: true, enumerable: false, configurable: true }); + Object.setPrototypeOf(W, Object.getPrototypeOf(Real)); + define(g, name, W); + } catch (e) { + // A window that takes no property: its workers draw their own. + } + }; + // A Web Worker's token, or '' for a document with no place (which says + // so at its first draw, asked for here). + wrap('Worker', (u) => { + const k = key(); + if (k === null) { from('random'); return ''; } + const s = now(); + const n = s.workers.get(u.pathname) || 0; + s.workers.set(u.pathname, n + 1); + return hex(k + 'worker\\n' + u.pathname + '\\n' + n); + }); + // A Shared Worker's token: the same in every document of the context, so + // that the documents still share one worker. + wrap('SharedWorker', (u, o) => { + if (!shared) return 'shared'; + const n = typeof o === 'string' ? o : o !== null && typeof o === 'object' && o.name !== undefined ? String(o.name) : ''; + return hex(seed + '\\nshared worker\\n' + u.pathname + '\\n' + n); + }); +})(); +`; +} + +/** + * A worker's script URL, marked (initScript), read back: its kind ('w' a + * classic worker's script, 'm' a module worker's, 'o' the app's own script + * that the answer to either loads, 'p' a module worker's seeding), its + * token, the instance n of a token ('o' and 'p'; -1 for 'w' and 'm'), and + * the URL with no mark, as the app asked for it. null for a URL with no + * mark keploy made. + */ +export type WorkerMark = { kind: 'w' | 'm' | 'o' | 'p'; token: string; n: number; url: string }; + +export function workerMark(href: string): WorkerMark | null { + const i = href.indexOf('#'); + const b = i < 0 ? href : href.slice(0, i); + const m = /[?&]__keploy_worker=([wmop])\.([0-9a-f]{32})(?:\.(\d{1,9}))?$/.exec(b); + if (m === null) return null; + const kind = m[1] as WorkerMark['kind']; + const numbered = kind === 'o' || kind === 'p'; + if (numbered !== (m[3] !== undefined)) return null; + return { kind, token: m[2], n: numbered ? Number(m[3]) : -1, url: b.slice(0, m.index) + (i < 0 ? '' : href.slice(i)) }; +} + +/** url marked as kind, for the instance n of token (workerMark). */ +export function markWorker(url: string, kind: WorkerMark['kind'], token: string, n = -1): string { + const i = url.indexOf('#'); + const b = i < 0 ? url : url.slice(0, i); + return `${b}${b.includes('?') ? '&' : '?'}${WORKER_PARAM}=${kind}.${token}${n >= 0 ? `.${n}` : ''}${i < 0 ? '' : url.slice(i)}`; +} + +/** + * The seeding of a worker's global scope, run before the app's script: + * one stream for each of Math.random, crypto.getRandomValues and + * crypto.randomUUID, from the worker's token (initScript) and n, the + * instance of the token (a Shared Worker started again after its documents + * closed is the next). Its location (href, search, toString) does not show + * the mark kind.token the worker's URL carries; a module worker's + * import.meta.url, and a stack trace, show it. + */ +export function workerScript(token: string, n: number, kind: 'w' | 'm'): string { + return `(() => { + const g = globalThis; + const k = ${JSON.stringify(`${token}\n${n}\n`)}; +${STREAM_JS} + const streams = { random: stream(k + 'Math.random'), grv: stream(k + 'crypto.getRandomValues'), uuid: stream(k + 'crypto.randomUUID') }; + const from = (name) => streams[name]; +${DRAW_JS} + const M = ${JSON.stringify(`${WORKER_PARAM}=${kind}.${token}`)}; + const strip = (h) => { + const s = String(h); + const i = s.indexOf('#'); + const b = i < 0 ? s : s.slice(0, i); + for (const c of ['?', '&']) if (b.endsWith(c + M)) return b.slice(0, b.length - M.length - 1) + (i < 0 ? '' : s.slice(i)); + return s; + }; + const L = typeof WorkerLocation === 'function' ? WorkerLocation.prototype : null; + if (L) { + for (const name of ['href', 'search']) { + try { + const d = Object.getOwnPropertyDescriptor(L, name); + if (!d || typeof d.get !== 'function') continue; + const get = d.get; + const v = name === 'href' + ? { get href() { return strip(get.call(this)); } } + : { get search() { const s = get.call(this); const i = s.indexOf(M); if (i < 1) return s; const t = s.slice(0, i - 1); return t === '?' ? '' : t; } }; + Object.defineProperty(L, name, { get: Object.getOwnPropertyDescriptor(v, name).get, set: d.set, enumerable: d.enumerable, configurable: true }); + } catch (e) {} + } + try { + const t = L.toString; + if (typeof t === 'function') define(L, 'toString', { toString() { return strip(t.call(this)); } }.toString); + } catch (e) {} } })(); `; } + +/** + * Answers each marked worker script the test's context asks for: a + * classic worker's with its seeding, then importScripts of the app's own + * script; a module worker's with an import of its seeding, then one of + * the app's own; the app's own script with what its server answered to + * the worker's request (fetched once, unchanged); a module worker's + * seeding with workerScript. An answer that is not ok goes to the worker + * as the server gave it, and a request keploy cannot fetch goes on with + * no mark. Each instance of a token (a Shared Worker started again) is the + * next n. + */ +export async function workerRoutes(context: BrowserContext): Promise { + const counts = new Map(); + const kept = new Map; body: Buffer }>(); + // The answer's headers, but its encoding and length: the body given is + // decoded, and of another length. + const headersOf = (h: Record): Record => { + const out: Record = {}; + for (const [k, v] of Object.entries(h)) if (k !== 'content-encoding' && k !== 'content-length') out[k] = v; + return out; + }; + await context.route((u) => u.searchParams.has(WORKER_PARAM), async (route) => { + const m = workerMark(route.request().url()); + if (m === null) { + await route.fallback(); + return; + } + if (m.kind === 'p') { + await route.fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); + return; + } + if (m.kind === 'o') { + const k = kept.get(`${m.token}.${m.n}`); + kept.delete(`${m.token}.${m.n}`); + if (k === undefined) await route.continue({ url: m.url }); + else await route.fulfill(k); + return; + } + const n = counts.get(m.token) ?? 0; + counts.set(m.token, n + 1); + let answer; + try { + answer = await route.fetch({ url: m.url }); + } catch { + await route.continue({ url: m.url }); + return; + } + const headers = headersOf(answer.headers()); + if (!answer.ok()) { + await route.fulfill({ status: answer.status(), headers, body: await answer.body() }); + return; + } + kept.set(`${m.token}.${n}`, { status: answer.status(), headers, body: await answer.body() }); + const own = JSON.stringify(markWorker(m.url, 'o', m.token, n)); + const body = m.kind === 'w' + ? `${workerScript(m.token, n, 'w')}importScripts(${own});\n` + : `import ${JSON.stringify(markWorker(m.url, 'p', m.token, n))};\nimport ${own};\n`; + await route.fulfill({ status: answer.status(), headers, body }); + }); +} diff --git a/playwright/test/draws.mjs b/playwright/test/draws.mjs index 1cae0a3..13fda8b 100644 --- a/playwright/test/draws.mjs +++ b/playwright/test/draws.mjs @@ -31,10 +31,11 @@ export const BLANK_DRAWS = '({ random: [Math.random(), Math.random()], bytes: Ar * reading it throws too. element * is the window's frameElement, read through a getter. doc is its document * (with URL beside it), for a document whose frames the row makes. opener - * is the window's opener (none when undefined). + * is the window's opener (none when undefined). globals: more of the + * window's globals (its URL, Worker, SharedWorker). */ export const OTHER = Symbol('a parent of another origin'); -export function documentAt(script, path = '/p', storage = new Map(), page = 't0', { report = '', told = [], origin = 'http://h', href = undefined, parent = undefined, doc = undefined, frames = undefined, element = undefined, opener = undefined } = {}) { +export function documentAt(script, path = '/p', storage = new Map(), page = 't0', { report = '', told = [], origin = 'http://h', href = undefined, parent = undefined, doc = undefined, frames = undefined, element = undefined, opener = undefined, globals = {} } = {}) { // As the browser's: a this that is no Crypto is refused. class Crypto { getRandomValues(a) { if (!(this instanceof Crypto)) throw new TypeError('Illegal invocation'); return webcrypto.getRandomValues(a); } @@ -50,6 +51,7 @@ export function documentAt(script, path = '/p', storage = new Map(), page = 't0' if (element !== undefined) Object.defineProperty(ctx, 'frameElement', { get() { return element; }, configurable: true, enumerable: true }); if (doc !== undefined) { ctx.document = doc; ctx.URL = URL; } if (opener !== undefined) ctx.opener = opener; + Object.assign(ctx, globals); vm.createContext(ctx); // A top window: its own parent and top. vm.runInContext('globalThis.parent = globalThis; globalThis.top = globalThis;', ctx); @@ -65,3 +67,21 @@ export function documentAt(script, path = '/p', storage = new Map(), page = 't0' if (page !== null) vm.runInContext(pageScript(page), ctx); return { ctx, run: (code) => vm.runInContext(code, ctx), draws: () => JSON.parse(JSON.stringify(vm.runInContext(DRAWS, ctx))) }; } + +// A worker's global scope: its own Crypto and a location whose href, +// search and toString come from WorkerLocation.prototype. +export function workerAt(href, prelude) { + class Crypto { + getRandomValues(a) { return webcrypto.getRandomValues(a); } + randomUUID() { return webcrypto.randomUUID(); } + } + class WorkerLocation { + get href() { return href; } + get search() { return new URL(href).search; } + toString() { return href; } + } + const ctx = { Crypto, crypto: new Crypto(), WorkerLocation, location: new WorkerLocation() }; + vm.createContext(ctx); + if (prelude) vm.runInContext(prelude, ctx); + return { run: (code) => vm.runInContext(code, ctx), draws: () => JSON.parse(JSON.stringify(vm.runInContext(DRAWS, ctx))) }; +} diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts new file mode 100644 index 0000000..35b6000 --- /dev/null +++ b/playwright/test/fixtures/workers.spec.ts @@ -0,0 +1,16 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). The page at /wk starts +// a classic Web Worker (which loads a script beside it with importScripts), +// a module Web Worker (which imports a module beside it) and a Shared +// Worker; each sends what it drew, its location and what it loaded. The +// test keeps them. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +test('workers', async ({ page }) => { + await page.goto(`${base}/wk`); + await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 3, null, { timeout: 20000 }); + keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 726440b..d808cec 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,13 +3,13 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, markWorker, workerMark, workerScript, workerRoutes } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { SEED } from '../dist/agent.js'; -import { OTHER, documentAt } from './draws.mjs'; +import { OTHER, documentAt, workerAt } from './draws.mjs'; // Two seeds, as keploy answers them. const S1 = '0123456789abcdef0123456789abcdef'; @@ -33,10 +33,11 @@ test('seeding is on unless KEPLOY_SEED_RANDOM turns it off; another value is ref test('the lines say what is seeded, what is not, where the seed lives, and how to turn it off or on', () => { assert.equal(SEEDED_LINE, 'keploy: each test\'s random numbers are seeded from a seed keploy keeps with the test\'s recording, so a replay draws the numbers its recording drew: ' + - 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, and in the test process Math.random and the instances ' + + 'in the test browser Math.random, crypto.getRandomValues and crypto.randomUUID, in its pages and the Web Workers and Shared Workers they start, ' + + 'and in the test process Math.random and the instances ' + '@faker-js/faker makes as it loads (faker, fakerDE, ...), from the test\'s begin to its end ' + '(its beforeEach and afterEach hooks too). ' + - 'Web Workers and Service Workers keep the browser\'s own, and so does a page the app opens that keploy cannot place (each one is named); ' + + 'Service Workers keep the browser\'s own, and so do a page the app opens and a worker that keploy cannot place or reach (each one is named); ' + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + 'Own Faker instances, those the suite makes itself as a file loads (new Faker(...)), draw their own: seed them with seedFaker(instance, name) ' + @@ -1328,3 +1329,167 @@ test('a document that a second run of the script takes over (WebKit runs it in a assert.equal(store.get(LOADS_KEY), '{"blank":1,"/p":1}'); assert.deepEqual(drew, documentAt(initScript(S1), '/p', new Map(), 't0').draws()); }); + +// Workers: the page's Worker and SharedWorker mark the script URL with the +// worker's token; the route answers the marked script with the worker's +// seeding, then the app's own script. +const TOKEN = /^[0-9a-f]{32}$/; +const workerGlobals = () => { + const made = []; + class Worker { constructor(url, opts) { made.push([String(url), opts]); } } + class SharedWorker { constructor(url, opts) { made.push([String(url), opts]); } } + return { made, globals: { URL, Worker, SharedWorker } }; +}; + +test('a worker\'s mark: what markWorker makes, workerMark reads back, the URL as the app asked for it, query and fragment kept; any other URL has none', () => { + const t = 'ab'.repeat(16); + for (const url of ['http://h/w.js', 'http://h/w.js?a=1', 'http://h/w.js?', 'http://h/w.js#f', 'http://h/w.js?a=1#f?x']) { + for (const [kind, n] of [['w', -1], ['m', -1], ['o', 0], ['p', 7]]) { + const m = markWorker(url, kind, t, n); + assert.ok(new URL(m).searchParams.has(WORKER_PARAM), m); + assert.deepEqual(workerMark(m), { kind, token: t, n, url }, m); + } + } + assert.equal(markWorker('http://h/w.js?a=1#f', 'w', t), `http://h/w.js?a=1&__keploy_worker=w.${t}#f`); + for (const u of ['http://h/w.js', `http://h/w.js?__keploy_worker=w.${t}&a=1`, `http://h/w.js?__keploy_worker=x.${t}`, `http://h/w.js?__keploy_worker=w.${t.toUpperCase()}`, + `http://h/w.js?__keploy_worker=w.${t}.0`, `http://h/w.js?__keploy_worker=o.${t}`, `http://h/w.js?__keploy_worker=w.${t.slice(1)}`]) { + assert.equal(workerMark(u), null, u); + } +}); + +test('a document\'s Web Worker starts at its script URL marked with its own token: the same each run, another for the next worker at that path, another seed or page; a module worker\'s is marked so; it keeps new, instanceof and its constructor', () => { + const at = (seed, page, script = '', path = '/p') => { + const w = workerGlobals(); + const d = documentAt(initScript(seed), path, new Map(), page, { globals: w.globals }); + d.run(script || 'globalThis.a = new Worker("/w.js?q=1#f"); globalThis.b = new Worker("w.js"); globalThis.c = new Worker("/x/m.js", { type: "module" });'); + return { d, made: w.made.map(([u, o]) => [workerMark(u), o === undefined ? undefined : JSON.parse(JSON.stringify(o))]) }; + }; + const { d, made } = at(S1, 't0'); + assert.deepEqual(made.map(([m]) => [m.kind, m.url, m.n]), [['w', 'http://h/w.js?q=1#f', -1], ['w', 'http://h/w.js', -1], ['m', 'http://h/x/m.js', -1]]); + assert.deepEqual(made[2][1], { type: 'module' }); + for (const [m] of made) assert.match(m.token, TOKEN); + assert.equal(new Set(made.map(([m]) => m.token)).size, 3); + assert.deepEqual(at(S1, 't0').made, made); + assert.notEqual(at(S2, 't0').made[0][0].token, made[0][0].token); + assert.notEqual(at(S1, 't1').made[0][0].token, made[0][0].token); + assert.equal(d.run('a instanceof Worker && Worker.prototype.constructor === Worker && Worker.name === "Worker" && Worker.length === 1'), true); + assert.throws(() => d.run('Worker("/w.js")'), TypeError); +}); + +test('a worker keploy cannot mark starts as the app asked and is named once with why; one of a document with no place starts as asked, the document named', () => { + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: w.globals }); + d.run('new Worker("blob:http://h/0f0e"); new Worker("data:text/javascript,1"); new Worker("blob:http://h/0f0e");'); + assert.deepEqual(w.made.map(([u]) => u), ['blob:http://h/0f0e', 'data:text/javascript,1', 'blob:http://h/0f0e']); + assert.deepEqual(told, [['blob:', 'worker'], ['data:', 'worker'], ['blob:', 'worker']]); + // The browser refuses the marked URL (a Trusted Types policy): as asked. + const told2 = []; + const made = []; + class Picky { constructor(url) { if (workerMark(String(url)) !== null) throw new TypeError('refused'); made.push(String(url)); } } + documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told: told2, globals: { URL, Worker: Picky } }).run('new Worker("/w.js")'); + assert.deepEqual([made, told2], [["/w.js"], [['http://h/w.js', 'worker']]]); + // No page names the document: its worker is not marked, the document is named. + const told3 = []; + const w3 = workerGlobals(); + documentAt(initScript(S1, 'tell'), '/p', new Map(), null, { report: 'tell', told: told3, globals: w3.globals }).run('new Worker("/w.js")'); + assert.deepEqual([w3.made.map(([u]) => u), told3], [['/w.js'], ['http://h/p']]); +}); + +test('a Shared Worker\'s token is the same in every document of the test, by script path and name, so its documents still share it; where the route cannot reach it (shared: false) it starts as asked and is named', () => { + const tokens = (seed, page, path, script, shared = true, told = []) => { + const w = workerGlobals(); + documentAt(initScript(seed, 'tell', '', shared), path, new Map(), page, { report: 'tell', told, globals: w.globals }).run(script); + return w.made.map(([u, o]) => [workerMark(u), o === undefined ? undefined : JSON.parse(JSON.stringify(o))]); + }; + const s = 'new SharedWorker("/s.js"); new SharedWorker("/s.js", "n"); new SharedWorker("/s.js", { name: "n", type: "module" }); new SharedWorker("/t.js");'; + const a = tokens(S1, 't0', '/p', s); + assert.deepEqual(a.map(([m]) => m.kind), ['w', 'w', 'm', 'w']); + assert.deepEqual(tokens(S1, 't1', '/q', s), a); + assert.equal(a[1][0].token, a[2][0].token); + assert.equal(new Set([a[0], a[1], a[3]].map(([m]) => m.token)).size, 3); + assert.notEqual(tokens(S2, 't0', '/p', s)[0][0].token, a[0][0].token); + const told = []; + const off = tokens(S1, 't0', '/p', 'new SharedWorker("/s.js")', false, told); + assert.deepEqual([off, told], [[[null, undefined]], [['http://h/s.js', 'shared']]]); +}); + +test('the lines for a worker keploy cannot reach say what it is, why, and the next step', () => { + assert.equal(unplacedLine('t', 'blob:', 'worker'), 'keploy: in t, the worker at blob: draws the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.'); + assert.equal(unplacedLine('t', 'http://h/s.js', 'shared'), 'keploy: in t, the Shared Worker at http://h/s.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + + 'record and replay the test in Chromium or WebKit to seed it.'); +}); + +test('a worker\'s seeding draws the same numbers for a token and instance every run, others for another; its location shows no mark, its own query and fragment kept', () => { + const t = 'cd'.repeat(16); + const a = workerAt(markWorker('http://h/w.js', 'w', t), workerScript(t, 0, 'w')).draws(); + assert.deepEqual(workerAt('http://h/w.js', workerScript(t, 0, 'w')).draws(), a); + assert.match(a.uuid, UUID_V4); + assert.notDeepEqual(workerAt('http://h/w.js', workerScript(t, 1, 'w')).draws().random, a.random); + assert.notDeepEqual(workerAt('http://h/w.js', workerScript('ef'.repeat(16), 0, 'w')).draws().random, a.random); + assert.notDeepEqual(workerAt('http://h/w.js', '').draws().random, a.random); + for (const [url, search] of [['http://h/w.js', ''], ['http://h/w.js?a=1', '?a=1'], ['http://h/w.js?', ''], ['http://h/w.js?a=1#f', '?a=1']]) { + for (const kind of ['w', 'm']) { + const w = workerAt(markWorker(url, kind, t), workerScript(t, 0, kind)); + assert.deepEqual([w.run('location.href'), w.run('location.search'), w.run('String(location)')], [url, search, url], `${kind} ${url}`); + } + } + // Another worker's mark is not this one's: shown as it is. + const other = markWorker('http://h/w.js', 'w', 'ef'.repeat(16)); + assert.equal(workerAt(other, workerScript(t, 0, 'w')).run('location.href'), other); +}); + +// A context whose route keeps its matcher and handler, and a route of it +// that records what the handler did. +function routed() { + const r = {}; + const context = { route: async (match, handler) => { r.match = match; r.handler = handler; } }; + r.ask = async (url, answer) => { + const did = []; + const route = { + request: () => ({ url: () => url }), + fetch: async (o) => { did.push(['fetch', o.url]); if (answer instanceof Error) throw answer; return answer; }, + fulfill: async (o) => { did.push(['fulfill', { ...o, body: String(o.body) }]); }, + continue: async (o) => { did.push(['continue', o.url]); }, + fallback: async () => { did.push(['fallback']); }, + }; + await r.handler(route); + return did; + }; + return { r, context }; +} +const answerOf = (status, body, headers = { 'content-type': 'text/javascript', 'content-encoding': 'gzip', 'content-length': '3', 'x-a': '1' }) => + ({ ok: () => status >= 200 && status < 300, status: () => status, headers: () => headers, body: async () => Buffer.from(body) }); + +test('workerRoutes answers a marked classic worker script with its seeding then importScripts of the app\'s own, fetched once and given unchanged; each start of a token is the next instance', async () => { + const { r, context } = routed(); + await workerRoutes(context); + const t = '01'.repeat(16); + assert.equal(r.match(new URL(markWorker('http://h/w.js', 'w', t))), true); + assert.equal(r.match(new URL('http://h/w.js?a=1')), false); + const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; + const own0 = markWorker('http://h/w.js?a=1', 'o', t, 0); + assert.deepEqual(await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')), [['fetch', 'http://h/w.js?a=1'], + ['fulfill', { status: 200, headers, body: `${workerScript(t, 0, 'w')}importScripts(${JSON.stringify(own0)});\n` }]]); + assert.deepEqual(await r.ask(own0), [['fulfill', { status: 200, headers, body: 'app()' }]]); + // Asked again (the browser's own retry): not kept, so as the app asked. + assert.deepEqual(await r.ask(own0), [['continue', 'http://h/w.js?a=1']]); + const second = await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')); + assert.equal(second[1][1].body, `${workerScript(t, 1, 'w')}importScripts(${JSON.stringify(markWorker('http://h/w.js?a=1', 'o', t, 1))});\n`); +}); + +test('workerRoutes answers a module worker script with an import of its seeding then one of the app\'s own; its seeding is workerScript; an answer not ok goes as given, one it cannot fetch goes on unmarked, a malformed mark falls back', async () => { + const { r, context } = routed(); + await workerRoutes(context); + const t = '23'.repeat(16); + const did = await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'export {}')); + assert.equal(did[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};\nimport ${JSON.stringify(markWorker('http://h/m.js', 'o', t, 0))};\n`); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0)), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 0, 'm') }]]); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), answerOf(404, 'no')), [['fetch', 'http://h/m.js'], + ['fulfill', { status: 404, headers: { 'content-type': 'text/javascript', 'x-a': '1' }, body: 'no' }]]); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), new Error('net')), [['fetch', 'http://h/m.js'], ['continue', 'http://h/m.js']]); + assert.deepEqual(await r.ask(`http://h/m.js?__keploy_worker=w.${t}.3`), [['fallback']]); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 7e56530..a829068 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -10,8 +10,8 @@ import http from 'node:http'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; -import { LOADS_KEY, OFF_LINE, SEEDED_LINE, initScript, processRandom, unplacedLine, unseededLine } from '../dist/random.js'; -import { BLANK_DRAWS, DRAWS, documentAt } from './draws.mjs'; +import { LOADS_KEY, OFF_LINE, SEEDED_LINE, initScript, processRandom, unplacedLine, unseededLine, workerMark, workerScript } from '../dist/random.js'; +import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; const PKG = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); const CLI = path.join(PKG, 'node_modules', '@playwright', 'test', 'cli.js'); @@ -609,7 +609,22 @@ async function drawsServer() { // parse.spec.ts's removed: /rhold.js is answered at /rrelease. if (u.pathname === '/rhold.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); rheld.push(res); return; } if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } + // workers.spec.ts's workers, and the scripts they load. + const wk = { + '/wk/ded.js': `importScripts('dep.js'); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep });`, + '/wk/dep.js': 'self.dep = \'dep\';', + '/wk/mod.js': `import { dep } from './lib.mjs'; postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep });`, + '/wk/lib.mjs': 'export const dep = \'lib\';', + '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href });`, + }[u.pathname]; + if (wk !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end(wk); return; } res.writeHead(200, { 'content-type': 'text/html' }); + if (u.pathname === '/wk') { + res.end(page('')); + return; + } if (u.pathname === '/draws') { res.end(page(``)); return; } if (u.pathname === '/frame') { res.end(page('frame')); return; } // /nest holds /frame of its own origin. @@ -1244,6 +1259,54 @@ for (const engine of ENGINES) { }); } +// The marks a document at /wk of page t0 gives the workers workers.spec.ts +// starts: its Web Workers', then its Shared Worker's. +function workerMarks(seed) { + const made = []; + class Worker { constructor(url) { made.push(workerMark(String(url))); } } + documentAt(initScript(seed), '/wk', new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }) + .run('new Worker(\'/wk/ded.js?x=1\'); new Worker(\'/wk/mod.js\', { type: \'module\' }); new SharedWorker(\'/wk/shared.js\', \'sn\');'); + return made; +} + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a page's classic and module Web Workers and its Shared Worker draw from the test's seed, the same at replay, at the app's location and loading what sits beside them; in Firefox the Shared Worker draws its own and is named`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); + const byKind = (r) => Object.fromEntries(r.drew.workers.map((w) => [w.kind, w])); + try { + const r = await drawRun(j, env(rec.url), ['workers.spec.ts'], ['workers']); + const w = byKind(r); + const [ded, mod, shared] = workerMarks(r.seeds.workers); + assert.deepEqual([ded.kind, mod.kind, shared.kind], ['w', 'm', 'w']); + assert.match(w.ded.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/ded\.js\?x=1$/); + assert.match(w.mod.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); + assert.match(w.shared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/shared\.js$/); + assert.deepEqual([w.ded.dep, w.mod.dep], ['dep', 'lib']); + assert.deepEqual(w.ded.draws, workerAt('http://h/wk/ded.js?x=1', workerScript(ded.token, 0, 'w')).draws()); + assert.deepEqual(w.mod.draws, workerAt('http://h/wk/mod.js', workerScript(mod.token, 0, 'm')).draws()); + const model = workerAt('http://h/wk/shared.js', workerScript(shared.token, 0, 'w')).draws(); + const said = r.tests.workers.stdout.split('\n').filter((l) => l.startsWith('keploy: in workers, ')).map((l) => l.replace(/:\d+\//, ':/')); + if (engine === 'firefox') { + assert.notDeepEqual(w.shared.draws, model); + assert.deepEqual(said, [unplacedLine('workers', 'http://127.0.0.1:/wk/shared.js', 'shared')]); + } else { + assert.deepEqual(w.shared.draws, model); + assert.deepEqual(said, []); + } + kept = r.seeds; + const p = byKind(await drawRun(j, env(rep.url), ['workers.spec.ts'], ['workers'])); + assert.deepEqual([p.ded.draws, p.mod.draws], [w.ded.draws, w.mod.draws]); + if (engine === 'firefox') assert.notDeepEqual(p.shared.draws, w.shared.draws); + else assert.deepEqual(p.shared.draws, w.shared.draws); + } finally { await rep.close(); await rec.close(); } + }); +} + test('seeded: keploy answers no seed (an older recording, or an older keploy): the browser draws its own, each test says so, and its end says own', async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); From a8145694e97223b9472cae4f92fbb81f0ac6545b Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 08:55:12 +0530 Subject: [PATCH 20/73] fix(playwright): answer a Shared Worker's script whole Chromium and WebKit let a context route reach none of what a Shared Worker loads after its script, so the importScripts or import of the app's own script that a Web Worker's answer makes reached the app's server with keploy's mark in its URL, and a module Shared Worker imported the app's script where its seeding should be. Chromium also reports a Shared Worker's script request but never its end (Playwright does not follow Shared Workers), so settle waited on it until the test timed out. A Shared Worker's script is now answered whole: a classic one with its seeding on the script's first line (after a hashbang, under the script's own 'use strict'), a module one with its seeding imported as a data: module ahead of its own imports. A module one whose policy refuses data: scripts gets the app's own script and the run says so. Each request the route fulfills counts as ended for settle. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/index.ts | 13 ++- playwright/src/random.ts | 166 ++++++++++++++++++++++++++------ playwright/src/settle.ts | 14 +++ playwright/test/random.test.mjs | 73 +++++++++++++- playwright/test/rows.test.mjs | 4 +- playwright/test/unit.test.mjs | 24 +++++ 7 files changed, 259 insertions(+), 37 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index b778f7a..8e5dfd0 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context takes it off before the request leaves the browser and answers with the worker's seeding, then the app's own script, unchanged, as its server answered it. In the worker, `location` shows no mark, and scripts and modules beside the worker load as before; a module worker's `import.meta.url` and a stack trace show it. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own that matches a worker's script URL whole needs a pattern that allows the mark (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context takes it off before the request leaves the browser and answers with the worker's seeding, then the app's own script, unchanged, as its server answered it. A Shared Worker's answer holds both: Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a classic one gets its seeding on its script's first line (no line of it moves) and a module one imports its seeding as a `data:` module ahead of its own imports; the test's end does not wait on a script keploy answered. In the worker, `location` shows no mark, and scripts and modules beside the worker load as before; a module worker's `import.meta.url` and a stack trace show it. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own that matches a worker's script URL whole needs a pattern that allows the mark (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Firefox a Shared Worker, whose script Firefox's route does not reach, and a module Shared Worker whose script is served with a Content-Security-Policy that refuses `data:` scripts. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 026d009..e8438b3 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -170,6 +170,7 @@ const keployTestType = base.extend({ // The lines that wait for a parent's parser to end, each said at the // test's end if it has not. const waiting = new Set<() => void>(); + const inflight = new Inflight(context); if (s.seed !== '') { // A document the seeding cannot place draws the browser's own numbers // and says so here, once for each address and reason in the test. @@ -215,7 +216,16 @@ const keployTestType = base.extend({ const mark = freshSeed(); // The workers' scripts: Firefox's route does not reach a Shared // Worker's, so there it is not marked, and says so. - await workerRoutes(context); + await workerRoutes(context, { + answered: (r) => inflight.answered(r), + unreached: (url, why) => { + const a = pageAddress(url); + const k = `${why}\n${a}`; + if (told.has(k)) return; + told.add(k); + process.stdout.write(`${unplacedLine(name, a, why)}\n`); + }, + }); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox') }); const placing = placePages(context, mark); restore = placing.restore; @@ -225,7 +235,6 @@ const keployTestType = base.extend({ const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; if (context.pages().length > 0) opened(); context.on('page', opened); - const inflight = new Inflight(context); try { await use(context); } finally { diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 2db90a4..0bd719f 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -17,7 +17,7 @@ // On by default; KEPLOY_SEED_RANDOM=off or the fixture option // keploySeedRandom: false turns it off, and the worker says what that costs. -import type { BrowserContext, Frame, Page } from '@playwright/test'; +import type { BrowserContext, Frame, Page, Request } from '@playwright/test'; import crypto from 'node:crypto'; import fs from 'node:fs'; import { createRequire } from 'node:module'; @@ -98,17 +98,19 @@ export function pageAddress(at: unknown): string { * storage to count its documents in (other than a page's first document), * or a frame of its origin inside one. */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'csp'; // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared', string> = { +const UNREACHED: Record<'worker' | 'shared' | 'csp', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + 'record and replay the test in Chromium or WebKit to seed it.', + csp: 'it is a module Shared Worker whose script is served with a Content-Security-Policy that refuses data: scripts, and keploy puts its seeding ahead of ' + + 'such a worker\'s modules only as a data: import; allow data: in that policy\'s script-src and worker-src for the test run to seed it.', }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -130,8 +132,8 @@ const UNPLACED: Record, str * (othersNow). */ export function unplacedLine(name: string, at: string, why: Unplaced = 'popup', others = 0): string { - if (why === 'worker' || why === 'shared') { - return `keploy: in ${name}, the ${why === 'shared' ? 'Shared Worker' : 'worker'} at ${at} draws the browser's own random numbers, so its draws may differ at replay: ${UNREACHED[why]}`; + if (why === 'worker' || why === 'shared' || why === 'csp') { + return `keploy: in ${name}, the ${why === 'worker' ? 'worker' : 'Shared Worker'} at ${at} draws the browser's own random numbers, so its draws may differ at replay: ${UNREACHED[why]}`; } const n = Math.floor(Number(others)); const reason = why === 'uncounted' @@ -647,7 +649,7 @@ const DRAW_JS = ` const define = (o, name, value) => { } }; const realRandom = Math.random; - // 53 bits, as the browser's own Math.random gives. + /* 53 bits, as the browser's own Math.random gives. */ define(Math, 'random', { random() { const rnd = from('random'); if (rnd === null) return realRandom(); @@ -657,8 +659,8 @@ const DRAW_JS = ` const define = (o, name, value) => { if (C && typeof C.getRandomValues === 'function') { const real = C.getRandomValues; define(C, 'getRandomValues', { getRandomValues(array) { - // The browser's own call first: it refuses what the browser refuses - // (a float array, more than 65536 bytes, a this that is no Crypto). + /* The browser's own call first: it refuses what the browser refuses + (a float array, more than 65536 bytes, a this that is no Crypto). */ const out = real.call(this, array); const grv = from('grv'); if (grv !== null) fill(grv, new Uint8Array(out.buffer, out.byteOffset, out.byteLength)); @@ -1204,7 +1206,7 @@ ${DRAW_JS} const h = u.href; const i = h.indexOf('#'); const b = i < 0 ? h : h.slice(0, i); - const m = b + (b.indexOf('?') < 0 ? '?' : '&') + ${JSON.stringify(WORKER_PARAM)} + '=' + (module ? 'm' : 'w') + '.' + why + (i < 0 ? '' : h.slice(i)); + const m = b + (b.indexOf('?') < 0 ? '?' : '&') + ${JSON.stringify(WORKER_PARAM)} + '=' + (name === 'SharedWorker' ? (module ? 'x' : 's') : module ? 'm' : 'w') + '.' + why + (i < 0 ? '' : h.slice(i)); try { return construct(Real, [m].concat(args.slice(1)), new.target); } catch (e) { // Refused with the mark (a Trusted Types policy): as the app asked. } @@ -1247,18 +1249,19 @@ ${DRAW_JS} /** * A worker's script URL, marked (initScript), read back: its kind ('w' a - * classic worker's script, 'm' a module worker's, 'o' the app's own script - * that the answer to either loads, 'p' a module worker's seeding), its - * token, the instance n of a token ('o' and 'p'; -1 for 'w' and 'm'), and + * classic Web Worker's script, 'm' a module Web Worker's, 's' a classic + * Shared Worker's, 'x' a module Shared Worker's, 'o' the app's own script + * that the answer to 'w' or 'm' loads, 'p' a module Web Worker's seeding), + * its token, the instance n of a token ('o' and 'p'; -1 for the others), and * the URL with no mark, as the app asked for it. null for a URL with no * mark keploy made. */ -export type WorkerMark = { kind: 'w' | 'm' | 'o' | 'p'; token: string; n: number; url: string }; +export type WorkerMark = { kind: 'w' | 'm' | 's' | 'x' | 'o' | 'p'; token: string; n: number; url: string }; export function workerMark(href: string): WorkerMark | null { const i = href.indexOf('#'); const b = i < 0 ? href : href.slice(0, i); - const m = /[?&]__keploy_worker=([wmop])\.([0-9a-f]{32})(?:\.(\d{1,9}))?$/.exec(b); + const m = /[?&]__keploy_worker=([wmsxop])\.([0-9a-f]{32})(?:\.(\d{1,9}))?$/.exec(b); if (m === null) return null; const kind = m[1] as WorkerMark['kind']; const numbered = kind === 'o' || kind === 'p'; @@ -1282,7 +1285,7 @@ export function markWorker(url: string, kind: WorkerMark['kind'], token: string, * the mark kind.token the worker's URL carries; a module worker's * import.meta.url, and a stack trace, show it. */ -export function workerScript(token: string, n: number, kind: 'w' | 'm'): string { +export function workerScript(token: string, n: number, kind: 'w' | 'm' | 's' | 'x'): string { return `(() => { const g = globalThis; const k = ${JSON.stringify(`${token}\n${n}\n`)}; @@ -1320,18 +1323,107 @@ ${DRAW_JS} `; } +// The seeding on one line: workerScript holds no line comment, and each of +// its statements ends with a semicolon (rows check both). +function oneLine(js: string): string { + return js.split('\n').map((l) => l.trim()).filter((l) => l !== '').join(' '); +} + +const SPACE = /^(?:\s|\/\/[^\n\r\u2028\u2029]*|\/\*[\s\S]*?\*\/)*/; +// What may follow a string literal on the next line and continue its +// expression, so that no semicolon is inserted after it. +const GOES_ON = /^(?:\+(?!\+)|-(?!-)|!=|[([.,?=*/%&|^<>`]|in(?![\w$])|instanceof(?![\w$]))/; + +/** + * Whether a classic script's directive prologue (the string literals that + * begin it, each a statement of its own) holds a Use Strict Directive: the + * literal 'use strict' or "use strict", written with no escape. + */ +export function strictPrologue(src: string): boolean { + let s = src; + for (;;) { + s = s.slice((SPACE.exec(s) ?? [''])[0].length); + const lit = /^(['"])((?:\\[\s\S]|(?!\1)[^\\\n\r\u2028\u2029])*)\1/.exec(s); + if (lit === null) return false; + s = s.slice(lit[0].length); + // The rest of its line: spaces and comments that end on it. + s = s.slice((/^(?:[ \t\v\f ]|\/\*[^\n\r\u2028\u2029]*?\*\/)*/.exec(s) ?? [''])[0].length); + if (s.startsWith(';')) s = s.slice(1); + else if (/^(?:[\n\r\u2028\u2029]|\/\/|\/\*)/.test(s)) { + // A line ends after it: a statement ends there unless the next line + // goes on with its expression. + if (GOES_ON.test(s.slice((SPACE.exec(s) ?? [''])[0].length))) return false; + } else if (s !== '' && !s.startsWith('}')) return false; + if (lit[2] === 'use strict') return true; + } +} + +/** + * Whether a module Shared Worker answered with these headers may import a + * data: module: each Content-Security-Policy that names a source list for + * its scripts (script-src-elem, else script-src, else default-src) or for + * its worker's loads (worker-src, else child-src, else script-src, else + * default-src) lists data: in it. + */ +export function cspAllowsData(headers: Record): boolean { + const v = headers['content-security-policy']; + if (v === undefined) return true; + for (const policy of v.split(/[,\n]/)) { + const ds = new Map(); + for (const d of policy.split(';')) { + const t = d.trim().split(/\s+/).filter((x) => x !== ''); + if (t.length > 0 && !ds.has(t[0].toLowerCase())) ds.set(t[0].toLowerCase(), t.slice(1).map((x) => x.toLowerCase())); + } + const first = (...names: string[]): string[] | undefined => names.map((x) => ds.get(x)).find((x) => x !== undefined); + for (const list of [first('script-src-elem', 'script-src', 'default-src'), first('worker-src', 'child-src', 'script-src', 'default-src')]) { + if (list !== undefined && !list.includes('data:')) return false; + } + } + return true; +} + +/** + * A Shared Worker's whole answer: its seeding, then the app's own script. + * Neither Chromium nor WebKit lets the route reach what a Shared Worker + * loads after its script, so this answer cannot load the app's script from + * a URL, as a Web Worker's does. A classic one gets the seeding on the + * app's first line, so no line of the app's moves: after a hashbang, and + * under 'use strict' when the app's directive prologue has it. A module + * one imports its seeding as a data: module first, which runs before every + * module the app's script imports. + */ +export function sharedScript(token: string, n: number, kind: 's' | 'x', own: string): string { + const bang = /^#![^\n\r\u2028\u2029]*(?:\r\n|[\n\r\u2028\u2029])?/.exec(own); + const head = bang === null ? '' : /[\n\r\u2028\u2029]$/.test(bang[0]) ? bang[0] : `${bang[0]}\n`; + const rest = own.slice(bang === null ? 0 : bang[0].length); + const seeding = workerScript(token, n, kind); + if (kind === 'x') return `${head}import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(seeding).toString('base64')}`)};${rest}`; + return `${head}${strictPrologue(rest) ? '\'use strict\';' : ''}${oneLine(seeding)}${rest}`; +} + +/** What the fixture hears from workerRoutes. */ +export type WorkerHooks = { + /** The route answered this request whole: none of it is in flight. */ + answered?: (r: Request) => void; + /** A worker the route cannot seed, by its script's URL as the app asked. */ + unreached?: (url: string, why: 'csp') => void; +}; + /** * Answers each marked worker script the test's context asks for: a - * classic worker's with its seeding, then importScripts of the app's own - * script; a module worker's with an import of its seeding, then one of - * the app's own; the app's own script with what its server answered to - * the worker's request (fetched once, unchanged); a module worker's - * seeding with workerScript. An answer that is not ok goes to the worker - * as the server gave it, and a request keploy cannot fetch goes on with - * no mark. Each instance of a token (a Shared Worker started again) is the - * next n. + * classic Web Worker's with its seeding, then importScripts of the app's + * own script; a module Web Worker's with an import of its seeding, then + * one of the app's own; the app's own script with what its server answered + * to the worker's request (fetched once, unchanged); a module worker's + * seeding with workerScript; a Shared Worker's with sharedScript (a module + * one whose policy refuses data: scripts with the app's own, unseeded, and + * unreached says so). An answer that is not ok goes to the worker as the + * server gave it, and a request keploy cannot fetch goes on with no mark. + * Each instance of a token (a Shared Worker started again) is the next n. + * Each request it fulfills is answered: Chromium reports no end for a + * Shared Worker's script, as Playwright does not follow Shared Workers. */ -export async function workerRoutes(context: BrowserContext): Promise { +export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}): Promise { const counts = new Map(); const kept = new Map; body: Buffer }>(); // The answer's headers, but its encoding and length: the body given is @@ -1342,20 +1434,24 @@ export async function workerRoutes(context: BrowserContext): Promise { return out; }; await context.route((u) => u.searchParams.has(WORKER_PARAM), async (route) => { + const fulfill = async (o: Parameters[0]): Promise => { + await route.fulfill(o); + hooks.answered?.(route.request()); + }; const m = workerMark(route.request().url()); if (m === null) { await route.fallback(); return; } if (m.kind === 'p') { - await route.fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); + await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); return; } if (m.kind === 'o') { const k = kept.get(`${m.token}.${m.n}`); kept.delete(`${m.token}.${m.n}`); if (k === undefined) await route.continue({ url: m.url }); - else await route.fulfill(k); + else await fulfill(k); return; } const n = counts.get(m.token) ?? 0; @@ -1369,7 +1465,19 @@ export async function workerRoutes(context: BrowserContext): Promise { } const headers = headersOf(answer.headers()); if (!answer.ok()) { - await route.fulfill({ status: answer.status(), headers, body: await answer.body() }); + await fulfill({ status: answer.status(), headers, body: await answer.body() }); + return; + } + if (m.kind === 's' || m.kind === 'x') { + const body = await answer.body(); + if (m.kind === 'x' && !cspAllowsData(headers)) { + hooks.unreached?.(m.url, 'csp'); + await fulfill({ status: answer.status(), headers, body }); + return; + } + // Scripts of workers are UTF-8; the browser drops a byte order mark. + const own = body.toString('utf8').replace(/^\ufeff/, ''); + await fulfill({ status: answer.status(), headers, body: sharedScript(m.token, n, m.kind, own) }); return; } kept.set(`${m.token}.${n}`, { status: answer.status(), headers, body: await answer.body() }); @@ -1377,6 +1485,6 @@ export async function workerRoutes(context: BrowserContext): Promise { const body = m.kind === 'w' ? `${workerScript(m.token, n, 'w')}importScripts(${own});\n` : `import ${JSON.stringify(markWorker(m.url, 'p', m.token, n))};\nimport ${own};\n`; - await route.fulfill({ status: answer.status(), headers, body }); + await fulfill({ status: answer.status(), headers, body }); }); } diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index bfb4626..28a3f18 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -11,9 +11,12 @@ export class Inflight { private readonly open = new Map(); private reported = 0; private waiters: (() => void)[] = []; + // The calls keploy's own route answered whole (answered). + private readonly ended = new WeakSet(); constructor(context: BrowserContext) { context.on('request', (r) => { + if (this.ended.has(r)) return; let frame: Frame | null = null; // A service worker's call has no frame. try { frame = r.frame(); } catch { frame = null; } @@ -92,6 +95,17 @@ export class Inflight { return new Promise((resolve) => this.waiters.push(resolve)); } + /** + * A call keploy's own route answered whole (a worker's script): nothing of + * it is in flight, whatever the browser reports after. Chromium reports + * no end for a Shared Worker's script: Playwright does not follow Shared + * Workers, and the end goes to the worker. + */ + answered(r: Request): void { + this.ended.add(r); + this.done(r); + } + private done(r: Request): void { if (this.open.delete(r)) this.wake(); } diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index d808cec..843eea2 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, markWorker, workerMark, workerScript, workerRoutes } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, markWorker, workerMark, workerScript, workerRoutes, sharedScript, strictPrologue, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -1351,7 +1351,7 @@ test('a worker\'s mark: what markWorker makes, workerMark reads back, the URL as } } assert.equal(markWorker('http://h/w.js?a=1#f', 'w', t), `http://h/w.js?a=1&__keploy_worker=w.${t}#f`); - for (const u of ['http://h/w.js', `http://h/w.js?__keploy_worker=w.${t}&a=1`, `http://h/w.js?__keploy_worker=x.${t}`, `http://h/w.js?__keploy_worker=w.${t.toUpperCase()}`, + for (const u of ['http://h/w.js', `http://h/w.js?__keploy_worker=w.${t}&a=1`, `http://h/w.js?__keploy_worker=z.${t}`, `http://h/w.js?__keploy_worker=w.${t.toUpperCase()}`, `http://h/w.js?__keploy_worker=w.${t}.0`, `http://h/w.js?__keploy_worker=o.${t}`, `http://h/w.js?__keploy_worker=w.${t.slice(1)}`]) { assert.equal(workerMark(u), null, u); } @@ -1404,7 +1404,7 @@ test('a Shared Worker\'s token is the same in every document of the test, by scr }; const s = 'new SharedWorker("/s.js"); new SharedWorker("/s.js", "n"); new SharedWorker("/s.js", { name: "n", type: "module" }); new SharedWorker("/t.js");'; const a = tokens(S1, 't0', '/p', s); - assert.deepEqual(a.map(([m]) => m.kind), ['w', 'w', 'm', 'w']); + assert.deepEqual(a.map(([m]) => m.kind), ['s', 's', 'x', 's']); assert.deepEqual(tokens(S1, 't1', '/q', s), a); assert.equal(a[1][0].token, a[2][0].token); assert.equal(new Set([a[0], a[1], a[3]].map(([m]) => m.token)).size, 3); @@ -1493,3 +1493,70 @@ test('workerRoutes answers a module worker script with an import of its seeding assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), new Error('net')), [['fetch', 'http://h/m.js'], ['continue', 'http://h/m.js']]); assert.deepEqual(await r.ask(`http://h/m.js?__keploy_worker=w.${t}.3`), [['fallback']]); }); + +test('strictPrologue: a Use Strict Directive anywhere in the directive prologue, after spaces and comments; not one that an expression goes on from, one with an escape, or one after a statement', () => { + const yes = ["'use strict';x()", '"use strict"\nx()', "// c\n/* d */ 'use strict'", "'use asm'; 'use strict'; f()", "'use strict'\n!x", "'use strict'\n++x", "'use strict' /* c */\n'next'", "'use strict'", "'a'\n'use strict'\n", "'use strict'}"]; + const no = ["'use strict'\n(x)", "'use strict'.x", "'use\\x20strict';", "x; 'use strict'", "'use strict'\n+x", "'use strict'\nin x", "'use strict' + 1", "'use strict'\n[1]", '', "'unterminated", "'a'\n.x\n'use strict'", "'use strict' // c\n, 1", "'a' x 'use strict'"]; + for (const src of yes) assert.equal(strictPrologue(src), true, JSON.stringify(src)); + for (const src of no) assert.equal(strictPrologue(src), false, JSON.stringify(src)); +}); + +test('cspAllowsData: no policy, or each policy whose script or worker source list is named lists data:; a policy that names one without it refuses', () => { + const csp = (v) => ({ 'content-security-policy': v }); + assert.equal(cspAllowsData({}), true); + for (const v of ["img-src 'self'", "script-src 'self' data:", "default-src 'self' DATA:", "script-src data:; worker-src data:", "script-src-elem data:; worker-src 'self' data:"]) assert.equal(cspAllowsData(csp(v)), true, v); + for (const v of ["script-src 'self'", "default-src 'self'", "script-src data:; worker-src 'self'", "script-src-elem 'self'; script-src data:", "child-src 'self'; script-src data:", + "script-src data:, default-src 'self'", "script-src data:\nscript-src 'none'", "script-src 'none'; script-src data:"]) assert.equal(cspAllowsData(csp(v)), false, v); +}); + +test('sharedScript: a classic Shared Worker\'s answer is its seeding on the app\'s first line (no line moves), after a hashbang, under the app\'s use strict; a module one\'s imports its seeding as a data: module first', () => { + const t = '45'.repeat(16); + const seeding = workerScript(t, 2, 's'); + assert.doesNotMatch(seeding, /(^|[^:'"\\])\/\//, 'the seeding holds no line comment'); + const line = sharedScript(t, 2, 's', '').split('\n'); + assert.equal(line.length, 1); + assert.equal(sharedScript(t, 2, 's', 'a()\nb()'), `${line[0]}a()\nb()`); + assert.equal(sharedScript(t, 2, 's', '"use strict";\na()'), `'use strict';${line[0]}"use strict";\na()`); + assert.equal(sharedScript(t, 2, 's', '#!/x\r\n\'use strict\'\na()'), `#!/x\r\n'use strict';${line[0]}'use strict'\na()`); + assert.equal(sharedScript(t, 2, 's', '#!/x'), `#!/x\n${line[0]}`); + // The one line draws as the seeding does, and its location shows no mark. + const href = markWorker('http://h/s.js?q=1', 's', t); + const w = workerAt(href, sharedScript(t, 2, 's', 'globalThis.app = location.href;')); + assert.deepEqual(w.draws(), workerAt(href, seeding).draws()); + assert.equal(w.run('app'), 'http://h/s.js?q=1'); + const m = sharedScript(t, 2, 'x', '#!/x\nimport "./a.js";'); + const data = /^#!\/x\nimport "data:text\/javascript;base64,([A-Za-z0-9+/=]+)";import "\.\/a\.js";$/.exec(m); + assert.ok(data, m.slice(0, 120)); + assert.equal(Buffer.from(data[1], 'base64').toString(), workerScript(t, 2, 'x')); + assert.equal(sharedScript(t, 2, 'x', 'x()'), `import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(workerScript(t, 2, 'x')).toString('base64')}`)};x()`); +}); + +test('workerRoutes answers a Shared Worker\'s script whole with sharedScript (the app\'s byte order mark dropped), a module one whose policy refuses data: as the app\'s own and says so; each request it fulfills is answered, none it lets go on', async () => { + const { r, context } = routed(); + const answered = []; + const unreached = []; + await workerRoutes(context, { answered: (q) => answered.push(q.url()), unreached: (u, why) => unreached.push([u, why]) }); + const t = '67'.repeat(16); + const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; + const s0 = markWorker('http://h/s.js', 's', t); + assert.deepEqual(await r.ask(s0, answerOf(200, '\ufeffapp()')), [['fetch', 'http://h/s.js'], ['fulfill', { status: 200, headers, body: sharedScript(t, 0, 's', 'app()') }]]); + assert.equal((await r.ask(s0, answerOf(200, 'app()')))[1][1].body, sharedScript(t, 1, 's', 'app()')); + const x0 = markWorker('http://h/m.js', 'x', t.replace('6', '8')); + assert.equal((await r.ask(x0, answerOf(200, 'export {}')))[1][1].body, sharedScript(t.replace('6', '8'), 0, 'x', 'export {}')); + const policy = { 'content-type': 'text/javascript', 'content-security-policy': "script-src 'self'" }; + const x1 = markWorker('http://h/c.js', 'x', t.replace('6', '9')); + assert.deepEqual(await r.ask(x1, answerOf(200, '\ufeffexport {}', policy)), [['fetch', 'http://h/c.js'], ['fulfill', { status: 200, headers: policy, body: '\ufeffexport {}' }]]); + assert.deepEqual(unreached, [['http://h/c.js', 'csp']]); + // A classic one is not refused by a policy: its seeding is in its own script. + const d0 = markWorker('http://h/d.js', 's', t.replace('6', 'a')); + assert.equal((await r.ask(d0, answerOf(200, 'app()', policy)))[1][1].body, sharedScript(t.replace('6', 'a'), 0, 's', 'app()')); + assert.deepEqual(answered, [s0, s0, x0, x1, d0]); + // Not answered: what goes on, and what falls back. + await r.ask(markWorker('http://h/e.js', 's', t), new Error('net')); + await r.ask(markWorker('http://h/w.js', 'o', t, 5)); + await r.ask(`http://h/m.js?__keploy_worker=s.${t}.3`); + assert.equal(answered.length, 5); + assert.equal(unplacedLine('t', 'http://h/c.js', 'csp'), 'keploy: in t, the Shared Worker at http://h/c.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + 'it is a module Shared Worker whose script is served with a Content-Security-Policy that refuses data: scripts, and keploy puts its seeding ahead of ' + + 'such a worker\'s modules only as a data: import; allow data: in that policy\'s script-src and worker-src for the test run to seed it.'); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index a829068..a17bb01 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1282,14 +1282,14 @@ for (const engine of ENGINES) { const r = await drawRun(j, env(rec.url), ['workers.spec.ts'], ['workers']); const w = byKind(r); const [ded, mod, shared] = workerMarks(r.seeds.workers); - assert.deepEqual([ded.kind, mod.kind, shared.kind], ['w', 'm', 'w']); + assert.deepEqual([ded.kind, mod.kind, shared.kind], ['w', 'm', 's']); assert.match(w.ded.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/ded\.js\?x=1$/); assert.match(w.mod.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); assert.match(w.shared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/shared\.js$/); assert.deepEqual([w.ded.dep, w.mod.dep], ['dep', 'lib']); assert.deepEqual(w.ded.draws, workerAt('http://h/wk/ded.js?x=1', workerScript(ded.token, 0, 'w')).draws()); assert.deepEqual(w.mod.draws, workerAt('http://h/wk/mod.js', workerScript(mod.token, 0, 'm')).draws()); - const model = workerAt('http://h/wk/shared.js', workerScript(shared.token, 0, 'w')).draws(); + const model = workerAt('http://h/wk/shared.js', workerScript(shared.token, 0, 's')).draws(); const said = r.tests.workers.stdout.split('\n').filter((l) => l.startsWith('keploy: in workers, ')).map((l) => l.replace(/:\d+\//, ':/')); if (engine === 'firefox') { assert.notDeepEqual(w.shared.draws, model); diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 4c04835..f7fcee9 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -275,6 +275,30 @@ async function ticks() { for (let i = 0; i < 5; i++) await Promise.resolve(); } +test('settle: a call keploy\'s route answered has ended, though the browser reports no end (Chromium, a Shared Worker\'s script); one reported after its answer is not waited for', async () => { + const ctx = fakeContext(); + const inflight = new Inflight(ctx); + const page = new EventEmitter(); + ctx.emit('page', page); + const script = fakeRequest(page); + const other = fakeRequest(page); + ctx.emit('request', script); + ctx.emit('request', other); + let settled = false; + inflight.settled().then(() => { settled = true; }); + inflight.answered(script); + await ticks(); + assert.equal(inflight.size, 1); + assert.equal(settled, false); + ctx.emit('requestfinished', other); + await ticks(); + assert.equal(settled, true); + const late = fakeRequest(page); + inflight.answered(late); + ctx.emit('request', late); + assert.equal(inflight.size, 0); +}); + test('settle: a cancel that comes while settle already waits wakes it (its page closes; then, for a service worker\'s call, the test closes its context)', async () => { for (const worker of [false, true]) { const c = worker ? 'a page\'s call and a service worker\'s' : 'a page\'s call'; From 66131fb78dd9317de4a741ee77bac9c978fda9fc Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 09:01:32 +0530 Subject: [PATCH 21/73] test(playwright): rows for module Shared Workers and the unseen mark The browser row for a page's workers now also starts module Shared Workers in every engine: one plain, which draws from the test's seed in Chromium and WebKit (its own in Firefox, named), and one served with a policy that refuses data: scripts, started twice by two names, which draws its own and is named once. The row also checks that the app's server is never asked for a URL carrying keploy's mark, at record and at replay, and that it was asked for each worker's script and what it loads. cspAllowsData's row adds directive names in upper and mixed case, and strictPrologue's adds the next line's -, !=, instanceof (the expression goes on), --x and an identifier that starts with "in" (it does not), and a line or block comment that ends the directive's line. Signed-off-by: slayerjain --- playwright/test/fixtures/workers.spec.ts | 9 +-- playwright/test/random.test.mjs | 9 ++- playwright/test/rows.test.mjs | 72 ++++++++++++++++++------ 3 files changed, 66 insertions(+), 24 deletions(-) diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index 35b6000..7ad058a 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -3,14 +3,15 @@ import { test } from '../../dist'; // Run in Chromium, Firefox and WebKit (PW_BROWSER). The page at /wk starts // a classic Web Worker (which loads a script beside it with importScripts), -// a module Web Worker (which imports a module beside it) and a Shared -// Worker; each sends what it drew, its location and what it loaded. The -// test keeps them. +// a module Web Worker (which imports a module beside it), a classic Shared +// Worker and three module Shared Workers (each importing a module beside +// it); each sends what it drew, its location and what it loaded. The test +// keeps them. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); test('workers', async ({ page }) => { await page.goto(`${base}/wk`); - await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 3, null, { timeout: 20000 }); + await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 6, null, { timeout: 20000 }); keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 843eea2..9987ee6 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1495,8 +1495,10 @@ test('workerRoutes answers a module worker script with an import of its seeding }); test('strictPrologue: a Use Strict Directive anywhere in the directive prologue, after spaces and comments; not one that an expression goes on from, one with an escape, or one after a statement', () => { - const yes = ["'use strict';x()", '"use strict"\nx()', "// c\n/* d */ 'use strict'", "'use asm'; 'use strict'; f()", "'use strict'\n!x", "'use strict'\n++x", "'use strict' /* c */\n'next'", "'use strict'", "'a'\n'use strict'\n", "'use strict'}"]; - const no = ["'use strict'\n(x)", "'use strict'.x", "'use\\x20strict';", "x; 'use strict'", "'use strict'\n+x", "'use strict'\nin x", "'use strict' + 1", "'use strict'\n[1]", '', "'unterminated", "'a'\n.x\n'use strict'", "'use strict' // c\n, 1", "'a' x 'use strict'"]; + const yes = ["'use strict';x()", '"use strict"\nx()', "// c\n/* d */ 'use strict'", "'use asm'; 'use strict'; f()", "'use strict'\n!x", "'use strict'\n++x", "'use strict' /* c */\n'next'", "'use strict'", "'a'\n'use strict'\n", "'use strict'}", + "'use strict'\n--x", "'use strict'\ninit()", "'use strict' // c\nx()", "'use strict' /* a\nb */ x()"]; + const no = ["'use strict'\n(x)", "'use strict'.x", "'use\\x20strict';", "x; 'use strict'", "'use strict'\n+x", "'use strict'\nin x", "'use strict' + 1", "'use strict'\n[1]", '', "'unterminated", "'a'\n.x\n'use strict'", "'use strict' // c\n, 1", "'a' x 'use strict'", + "'use strict'\n- x", "'use strict'\n!= 1", "'use strict'\ninstanceof X"]; for (const src of yes) assert.equal(strictPrologue(src), true, JSON.stringify(src)); for (const src of no) assert.equal(strictPrologue(src), false, JSON.stringify(src)); }); @@ -1506,7 +1508,8 @@ test('cspAllowsData: no policy, or each policy whose script or worker source lis assert.equal(cspAllowsData({}), true); for (const v of ["img-src 'self'", "script-src 'self' data:", "default-src 'self' DATA:", "script-src data:; worker-src data:", "script-src-elem data:; worker-src 'self' data:"]) assert.equal(cspAllowsData(csp(v)), true, v); for (const v of ["script-src 'self'", "default-src 'self'", "script-src data:; worker-src 'self'", "script-src-elem 'self'; script-src data:", "child-src 'self'; script-src data:", - "script-src data:, default-src 'self'", "script-src data:\nscript-src 'none'", "script-src 'none'; script-src data:"]) assert.equal(cspAllowsData(csp(v)), false, v); + "script-src data:, default-src 'self'", "script-src data:\nscript-src 'none'", "script-src 'none'; script-src data:", "SCRIPT-SRC 'self'", + "script-src data:; Worker-Src 'self'"]) assert.equal(cspAllowsData(csp(v)), false, v); }); test('sharedScript: a classic Shared Worker\'s answer is its seeding on the app\'s first line (no line moves), after a hashbang, under the app\'s use strict; a module one\'s imports its seeding as a data: module first', () => { diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index a17bb01..d71a4bd 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -601,7 +601,10 @@ async function drawsServer() { }; })(); const rheld = []; + // Every URL the server was asked for, as the request named it. + const seen = []; const srv = http.createServer((req, res) => { + seen.push(req.url); const u = new URL(req.url, 'http://x'); // /csp is served with a CSP sandbox: an opaque origin, no session storage. if (u.pathname === '/csp') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('csp')); return; } @@ -616,13 +619,20 @@ async function drawsServer() { '/wk/mod.js': `import { dep } from './lib.mjs'; postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep });`, '/wk/lib.mjs': 'export const dep = \'lib\';', '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href });`, + // Module Shared Workers: one plain, one (started twice, by two names) + // served with a policy that refuses data: scripts. + '/wk/mshared.js': `import { dep } from './lib.mjs'; self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep });`, + '/wk/cshared.js': `import { dep } from './lib.mjs'; self.onconnect = (e) => e.ports[0].postMessage({ kind: self.name, draws: ${DRAWS}, href: location.href, dep });`, }[u.pathname]; - if (wk !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end(wk); return; } + if (wk !== undefined) { + res.writeHead(200, { 'content-type': 'text/javascript', ...(u.pathname === '/wk/cshared.js' ? { 'content-security-policy': 'script-src \'self\'' } : {}) }); + res.end(wk); + return; + } res.writeHead(200, { 'content-type': 'text/html' }); if (u.pathname === '/wk') { - res.end(page('')); + const start = (s) => (s.startsWith('new Worker(') ? `${s}.onmessage = take;` : `{ const s = ${s}; s.port.onmessage = take; s.port.start(); }`); + res.end(page(``)); return; } if (u.pathname === '/draws') { res.end(page(``)); return; } @@ -740,7 +750,7 @@ async function drawsServer() { }); // Every address: the frame is asked for at localhost, which may be ::1. await new Promise((r) => srv.listen(0, r)); - return { url: `http://127.0.0.1:${srv.address().port}`, close: () => new Promise((r) => { srv.closeAllConnections(); srv.close(r); }) }; + return { url: `http://127.0.0.1:${srv.address().port}`, seen, close: () => new Promise((r) => { srv.closeAllConnections(); srv.close(r); }) }; } // One run of seeded.spec.ts (or another spec of the /draws page): what each @@ -760,7 +770,7 @@ async function drawRun(j, env, specs = ['seeded.spec.ts'], names = ['one', 'two' const ends = sent.filter((c) => c.path === '/agent/scope/end').map((c) => [c.body.name, c.body.random]); // The seed each test's begin sent. const seeds = Object.fromEntries(begins.map((b) => [b.name.replace(/^own › /, ''), b.seed])); - return { drew, said, lines, tests, tmp, begins, ends, seeds }; + return { drew, said, lines, tests, tmp, begins, ends, seeds, seen: d.seen }; } finally { await d.close(); } } @@ -1259,18 +1269,29 @@ for (const engine of ENGINES) { }); } +// The workers the page at /wk starts, in order: a classic and a module Web +// Worker, a classic Shared Worker, and module Shared Workers. +const WK_STARTS = [ + 'new Worker(\'/wk/ded.js?x=1\')', + 'new Worker(\'/wk/mod.js\', { type: \'module\' })', + 'new SharedWorker(\'/wk/shared.js\', \'sn\')', + 'new SharedWorker(\'/wk/mshared.js\', { type: \'module\', name: \'mn\' })', + 'new SharedWorker(\'/wk/cshared.js\', { type: \'module\', name: \'c1\' })', + 'new SharedWorker(\'/wk/cshared.js\', { type: \'module\', name: \'c2\' })', +]; + // The marks a document at /wk of page t0 gives the workers workers.spec.ts -// starts: its Web Workers', then its Shared Worker's. +// starts (WK_STARTS). function workerMarks(seed) { const made = []; class Worker { constructor(url) { made.push(workerMark(String(url))); } } documentAt(initScript(seed), '/wk', new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }) - .run('new Worker(\'/wk/ded.js?x=1\'); new Worker(\'/wk/mod.js\', { type: \'module\' }); new SharedWorker(\'/wk/shared.js\', \'sn\');'); + .run(WK_STARTS.map((s) => `${s};`).join(' ')); return made; } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a page's classic and module Web Workers and its Shared Worker draw from the test's seed, the same at replay, at the app's location and loading what sits beside them; in Firefox the Shared Worker draws its own and is named`, async () => { + test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, at the app's location and loading what sits beside them, and the app's server never sees keploy's mark; a module Shared Worker whose policy refuses data: draws its own and is named once; in Firefox each Shared Worker draws its own and is named`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1281,28 +1302,45 @@ for (const engine of ENGINES) { try { const r = await drawRun(j, env(rec.url), ['workers.spec.ts'], ['workers']); const w = byKind(r); - const [ded, mod, shared] = workerMarks(r.seeds.workers); - assert.deepEqual([ded.kind, mod.kind, shared.kind], ['w', 'm', 's']); + const [ded, mod, shared, mshared, c1, c2] = workerMarks(r.seeds.workers); + assert.deepEqual([ded, mod, shared, mshared, c1, c2].map((m) => m.kind), ['w', 'm', 's', 'x', 'x', 'x']); + assert.notEqual(c1.token, c2.token); assert.match(w.ded.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/ded\.js\?x=1$/); assert.match(w.mod.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); assert.match(w.shared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/shared\.js$/); - assert.deepEqual([w.ded.dep, w.mod.dep], ['dep', 'lib']); + assert.match(w.mshared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); + for (const c of [w.c1, w.c2]) assert.match(c.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); + assert.deepEqual([w.ded.dep, w.mod.dep, w.mshared.dep, w.c1.dep, w.c2.dep], ['dep', 'lib', 'lib', 'lib', 'lib']); assert.deepEqual(w.ded.draws, workerAt('http://h/wk/ded.js?x=1', workerScript(ded.token, 0, 'w')).draws()); assert.deepEqual(w.mod.draws, workerAt('http://h/wk/mod.js', workerScript(mod.token, 0, 'm')).draws()); const model = workerAt('http://h/wk/shared.js', workerScript(shared.token, 0, 's')).draws(); + const mmodel = workerAt('http://h/wk/mshared.js', workerScript(mshared.token, 0, 'x')).draws(); + for (const [c, m] of [[w.c1, c1], [w.c2, c2]]) assert.notDeepEqual(c.draws, workerAt('http://h/wk/cshared.js', workerScript(m.token, 0, 'x')).draws()); const said = r.tests.workers.stdout.split('\n').filter((l) => l.startsWith('keploy: in workers, ')).map((l) => l.replace(/:\d+\//, ':/')); + const at = (p) => `http://127.0.0.1:/wk/${p}`; if (engine === 'firefox') { assert.notDeepEqual(w.shared.draws, model); - assert.deepEqual(said, [unplacedLine('workers', 'http://127.0.0.1:/wk/shared.js', 'shared')]); + assert.notDeepEqual(w.mshared.draws, mmodel); + assert.deepEqual(said.sort(), ['shared.js', 'mshared.js', 'cshared.js'].map((p) => unplacedLine('workers', at(p), 'shared')).sort()); } else { assert.deepEqual(w.shared.draws, model); - assert.deepEqual(said, []); + assert.deepEqual(w.mshared.draws, mmodel); + assert.deepEqual(said, [unplacedLine('workers', at('cshared.js'), 'csp')]); } + // What the app's server was asked for: never keploy's mark, and each + // worker's script and what it loads. + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); + for (const u of ['/wk/ded.js?x=1', '/wk/dep.js', '/wk/mod.js', '/wk/lib.mjs', '/wk/shared.js', '/wk/mshared.js', '/wk/cshared.js']) assert.ok(r.seen.includes(u), `${u}: ${r.seen.join(' ')}`); kept = r.seeds; - const p = byKind(await drawRun(j, env(rep.url), ['workers.spec.ts'], ['workers'])); + const q = await drawRun(j, env(rep.url), ['workers.spec.ts'], ['workers']); + const p = byKind(q); assert.deepEqual([p.ded.draws, p.mod.draws], [w.ded.draws, w.mod.draws]); - if (engine === 'firefox') assert.notDeepEqual(p.shared.draws, w.shared.draws); - else assert.deepEqual(p.shared.draws, w.shared.draws); + for (const k of ['shared', 'mshared']) { + if (engine === 'firefox') assert.notDeepEqual(p[k].draws, w[k].draws, k); + else assert.deepEqual(p[k].draws, w[k].draws, k); + } + for (const k of ['c1', 'c2']) assert.notDeepEqual(p[k].draws, w[k].draws, k); + assert.deepEqual(q.seen.filter((u) => u.includes('__keploy_worker')), []); } finally { await rep.close(); await rec.close(); } }); } From 1f285ba03da5a5087fe93b20a5dc62968bd94132 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 09:34:34 +0530 Subject: [PATCH 22/73] fix(playwright): run a worker's own script at its URL with no mark keploy's route answers a marked worker script with the worker's seeding, which then loads the app's own script from the URL the app asked for. The app's script so runs at its own URL: its import.meta.url, the URLs its imports resolve against and a stack trace through it show no mark, as recorded and as replayed. Before, a module worker's import.meta.url and a stack trace carried the mark, and a module Shared Worker whose policy refuses data: scripts ran unseeded at the marked URL, its location showing the mark. - A classic worker (Web or Shared) runs its seeding, then importScripts of the app's script; a module Web Worker imports its seeding, then the app's script; a module Shared Worker imports its seeding as a data: module, then the app's script. - Where a module Shared Worker's policy refuses data: scripts, the answer runs the seeding itself, then imports the app's script, and gives the connect events that came before that script ran to its handlers after, in order. That worker is now seeded, and its line is gone. - keploy's answer keeps the status and headers the app's server gives the script, so the server is asked for it twice: by keploy, and by the worker. When keploy cannot fetch it, the worker gets its seeding and loads the app's script itself. A route answered with a redirect to the unmarked URL is not an option: Chromium never follows a fulfilled 307 for a worker's script, and Playwright refuses to fulfill with a redirect in WebKit. The browser row checks import.meta.url and stacks, a second connection to a held Shared Worker, and that module Shared Worker seeded at replay. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/index.ts | 11 +- playwright/src/random.ts | 186 +++++++++-------------- playwright/test/draws.mjs | 7 +- playwright/test/fixtures/workers.spec.ts | 4 +- playwright/test/random.test.mjs | 147 +++++++++++------- playwright/test/rows.test.mjs | 49 ++++-- 7 files changed, 203 insertions(+), 203 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 8e5dfd0..765b026 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context takes it off before the request leaves the browser and answers with the worker's seeding, then the app's own script, unchanged, as its server answered it. A Shared Worker's answer holds both: Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a classic one gets its seeding on its script's first line (no line of it moves) and a module one imports its seeding as a `data:` module ahead of its own imports; the test's end does not wait on a script keploy answered. In the worker, `location` shows no mark, and scripts and modules beside the worker load as before; a module worker's `import.meta.url` and a stack trace show it. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own that matches a worker's script URL whole needs a pattern that allows the mark (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Firefox a Shared Worker, whose script Firefox's route does not reach, and a module Shared Worker whose script is served with a Content-Security-Policy that refuses `data:` scripts. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server never sees the mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark, and the worker's `location` shows none. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index e8438b3..56a240c 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -216,16 +216,7 @@ const keployTestType = base.extend({ const mark = freshSeed(); // The workers' scripts: Firefox's route does not reach a Shared // Worker's, so there it is not marked, and says so. - await workerRoutes(context, { - answered: (r) => inflight.answered(r), - unreached: (url, why) => { - const a = pageAddress(url); - const k = `${why}\n${a}`; - if (told.has(k)) return; - told.add(k); - process.stdout.write(`${unplacedLine(name, a, why)}\n`); - }, - }); + await workerRoutes(context, { answered: (r) => inflight.answered(r) }); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox') }); const placing = placePages(context, mark); restore = placing.restore; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 0bd719f..c4f3789 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -98,19 +98,17 @@ export function pageAddress(at: unknown): string { * storage to count its documents in (other than a page's first document), * or a frame of its origin inside one. */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'csp'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared'; // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared' | 'csp', string> = { +const UNREACHED: Record<'worker' | 'shared', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + 'record and replay the test in Chromium or WebKit to seed it.', - csp: 'it is a module Shared Worker whose script is served with a Content-Security-Policy that refuses data: scripts, and keploy puts its seeding ahead of ' + - 'such a worker\'s modules only as a data: import; allow data: in that policy\'s script-src and worker-src for the test run to seed it.', }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -132,7 +130,7 @@ const UNPLACED: Record { @@ -1323,41 +1321,6 @@ ${DRAW_JS} `; } -// The seeding on one line: workerScript holds no line comment, and each of -// its statements ends with a semicolon (rows check both). -function oneLine(js: string): string { - return js.split('\n').map((l) => l.trim()).filter((l) => l !== '').join(' '); -} - -const SPACE = /^(?:\s|\/\/[^\n\r\u2028\u2029]*|\/\*[\s\S]*?\*\/)*/; -// What may follow a string literal on the next line and continue its -// expression, so that no semicolon is inserted after it. -const GOES_ON = /^(?:\+(?!\+)|-(?!-)|!=|[([.,?=*/%&|^<>`]|in(?![\w$])|instanceof(?![\w$]))/; - -/** - * Whether a classic script's directive prologue (the string literals that - * begin it, each a statement of its own) holds a Use Strict Directive: the - * literal 'use strict' or "use strict", written with no escape. - */ -export function strictPrologue(src: string): boolean { - let s = src; - for (;;) { - s = s.slice((SPACE.exec(s) ?? [''])[0].length); - const lit = /^(['"])((?:\\[\s\S]|(?!\1)[^\\\n\r\u2028\u2029])*)\1/.exec(s); - if (lit === null) return false; - s = s.slice(lit[0].length); - // The rest of its line: spaces and comments that end on it. - s = s.slice((/^(?:[ \t\v\f ]|\/\*[^\n\r\u2028\u2029]*?\*\/)*/.exec(s) ?? [''])[0].length); - if (s.startsWith(';')) s = s.slice(1); - else if (/^(?:[\n\r\u2028\u2029]|\/\/|\/\*)/.test(s)) { - // A line ends after it: a statement ends there unless the next line - // goes on with its expression. - if (GOES_ON.test(s.slice((SPACE.exec(s) ?? [''])[0].length))) return false; - } else if (s !== '' && !s.startsWith('}')) return false; - if (lit[2] === 'use strict') return true; - } -} - /** * Whether a module Shared Worker answered with these headers may import a * data: module: each Content-Security-Policy that names a source list for @@ -1383,51 +1346,65 @@ export function cspAllowsData(headers: Record): boolean { } /** - * A Shared Worker's whole answer: its seeding, then the app's own script. - * Neither Chromium nor WebKit lets the route reach what a Shared Worker - * loads after its script, so this answer cannot load the app's script from - * a URL, as a Web Worker's does. A classic one gets the seeding on the - * app's first line, so no line of the app's moves: after a hashbang, and - * under 'use strict' when the app's directive prologue has it. A module - * one imports its seeding as a data: module first, which runs before every - * module the app's script imports. + * keploy's answer to a marked worker script ('w', 'm', 's' or 'x', for the + * instance n of token): the worker's seeding, then the app's own script, + * loaded from own, the URL the app asked for, with no mark. The app's + * script so runs at its own URL: its import.meta.url, the URLs its imports + * resolve against and a stack trace through it show no mark, and the + * worker's location hides the one its script URL carries (workerScript). + * A classic worker imports the app's script (importScripts) after its + * seeding. A module Web Worker imports its seeding (the route answers that + * import, marked 'p') and then the app's script. Chromium and WebKit let the + * route reach none of what a Shared Worker loads after its script, so a + * module Shared Worker imports its seeding as a data: module, which runs + * before every module the app's script imports; where its policy refuses + * data: scripts (dataOk false), its seeding runs in the answer itself, which + * then imports the app's script (import()), and holds each connect event + * that comes before the app's script has run, to give it, in order, to the + * handlers the app's script sets. */ -export function sharedScript(token: string, n: number, kind: 's' | 'x', own: string): string { - const bang = /^#![^\n\r\u2028\u2029]*(?:\r\n|[\n\r\u2028\u2029])?/.exec(own); - const head = bang === null ? '' : /[\n\r\u2028\u2029]$/.test(bang[0]) ? bang[0] : `${bang[0]}\n`; - const rest = own.slice(bang === null ? 0 : bang[0].length); - const seeding = workerScript(token, n, kind); - if (kind === 'x') return `${head}import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(seeding).toString('base64')}`)};${rest}`; - return `${head}${strictPrologue(rest) ? '\'use strict\';' : ''}${oneLine(seeding)}${rest}`; +export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true): string { + const url = JSON.stringify(own); + if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind)}importScripts(${url});\n`; + if (kind === 'm') return `import ${JSON.stringify(markWorker(own, 'p', token, n))};\nimport ${url};\n`; + const seeding = workerScript(token, n, 'x'); + if (dataOk) return `import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(seeding).toString('base64')}`)};\nimport ${url};\n`; + return `${seeding}const held = []; +const hold = (e) => { held.push(e); }; +addEventListener('connect', hold); +try { + await import(${url}); +} finally { + removeEventListener('connect', hold); + for (const e of held) dispatchEvent(new MessageEvent('connect', { data: e.data, origin: e.origin, lastEventId: e.lastEventId, source: e.source, ports: e.ports })); +} +`; } /** What the fixture hears from workerRoutes. */ export type WorkerHooks = { /** The route answered this request whole: none of it is in flight. */ answered?: (r: Request) => void; - /** A worker the route cannot seed, by its script's URL as the app asked. */ - unreached?: (url: string, why: 'csp') => void; }; /** - * Answers each marked worker script the test's context asks for: a - * classic Web Worker's with its seeding, then importScripts of the app's - * own script; a module Web Worker's with an import of its seeding, then - * one of the app's own; the app's own script with what its server answered - * to the worker's request (fetched once, unchanged); a module worker's - * seeding with workerScript; a Shared Worker's with sharedScript (a module - * one whose policy refuses data: scripts with the app's own, unseeded, and - * unreached says so). An answer that is not ok goes to the worker as the - * server gave it, and a request keploy cannot fetch goes on with no mark. - * Each instance of a token (a Shared Worker started again) is the next n. - * Each request it fulfills is answered: Chromium reports no end for a - * Shared Worker's script, as Playwright does not follow Shared Workers. + * Answers each marked worker script the test's context asks for with + * workerAnswer, under the status and headers the app's server answered for + * the script with no mark (a worker's policy comes from its script's + * answer), and a module Web Worker's seeding (marked 'p') with + * workerScript. So the app's server is asked for a worker's script twice: + * here, for that answer, and by the worker, for the app's script. An answer + * that is not ok goes to the worker as the server gave it; when keploy + * cannot fetch the script, the worker gets its seeding with no header of the + * app's and loads the app's script itself, as the browser would. Each + * instance of a token (a Shared Worker started again) is the next n. Each + * request it fulfills is answered: Chromium reports no end for a Shared + * Worker's script, as Playwright does not follow Shared Workers. */ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}): Promise { const counts = new Map(); - const kept = new Map; body: Buffer }>(); // The answer's headers, but its encoding and length: the body given is - // decoded, and of another length. + // keploy's, of another length. const headersOf = (h: Record): Record => { const out: Record = {}; for (const [k, v] of Object.entries(h)) if (k !== 'content-encoding' && k !== 'content-length') out[k] = v; @@ -1447,44 +1424,21 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); return; } - if (m.kind === 'o') { - const k = kept.get(`${m.token}.${m.n}`); - kept.delete(`${m.token}.${m.n}`); - if (k === undefined) await route.continue({ url: m.url }); - else await fulfill(k); - return; - } const n = counts.get(m.token) ?? 0; counts.set(m.token, n + 1); - let answer; + let status = 200; + let headers: Record = { 'content-type': 'text/javascript' }; try { - answer = await route.fetch({ url: m.url }); - } catch { - await route.continue({ url: m.url }); - return; - } - const headers = headersOf(answer.headers()); - if (!answer.ok()) { - await fulfill({ status: answer.status(), headers, body: await answer.body() }); - return; - } - if (m.kind === 's' || m.kind === 'x') { - const body = await answer.body(); - if (m.kind === 'x' && !cspAllowsData(headers)) { - hooks.unreached?.(m.url, 'csp'); - await fulfill({ status: answer.status(), headers, body }); + const answer = await route.fetch({ url: m.url }); + if (!answer.ok()) { + await fulfill({ status: answer.status(), headers: headersOf(answer.headers()), body: await answer.body() }); return; } - // Scripts of workers are UTF-8; the browser drops a byte order mark. - const own = body.toString('utf8').replace(/^\ufeff/, ''); - await fulfill({ status: answer.status(), headers, body: sharedScript(m.token, n, m.kind, own) }); - return; + status = answer.status(); + headers = headersOf(answer.headers()); + } catch { + // The worker loads the app's script itself, and fails as it would. } - kept.set(`${m.token}.${n}`, { status: answer.status(), headers, body: await answer.body() }); - const own = JSON.stringify(markWorker(m.url, 'o', m.token, n)); - const body = m.kind === 'w' - ? `${workerScript(m.token, n, 'w')}importScripts(${own});\n` - : `import ${JSON.stringify(markWorker(m.url, 'p', m.token, n))};\nimport ${own};\n`; - await fulfill({ status: answer.status(), headers, body }); + await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind, m.url, m.kind !== 'x' || cspAllowsData(headers)) }); }); } diff --git a/playwright/test/draws.mjs b/playwright/test/draws.mjs index 13fda8b..7ec969c 100644 --- a/playwright/test/draws.mjs +++ b/playwright/test/draws.mjs @@ -69,8 +69,9 @@ export function documentAt(script, path = '/p', storage = new Map(), page = 't0' } // A worker's global scope: its own Crypto and a location whose href, -// search and toString come from WorkerLocation.prototype. -export function workerAt(href, prelude) { +// search and toString come from WorkerLocation.prototype, and the globals +// given. +export function workerAt(href, prelude, globals = {}) { class Crypto { getRandomValues(a) { return webcrypto.getRandomValues(a); } randomUUID() { return webcrypto.randomUUID(); } @@ -80,7 +81,7 @@ export function workerAt(href, prelude) { get search() { return new URL(href).search; } toString() { return href; } } - const ctx = { Crypto, crypto: new Crypto(), WorkerLocation, location: new WorkerLocation() }; + const ctx = { Crypto, crypto: new Crypto(), WorkerLocation, location: new WorkerLocation(), ...globals }; vm.createContext(ctx); if (prelude) vm.runInContext(prelude, ctx); return { run: (code) => vm.runInContext(code, ctx), draws: () => JSON.parse(JSON.stringify(vm.runInContext(DRAWS, ctx))) }; diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index 7ad058a..30e5f3c 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -5,13 +5,13 @@ import { test } from '../../dist'; // a classic Web Worker (which loads a script beside it with importScripts), // a module Web Worker (which imports a module beside it), a classic Shared // Worker and three module Shared Workers (each importing a module beside -// it); each sends what it drew, its location and what it loaded. The test +// it; one connected to twice); each sends what it drew, its location and what it loaded. The test // keeps them. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); test('workers', async ({ page }) => { await page.goto(`${base}/wk`); - await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 6, null, { timeout: 20000 }); + await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 7, null, { timeout: 20000 }); keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 9987ee6..080f91c 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, markWorker, workerMark, workerScript, workerRoutes, sharedScript, strictPrologue, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, markWorker, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -1344,7 +1344,7 @@ const workerGlobals = () => { test('a worker\'s mark: what markWorker makes, workerMark reads back, the URL as the app asked for it, query and fragment kept; any other URL has none', () => { const t = 'ab'.repeat(16); for (const url of ['http://h/w.js', 'http://h/w.js?a=1', 'http://h/w.js?', 'http://h/w.js#f', 'http://h/w.js?a=1#f?x']) { - for (const [kind, n] of [['w', -1], ['m', -1], ['o', 0], ['p', 7]]) { + for (const [kind, n] of [['w', -1], ['m', -1], ['s', -1], ['x', -1], ['p', 0], ['p', 7]]) { const m = markWorker(url, kind, t, n); assert.ok(new URL(m).searchParams.has(WORKER_PARAM), m); assert.deepEqual(workerMark(m), { kind, token: t, n, url }, m); @@ -1352,7 +1352,7 @@ test('a worker\'s mark: what markWorker makes, workerMark reads back, the URL as } assert.equal(markWorker('http://h/w.js?a=1#f', 'w', t), `http://h/w.js?a=1&__keploy_worker=w.${t}#f`); for (const u of ['http://h/w.js', `http://h/w.js?__keploy_worker=w.${t}&a=1`, `http://h/w.js?__keploy_worker=z.${t}`, `http://h/w.js?__keploy_worker=w.${t.toUpperCase()}`, - `http://h/w.js?__keploy_worker=w.${t}.0`, `http://h/w.js?__keploy_worker=o.${t}`, `http://h/w.js?__keploy_worker=w.${t.slice(1)}`]) { + `http://h/w.js?__keploy_worker=w.${t}.0`, `http://h/w.js?__keploy_worker=o.${t}`, `http://h/w.js?__keploy_worker=o.${t}.0`, `http://h/w.js?__keploy_worker=p.${t}`, `http://h/w.js?__keploy_worker=w.${t.slice(1)}`]) { assert.equal(workerMark(u), null, u); } }); @@ -1464,43 +1464,38 @@ function routed() { const answerOf = (status, body, headers = { 'content-type': 'text/javascript', 'content-encoding': 'gzip', 'content-length': '3', 'x-a': '1' }) => ({ ok: () => status >= 200 && status < 300, status: () => status, headers: () => headers, body: async () => Buffer.from(body) }); -test('workerRoutes answers a marked classic worker script with its seeding then importScripts of the app\'s own, fetched once and given unchanged; each start of a token is the next instance', async () => { +test('workerRoutes answers a marked classic worker script with its seeding then importScripts of the app\'s own at its URL with no mark, under the status and headers the app\'s server gave it; each start of a token is the next instance', async () => { const { r, context } = routed(); await workerRoutes(context); const t = '01'.repeat(16); assert.equal(r.match(new URL(markWorker('http://h/w.js', 'w', t))), true); assert.equal(r.match(new URL('http://h/w.js?a=1')), false); const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; - const own0 = markWorker('http://h/w.js?a=1', 'o', t, 0); - assert.deepEqual(await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')), [['fetch', 'http://h/w.js?a=1'], - ['fulfill', { status: 200, headers, body: `${workerScript(t, 0, 'w')}importScripts(${JSON.stringify(own0)});\n` }]]); - assert.deepEqual(await r.ask(own0), [['fulfill', { status: 200, headers, body: 'app()' }]]); - // Asked again (the browser's own retry): not kept, so as the app asked. - assert.deepEqual(await r.ask(own0), [['continue', 'http://h/w.js?a=1']]); + assert.deepEqual(await r.ask(markWorker('http://h/w.js?a=1#f', 'w', t), answerOf(201, 'app()')), [['fetch', 'http://h/w.js?a=1#f'], + ['fulfill', { status: 201, headers, body: `${workerScript(t, 0, 'w')}importScripts("http://h/w.js?a=1#f");\n` }]]); const second = await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')); - assert.equal(second[1][1].body, `${workerScript(t, 1, 'w')}importScripts(${JSON.stringify(markWorker('http://h/w.js?a=1', 'o', t, 1))});\n`); + assert.equal(second[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1')); + assert.equal(workerAnswer(t, 1, 'w', 'http://h/w.js?a=1'), `${workerScript(t, 1, 'w')}importScripts("http://h/w.js?a=1");\n`); }); -test('workerRoutes answers a module worker script with an import of its seeding then one of the app\'s own; its seeding is workerScript; an answer not ok goes as given, one it cannot fetch goes on unmarked, a malformed mark falls back', async () => { +test('workerRoutes answers a module worker script with an import of its seeding then one of the app\'s own; its seeding is workerScript; an answer not ok goes as given; one it cannot fetch gets its seeding with no header of the app\'s; a malformed mark falls back', async () => { const { r, context } = routed(); - await workerRoutes(context); + const answered = []; + await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); const t = '23'.repeat(16); const did = await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'export {}')); - assert.equal(did[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};\nimport ${JSON.stringify(markWorker('http://h/m.js', 'o', t, 0))};\n`); + assert.equal(did[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};\nimport "http://h/m.js";\n`); assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0)), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 0, 'm') }]]); assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), answerOf(404, 'no')), [['fetch', 'http://h/m.js'], ['fulfill', { status: 404, headers: { 'content-type': 'text/javascript', 'x-a': '1' }, body: 'no' }]]); - assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), new Error('net')), [['fetch', 'http://h/m.js'], ['continue', 'http://h/m.js']]); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), new Error('net')), [['fetch', 'http://h/m.js'], + ['fulfill', { status: 200, headers: { 'content-type': 'text/javascript' }, body: workerAnswer(t, 2, 'w', 'http://h/m.js') }]]); + const x = markWorker('http://h/x.js', 'x', t); + assert.equal((await r.ask(x, new Error('net')))[1][1].body, workerAnswer(t, 3, 'x', 'http://h/x.js', true)); + assert.equal(answered.length, 5); assert.deepEqual(await r.ask(`http://h/m.js?__keploy_worker=w.${t}.3`), [['fallback']]); -}); - -test('strictPrologue: a Use Strict Directive anywhere in the directive prologue, after spaces and comments; not one that an expression goes on from, one with an escape, or one after a statement', () => { - const yes = ["'use strict';x()", '"use strict"\nx()', "// c\n/* d */ 'use strict'", "'use asm'; 'use strict'; f()", "'use strict'\n!x", "'use strict'\n++x", "'use strict' /* c */\n'next'", "'use strict'", "'a'\n'use strict'\n", "'use strict'}", - "'use strict'\n--x", "'use strict'\ninit()", "'use strict' // c\nx()", "'use strict' /* a\nb */ x()"]; - const no = ["'use strict'\n(x)", "'use strict'.x", "'use\\x20strict';", "x; 'use strict'", "'use strict'\n+x", "'use strict'\nin x", "'use strict' + 1", "'use strict'\n[1]", '', "'unterminated", "'a'\n.x\n'use strict'", "'use strict' // c\n, 1", "'a' x 'use strict'", - "'use strict'\n- x", "'use strict'\n!= 1", "'use strict'\ninstanceof X"]; - for (const src of yes) assert.equal(strictPrologue(src), true, JSON.stringify(src)); - for (const src of no) assert.equal(strictPrologue(src), false, JSON.stringify(src)); + assert.deepEqual(await r.ask(`http://h/m.js?__keploy_worker=o.${t}.3`), [['fallback']]); + assert.equal(answered.length, 5); }); test('cspAllowsData: no policy, or each policy whose script or worker source list is named lists data:; a policy that names one without it refuses', () => { @@ -1512,54 +1507,94 @@ test('cspAllowsData: no policy, or each policy whose script or worker source lis "script-src data:; Worker-Src 'self'"]) assert.equal(cspAllowsData(csp(v)), false, v); }); -test('sharedScript: a classic Shared Worker\'s answer is its seeding on the app\'s first line (no line moves), after a hashbang, under the app\'s use strict; a module one\'s imports its seeding as a data: module first', () => { +test('workerAnswer: a classic Shared Worker runs its seeding, then imports the app\'s script at its URL; its location shows no mark; a module one imports its seeding as a data: module, then the app\'s script', () => { const t = '45'.repeat(16); - const seeding = workerScript(t, 2, 's'); - assert.doesNotMatch(seeding, /(^|[^:'"\\])\/\//, 'the seeding holds no line comment'); - const line = sharedScript(t, 2, 's', '').split('\n'); - assert.equal(line.length, 1); - assert.equal(sharedScript(t, 2, 's', 'a()\nb()'), `${line[0]}a()\nb()`); - assert.equal(sharedScript(t, 2, 's', '"use strict";\na()'), `'use strict';${line[0]}"use strict";\na()`); - assert.equal(sharedScript(t, 2, 's', '#!/x\r\n\'use strict\'\na()'), `#!/x\r\n'use strict';${line[0]}'use strict'\na()`); - assert.equal(sharedScript(t, 2, 's', '#!/x'), `#!/x\n${line[0]}`); - // The one line draws as the seeding does, and its location shows no mark. const href = markWorker('http://h/s.js?q=1', 's', t); - const w = workerAt(href, sharedScript(t, 2, 's', 'globalThis.app = location.href;')); - assert.deepEqual(w.draws(), workerAt(href, seeding).draws()); - assert.equal(w.run('app'), 'http://h/s.js?q=1'); - const m = sharedScript(t, 2, 'x', '#!/x\nimport "./a.js";'); - const data = /^#!\/x\nimport "data:text\/javascript;base64,([A-Za-z0-9+/=]+)";import "\.\/a\.js";$/.exec(m); + const imported = []; + const w = workerAt(href, workerAnswer(t, 2, 's', 'http://h/s.js?q=1'), { importScripts: (...u) => { imported.push(u); } }); + assert.deepEqual(imported, [['http://h/s.js?q=1']]); + assert.deepEqual(w.draws(), workerAt(href, workerScript(t, 2, 's')).draws()); + assert.equal(w.run('location.href'), 'http://h/s.js?q=1'); + const m = workerAnswer(t, 2, 'x', 'http://h/m.js?q=1'); + const data = /^import "data:text\/javascript;base64,([A-Za-z0-9+/=]+)";\nimport "http:\/\/h\/m\.js\?q=1";\n$/.exec(m); assert.ok(data, m.slice(0, 120)); assert.equal(Buffer.from(data[1], 'base64').toString(), workerScript(t, 2, 'x')); - assert.equal(sharedScript(t, 2, 'x', 'x()'), `import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(workerScript(t, 2, 'x')).toString('base64')}`)};x()`); }); -test('workerRoutes answers a Shared Worker\'s script whole with sharedScript (the app\'s byte order mark dropped), a module one whose policy refuses data: as the app\'s own and says so; each request it fulfills is answered, none it lets go on', async () => { +// A module Shared Worker's answer where data: is refused, run as a script: +// its import() is load, and its global holds an EventTarget's listeners. +function heldWorker(t, load) { + const body = workerAnswer(t, 0, 'x', 'http://h/c.js', false); + assert.equal(body.split('await import("http://h/c.js")').length, 2, body.slice(-400)); + const events = new EventTarget(); + const href = markWorker('http://h/c.js', 'x', t); + const w = workerAt(href, '', { + addEventListener: events.addEventListener.bind(events), + removeEventListener: events.removeEventListener.bind(events), + dispatchEvent: events.dispatchEvent.bind(events), + MessageEvent, load, + }); + const done = w.run(`(async () => { ${body.replace('await import(', 'await load(')} })()`); + return { w, events, done }; +} + +test('workerAnswer: a module Shared Worker whose policy refuses data: runs its seeding in its answer, then imports the app\'s script; each connect event before that script has run is held and given after, in order, with its ports; one after goes to the app\'s handler as it comes', async () => { + const t = '89'.repeat(16); + let loaded; + const got = []; + const { w, events, done } = heldWorker(t, () => new Promise((resolve) => { loaded = resolve; })); + assert.deepEqual(w.draws(), workerAt(markWorker('http://h/c.js', 'x', t), workerScript(t, 0, 'x')).draws()); + assert.equal(w.run('location.href'), 'http://h/c.js'); + const a = new MessageChannel(); + const b = new MessageChannel(); + events.dispatchEvent(new MessageEvent('connect', { data: '', ports: [a.port1] })); + events.dispatchEvent(new MessageEvent('connect', { data: '', ports: [b.port1] })); + // The app's script sets its handler as it runs. + events.addEventListener('connect', (e) => got.push(e.ports[0])); + assert.deepEqual(got, []); + loaded(); + await done; + assert.equal(got.length, 2); + assert.ok(got[0] === a.port1 && got[1] === b.port1); + const c = new MessageChannel(); + events.dispatchEvent(new MessageEvent('connect', { data: '', ports: [c.port1] })); + assert.equal(got.length, 3); + assert.ok(got[2] === c.port1); + for (const ch of [a, b, c]) { ch.port1.close(); ch.port2.close(); } +}); + +test('workerAnswer: a module Shared Worker whose app script fails to load fails as it would, and the connect events it held are still given', async () => { + const t = '8a'.repeat(16); + const { events, done } = heldWorker(t, () => Promise.reject(new TypeError('no module'))); + const got = []; + events.addEventListener('connect', (e) => got.push(e)); + const ch = new MessageChannel(); + events.dispatchEvent(new MessageEvent('connect', { data: '', ports: [ch.port1] })); + await assert.rejects(done, /no module/); + assert.equal(got.length, 2); + ch.port1.close(); ch.port2.close(); +}); + +test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module one whose policy refuses data: with its seeding in the answer; each request it fulfills is answered', async () => { const { r, context } = routed(); const answered = []; - const unreached = []; - await workerRoutes(context, { answered: (q) => answered.push(q.url()), unreached: (u, why) => unreached.push([u, why]) }); + await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); const t = '67'.repeat(16); const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; const s0 = markWorker('http://h/s.js', 's', t); - assert.deepEqual(await r.ask(s0, answerOf(200, '\ufeffapp()')), [['fetch', 'http://h/s.js'], ['fulfill', { status: 200, headers, body: sharedScript(t, 0, 's', 'app()') }]]); - assert.equal((await r.ask(s0, answerOf(200, 'app()')))[1][1].body, sharedScript(t, 1, 's', 'app()')); + assert.deepEqual(await r.ask(s0, answerOf(200, 'app()')), [['fetch', 'http://h/s.js'], ['fulfill', { status: 200, headers, body: workerAnswer(t, 0, 's', 'http://h/s.js') }]]); + assert.equal((await r.ask(s0, answerOf(200, 'app()')))[1][1].body, workerAnswer(t, 1, 's', 'http://h/s.js')); const x0 = markWorker('http://h/m.js', 'x', t.replace('6', '8')); - assert.equal((await r.ask(x0, answerOf(200, 'export {}')))[1][1].body, sharedScript(t.replace('6', '8'), 0, 'x', 'export {}')); + assert.equal((await r.ask(x0, answerOf(200, 'export {}')))[1][1].body, workerAnswer(t.replace('6', '8'), 0, 'x', 'http://h/m.js', true)); const policy = { 'content-type': 'text/javascript', 'content-security-policy': "script-src 'self'" }; const x1 = markWorker('http://h/c.js', 'x', t.replace('6', '9')); - assert.deepEqual(await r.ask(x1, answerOf(200, '\ufeffexport {}', policy)), [['fetch', 'http://h/c.js'], ['fulfill', { status: 200, headers: policy, body: '\ufeffexport {}' }]]); - assert.deepEqual(unreached, [['http://h/c.js', 'csp']]); - // A classic one is not refused by a policy: its seeding is in its own script. + assert.deepEqual(await r.ask(x1, answerOf(200, 'export {}', policy)), [['fetch', 'http://h/c.js'], ['fulfill', { status: 200, headers: policy, body: workerAnswer(t.replace('6', '9'), 0, 'x', 'http://h/c.js', false) }]]); + assert.notEqual(workerAnswer(t, 0, 'x', 'http://h/c.js', false), workerAnswer(t, 0, 'x', 'http://h/c.js', true)); + // A classic one is not refused by a policy: it imports the app's script. const d0 = markWorker('http://h/d.js', 's', t.replace('6', 'a')); - assert.equal((await r.ask(d0, answerOf(200, 'app()', policy)))[1][1].body, sharedScript(t.replace('6', 'a'), 0, 's', 'app()')); + assert.equal((await r.ask(d0, answerOf(200, 'app()', policy)))[1][1].body, workerAnswer(t.replace('6', 'a'), 0, 's', 'http://h/d.js')); assert.deepEqual(answered, [s0, s0, x0, x1, d0]); - // Not answered: what goes on, and what falls back. - await r.ask(markWorker('http://h/e.js', 's', t), new Error('net')); - await r.ask(markWorker('http://h/w.js', 'o', t, 5)); + // Not answered: what falls back. await r.ask(`http://h/m.js?__keploy_worker=s.${t}.3`); assert.equal(answered.length, 5); - assert.equal(unplacedLine('t', 'http://h/c.js', 'csp'), 'keploy: in t, the Shared Worker at http://h/c.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + - 'it is a module Shared Worker whose script is served with a Content-Security-Policy that refuses data: scripts, and keploy puts its seeding ahead of ' + - 'such a worker\'s modules only as a data: import; allow data: in that policy\'s script-src and worker-src for the test run to seed it.'); }); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index d71a4bd..5f1b962 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -614,15 +614,16 @@ async function drawsServer() { if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } // workers.spec.ts's workers, and the scripts they load. const wk = { - '/wk/ded.js': `importScripts('dep.js'); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep });`, + '/wk/ded.js': `importScripts('dep.js'); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack) });`, '/wk/dep.js': 'self.dep = \'dep\';', - '/wk/mod.js': `import { dep } from './lib.mjs'; postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep });`, + '/wk/mod.js': `import { dep } from './lib.mjs'; postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url });`, '/wk/lib.mjs': 'export const dep = \'lib\';', - '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href });`, - // Module Shared Workers: one plain, one (started twice, by two names) - // served with a policy that refuses data: scripts. - '/wk/mshared.js': `import { dep } from './lib.mjs'; self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep });`, - '/wk/cshared.js': `import { dep } from './lib.mjs'; self.onconnect = (e) => e.ports[0].postMessage({ kind: self.name, draws: ${DRAWS}, href: location.href, dep });`, + '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack) });`, + // Module Shared Workers: one plain, one (started by two names, c1 + // twice) served with a policy that refuses data: scripts. Each + // connection after a worker's first is named by its number (c1+2). + '/wk/mshared.js': `import { dep } from './lib.mjs'; self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url });`, + '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; self.onconnect = (e) => { n++; e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url }); };`, }[u.pathname]; if (wk !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', ...(u.pathname === '/wk/cshared.js' ? { 'content-security-policy': 'script-src \'self\'' } : {}) }); @@ -1278,6 +1279,7 @@ const WK_STARTS = [ 'new SharedWorker(\'/wk/mshared.js\', { type: \'module\', name: \'mn\' })', 'new SharedWorker(\'/wk/cshared.js\', { type: \'module\', name: \'c1\' })', 'new SharedWorker(\'/wk/cshared.js\', { type: \'module\', name: \'c2\' })', + 'new SharedWorker(\'/wk/cshared.js\', { type: \'module\', name: \'c1\' })', ]; // The marks a document at /wk of page t0 gives the workers workers.spec.ts @@ -1291,7 +1293,7 @@ function workerMarks(seed) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, at the app's location and loading what sits beside them, and the app's server never sees keploy's mark; a module Shared Worker whose policy refuses data: draws its own and is named once; in Firefox each Shared Worker draws its own and is named`, async () => { + test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, and the app's server never sees keploy's mark; in Firefox each Shared Worker draws its own and is named`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1302,30 +1304,44 @@ for (const engine of ENGINES) { try { const r = await drawRun(j, env(rec.url), ['workers.spec.ts'], ['workers']); const w = byKind(r); - const [ded, mod, shared, mshared, c1, c2] = workerMarks(r.seeds.workers); - assert.deepEqual([ded, mod, shared, mshared, c1, c2].map((m) => m.kind), ['w', 'm', 's', 'x', 'x', 'x']); + const [ded, mod, shared, mshared, c1, c2, c1again] = workerMarks(r.seeds.workers); + assert.deepEqual([ded, mod, shared, mshared, c1, c2, c1again].map((m) => m.kind), ['w', 'm', 's', 'x', 'x', 'x', 'x']); assert.notEqual(c1.token, c2.token); + assert.equal(c1again.token, c1.token); assert.match(w.ded.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/ded\.js\?x=1$/); assert.match(w.mod.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); assert.match(w.shared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/shared\.js$/); assert.match(w.mshared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); - for (const c of [w.c1, w.c2]) assert.match(c.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); - assert.deepEqual([w.ded.dep, w.mod.dep, w.mshared.dep, w.c1.dep, w.c2.dep], ['dep', 'lib', 'lib', 'lib', 'lib']); + for (const c of [w.c1, w.c2, w['c1+2']]) assert.match(c.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); + // The app's script runs at its own URL: its import.meta.url and a + // stack through it show no mark. + assert.match(w.mod.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); + assert.match(w.mshared.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); + for (const c of [w.c1, w.c2, w['c1+2']]) assert.match(c.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); + for (const k of ['ded', 'shared']) { + assert.match(w[k].stack, new RegExp(`/wk/${k === 'ded' ? 'ded' : 'shared'}\\.js`), w[k].stack); + assert.doesNotMatch(w[k].stack, /__keploy_worker/, w[k].stack); + } + assert.deepEqual([w.ded.dep, w.mod.dep, w.mshared.dep, w.c1.dep, w.c2.dep, w['c1+2'].dep], ['dep', 'lib', 'lib', 'lib', 'lib', 'lib']); + // c1's second connection is c1's worker's: what it drew as it loaded. + assert.deepEqual(w['c1+2'].draws, w.c1.draws); assert.deepEqual(w.ded.draws, workerAt('http://h/wk/ded.js?x=1', workerScript(ded.token, 0, 'w')).draws()); assert.deepEqual(w.mod.draws, workerAt('http://h/wk/mod.js', workerScript(mod.token, 0, 'm')).draws()); const model = workerAt('http://h/wk/shared.js', workerScript(shared.token, 0, 's')).draws(); const mmodel = workerAt('http://h/wk/mshared.js', workerScript(mshared.token, 0, 'x')).draws(); - for (const [c, m] of [[w.c1, c1], [w.c2, c2]]) assert.notDeepEqual(c.draws, workerAt('http://h/wk/cshared.js', workerScript(m.token, 0, 'x')).draws()); + const cmodel = [c1, c2].map((m) => workerAt('http://h/wk/cshared.js', workerScript(m.token, 0, 'x')).draws()); const said = r.tests.workers.stdout.split('\n').filter((l) => l.startsWith('keploy: in workers, ')).map((l) => l.replace(/:\d+\//, ':/')); const at = (p) => `http://127.0.0.1:/wk/${p}`; if (engine === 'firefox') { assert.notDeepEqual(w.shared.draws, model); assert.notDeepEqual(w.mshared.draws, mmodel); + assert.notDeepEqual([w.c1.draws, w.c2.draws], cmodel); assert.deepEqual(said.sort(), ['shared.js', 'mshared.js', 'cshared.js'].map((p) => unplacedLine('workers', at(p), 'shared')).sort()); } else { assert.deepEqual(w.shared.draws, model); assert.deepEqual(w.mshared.draws, mmodel); - assert.deepEqual(said, [unplacedLine('workers', at('cshared.js'), 'csp')]); + assert.deepEqual([w.c1.draws, w.c2.draws], cmodel); + assert.deepEqual(said, []); } // What the app's server was asked for: never keploy's mark, and each // worker's script and what it loads. @@ -1339,7 +1355,10 @@ for (const engine of ENGINES) { if (engine === 'firefox') assert.notDeepEqual(p[k].draws, w[k].draws, k); else assert.deepEqual(p[k].draws, w[k].draws, k); } - for (const k of ['c1', 'c2']) assert.notDeepEqual(p[k].draws, w[k].draws, k); + for (const k of ['c1', 'c2']) { + if (engine === 'firefox') assert.notDeepEqual(p[k].draws, w[k].draws, k); + else assert.deepEqual(p[k].draws, w[k].draws, k); + } assert.deepEqual(q.seen.filter((u) => u.includes('__keploy_worker')), []); } finally { await rep.close(); await rec.close(); } }); From 3bf5c95224077cc50a94f4c936ceed447d8a8276 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 09:35:31 +0530 Subject: [PATCH 23/73] refactor(playwright): read every worker answer's policy for data: workerRoutes asked cspAllowsData only for a module Shared Worker, though workerAnswer reads dataOk for no other kind: the guard changed nothing. The module Shared Worker row now also checks that its answer stops holding connect events once the app's script has run. Signed-off-by: slayerjain --- playwright/src/random.ts | 2 +- playwright/test/random.test.mjs | 14 +++++++++----- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/playwright/src/random.ts b/playwright/src/random.ts index c4f3789..0a03988 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1439,6 +1439,6 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } catch { // The worker loads the app's script itself, and fails as it would. } - await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind, m.url, m.kind !== 'x' || cspAllowsData(headers)) }); + await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind, m.url, cspAllowsData(headers)) }); }); } diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 080f91c..693efee 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1528,21 +1528,22 @@ function heldWorker(t, load) { assert.equal(body.split('await import("http://h/c.js")').length, 2, body.slice(-400)); const events = new EventTarget(); const href = markWorker('http://h/c.js', 'x', t); + const listeners = { added: [], removed: [] }; const w = workerAt(href, '', { - addEventListener: events.addEventListener.bind(events), - removeEventListener: events.removeEventListener.bind(events), + addEventListener: (type, f) => { listeners.added.push([type, f]); events.addEventListener(type, f); }, + removeEventListener: (type, f) => { listeners.removed.push([type, f]); events.removeEventListener(type, f); }, dispatchEvent: events.dispatchEvent.bind(events), MessageEvent, load, }); const done = w.run(`(async () => { ${body.replace('await import(', 'await load(')} })()`); - return { w, events, done }; + return { w, events, done, listeners }; } -test('workerAnswer: a module Shared Worker whose policy refuses data: runs its seeding in its answer, then imports the app\'s script; each connect event before that script has run is held and given after, in order, with its ports; one after goes to the app\'s handler as it comes', async () => { +test('workerAnswer: a module Shared Worker whose policy refuses data: runs its seeding in its answer, then imports the app\'s script; each connect event before that script has run is held and given after, in order, with its ports; then it holds none, and one after goes to the app\'s handler as it comes', async () => { const t = '89'.repeat(16); let loaded; const got = []; - const { w, events, done } = heldWorker(t, () => new Promise((resolve) => { loaded = resolve; })); + const { w, events, done, listeners } = heldWorker(t, () => new Promise((resolve) => { loaded = resolve; })); assert.deepEqual(w.draws(), workerAt(markWorker('http://h/c.js', 'x', t), workerScript(t, 0, 'x')).draws()); assert.equal(w.run('location.href'), 'http://h/c.js'); const a = new MessageChannel(); @@ -1554,6 +1555,9 @@ test('workerAnswer: a module Shared Worker whose policy refuses data: runs its s assert.deepEqual(got, []); loaded(); await done; + // It holds no more once the app's script has run. + assert.equal(listeners.added.length, 1); + assert.deepEqual(listeners.removed, listeners.added); assert.equal(got.length, 2); assert.ok(got[0] === a.port1 && got[1] === b.port1); const c = new MessageChannel(); From d606bdcc17081dd38c63d2b9999d598740cf97dc Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 09:55:30 +0530 Subject: [PATCH 24/73] fix(playwright): keep keploy's worker answer off the app's stack A classic Web or Shared Worker ran the app's script through importScripts called from keploy's answer, so in Chromium and Firefox a stack trace taken at the top level of the app's script ended in a frame at the marked URL. keploy's answer now queues the import in a microtask (queueMicrotask with importScripts bound to the URL): it runs as the answer ends, before the worker takes any message or connection, with none of keploy's code on the stack, and what the app's script throws is reported as the worker's own uncaught error, as before. A module Shared Worker whose policy refuses data: gives each connect event it held to the app's handlers in a microtask of its own, for the same reason. Rows: the browser row checks the stack at the top level of every worker kind and in a held connection's handler, and that a message the page posts as it starts a Web Worker reaches the app's handler; the unit rows check that the answer queues one microtask per import or held event and runs nothing of the app's before it. Signed-off-by: slayerjain --- playwright/src/random.ts | 29 ++++++++++-------- playwright/test/fixtures/workers.spec.ts | 7 +++-- playwright/test/random.test.mjs | 39 ++++++++++++++++-------- playwright/test/rows.test.mjs | 24 ++++++++++----- 4 files changed, 64 insertions(+), 35 deletions(-) diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 0a03988..131382f 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1352,20 +1352,25 @@ export function cspAllowsData(headers: Record): boolean { * script so runs at its own URL: its import.meta.url, the URLs its imports * resolve against and a stack trace through it show no mark, and the * worker's location hides the one its script URL carries (workerScript). - * A classic worker imports the app's script (importScripts) after its - * seeding. A module Web Worker imports its seeding (the route answers that - * import, marked 'p') and then the app's script. Chromium and WebKit let the - * route reach none of what a Shared Worker loads after its script, so a - * module Shared Worker imports its seeding as a data: module, which runs - * before every module the app's script imports; where its policy refuses - * data: scripts (dataOk false), its seeding runs in the answer itself, which - * then imports the app's script (import()), and holds each connect event - * that comes before the app's script has run, to give it, in order, to the - * handlers the app's script sets. + * No code of the answer's is on the stack when the app's code runs: a + * classic worker imports the app's script (importScripts) after its + * seeding, in a microtask the answer queues, which runs as the answer ends, + * before the worker takes any message or connection, and reports what the + * app's script throws as the worker's own error, as an uncaught one at its + * top level is. A module Web Worker imports its seeding (the route answers + * that import, marked 'p') and then the app's script. Chromium and WebKit + * let the route reach none of what a Shared Worker loads after its script, + * so a module Shared Worker imports its seeding as a data: module, which + * runs before every module the app's script imports; where its policy + * refuses data: scripts (dataOk false), its seeding runs in the answer + * itself, which then imports the app's script (import()), and holds each + * connect event that comes before the app's script has run, to give it, in + * order, each in a microtask of its own, to the handlers the app's script + * sets. */ export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true): string { const url = JSON.stringify(own); - if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind)}importScripts(${url});\n`; + if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; if (kind === 'm') return `import ${JSON.stringify(markWorker(own, 'p', token, n))};\nimport ${url};\n`; const seeding = workerScript(token, n, 'x'); if (dataOk) return `import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(seeding).toString('base64')}`)};\nimport ${url};\n`; @@ -1376,7 +1381,7 @@ try { await import(${url}); } finally { removeEventListener('connect', hold); - for (const e of held) dispatchEvent(new MessageEvent('connect', { data: e.data, origin: e.origin, lastEventId: e.lastEventId, source: e.source, ports: e.ports })); + for (const e of held) queueMicrotask(dispatchEvent.bind(globalThis, new MessageEvent('connect', { data: e.data, origin: e.origin, lastEventId: e.lastEventId, source: e.source, ports: e.ports }))); } `; } diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index 30e5f3c..3f76099 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -5,13 +5,14 @@ import { test } from '../../dist'; // a classic Web Worker (which loads a script beside it with importScripts), // a module Web Worker (which imports a module beside it), a classic Shared // Worker and three module Shared Workers (each importing a module beside -// it; one connected to twice); each sends what it drew, its location and what it loaded. The test -// keeps them. +// it; one connected to twice); each sends what it drew, its location, what +// it loaded and its stack, and each Web Worker answers the message the page +// posts as it starts it. The test keeps them. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); test('workers', async ({ page }) => { await page.goto(`${base}/wk`); - await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 7, null, { timeout: 20000 }); + await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 9, null, { timeout: 20000 }); keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 693efee..9189cff 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1464,7 +1464,7 @@ function routed() { const answerOf = (status, body, headers = { 'content-type': 'text/javascript', 'content-encoding': 'gzip', 'content-length': '3', 'x-a': '1' }) => ({ ok: () => status >= 200 && status < 300, status: () => status, headers: () => headers, body: async () => Buffer.from(body) }); -test('workerRoutes answers a marked classic worker script with its seeding then importScripts of the app\'s own at its URL with no mark, under the status and headers the app\'s server gave it; each start of a token is the next instance', async () => { +test('workerRoutes answers a marked classic worker script with its seeding then, in a microtask, importScripts of the app\'s own at its URL with no mark, under the status and headers the app\'s server gave it; each start of a token is the next instance', async () => { const { r, context } = routed(); await workerRoutes(context); const t = '01'.repeat(16); @@ -1472,10 +1472,10 @@ test('workerRoutes answers a marked classic worker script with its seeding then assert.equal(r.match(new URL('http://h/w.js?a=1')), false); const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; assert.deepEqual(await r.ask(markWorker('http://h/w.js?a=1#f', 'w', t), answerOf(201, 'app()')), [['fetch', 'http://h/w.js?a=1#f'], - ['fulfill', { status: 201, headers, body: `${workerScript(t, 0, 'w')}importScripts("http://h/w.js?a=1#f");\n` }]]); + ['fulfill', { status: 201, headers, body: `${workerScript(t, 0, 'w')}queueMicrotask(importScripts.bind(globalThis, "http://h/w.js?a=1#f"));\n` }]]); const second = await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')); assert.equal(second[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1')); - assert.equal(workerAnswer(t, 1, 'w', 'http://h/w.js?a=1'), `${workerScript(t, 1, 'w')}importScripts("http://h/w.js?a=1");\n`); + assert.equal(workerAnswer(t, 1, 'w', 'http://h/w.js?a=1'), `${workerScript(t, 1, 'w')}queueMicrotask(importScripts.bind(globalThis, "http://h/w.js?a=1"));\n`); }); test('workerRoutes answers a module worker script with an import of its seeding then one of the app\'s own; its seeding is workerScript; an answer not ok goes as given; one it cannot fetch gets its seeding with no header of the app\'s; a malformed mark falls back', async () => { @@ -1507,14 +1507,24 @@ test('cspAllowsData: no policy, or each policy whose script or worker source lis "script-src data:; Worker-Src 'self'"]) assert.equal(cspAllowsData(csp(v)), false, v); }); -test('workerAnswer: a classic Shared Worker runs its seeding, then imports the app\'s script at its URL; its location shows no mark; a module one imports its seeding as a data: module, then the app\'s script', () => { +test('workerAnswer: a classic Web or Shared Worker runs its seeding, then imports the app\'s script at its URL in a microtask it queues, with none of its own code on the stack; its location shows no mark; a module one imports its seeding as a data: module, then the app\'s script', () => { const t = '45'.repeat(16); - const href = markWorker('http://h/s.js?q=1', 's', t); - const imported = []; - const w = workerAt(href, workerAnswer(t, 2, 's', 'http://h/s.js?q=1'), { importScripts: (...u) => { imported.push(u); } }); - assert.deepEqual(imported, [['http://h/s.js?q=1']]); - assert.deepEqual(w.draws(), workerAt(href, workerScript(t, 2, 's')).draws()); - assert.equal(w.run('location.href'), 'http://h/s.js?q=1'); + for (const kind of ['s', 'w']) { + const href = markWorker('http://h/s.js?q=1', kind, t); + const imported = []; + const queued = []; + const w = workerAt(href, workerAnswer(t, 2, kind, 'http://h/s.js?q=1'), { + importScripts: function (...u) { imported.push([this, ...u]); }, + queueMicrotask: (f) => { queued.push(f); }, + }); + // Nothing of the app's runs while the answer's code is on the stack. + assert.deepEqual(imported, [], kind); + assert.equal(queued.length, 1, kind); + queued[0](); + assert.deepEqual(imported, [[w.run('globalThis'), 'http://h/s.js?q=1']], kind); + assert.deepEqual(w.draws(), workerAt(href, workerScript(t, 2, kind)).draws(), kind); + assert.equal(w.run('location.href'), 'http://h/s.js?q=1', kind); + } const m = workerAnswer(t, 2, 'x', 'http://h/m.js?q=1'); const data = /^import "data:text\/javascript;base64,([A-Za-z0-9+/=]+)";\nimport "http:\/\/h\/m\.js\?q=1";\n$/.exec(m); assert.ok(data, m.slice(0, 120)); @@ -1529,21 +1539,23 @@ function heldWorker(t, load) { const events = new EventTarget(); const href = markWorker('http://h/c.js', 'x', t); const listeners = { added: [], removed: [] }; + const queued = []; const w = workerAt(href, '', { addEventListener: (type, f) => { listeners.added.push([type, f]); events.addEventListener(type, f); }, removeEventListener: (type, f) => { listeners.removed.push([type, f]); events.removeEventListener(type, f); }, dispatchEvent: events.dispatchEvent.bind(events), MessageEvent, load, + queueMicrotask: (f) => { queued.push(f); queueMicrotask(f); }, }); const done = w.run(`(async () => { ${body.replace('await import(', 'await load(')} })()`); - return { w, events, done, listeners }; + return { w, events, done, listeners, queued }; } test('workerAnswer: a module Shared Worker whose policy refuses data: runs its seeding in its answer, then imports the app\'s script; each connect event before that script has run is held and given after, in order, with its ports; then it holds none, and one after goes to the app\'s handler as it comes', async () => { const t = '89'.repeat(16); let loaded; const got = []; - const { w, events, done, listeners } = heldWorker(t, () => new Promise((resolve) => { loaded = resolve; })); + const { w, events, done, listeners, queued } = heldWorker(t, () => new Promise((resolve) => { loaded = resolve; })); assert.deepEqual(w.draws(), workerAt(markWorker('http://h/c.js', 'x', t), workerScript(t, 0, 'x')).draws()); assert.equal(w.run('location.href'), 'http://h/c.js'); const a = new MessageChannel(); @@ -1560,6 +1572,9 @@ test('workerAnswer: a module Shared Worker whose policy refuses data: runs its s assert.deepEqual(listeners.removed, listeners.added); assert.equal(got.length, 2); assert.ok(got[0] === a.port1 && got[1] === b.port1); + // Each in a microtask of its own: none of the answer's code is on the + // stack of the app's handler. + assert.equal(queued.length, 2); const c = new MessageChannel(); events.dispatchEvent(new MessageEvent('connect', { data: '', ports: [c.port1] })); assert.equal(got.length, 3); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 5f1b962..f7817aa 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -614,16 +614,17 @@ async function drawsServer() { if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } // workers.spec.ts's workers, and the scripts they load. const wk = { - '/wk/ded.js': `importScripts('dep.js'); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack) });`, + // Each Web Worker answers the message its page posts as it starts it. + '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack) });`, '/wk/dep.js': 'self.dep = \'dep\';', - '/wk/mod.js': `import { dep } from './lib.mjs'; postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url });`, + '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack) });`, '/wk/lib.mjs': 'export const dep = \'lib\';', '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack) });`, // Module Shared Workers: one plain, one (started by two names, c1 // twice) served with a policy that refuses data: scripts. Each // connection after a worker's first is named by its number (c1+2). - '/wk/mshared.js': `import { dep } from './lib.mjs'; self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url });`, - '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; self.onconnect = (e) => { n++; e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url }); };`, + '/wk/mshared.js': `import { dep } from './lib.mjs'; const top = String(new Error('at').stack); self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top });`, + '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; const top = String(new Error('at').stack); self.onconnect = (e) => { n++; e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url, top, stack: String(new Error('at').stack) }); };`, }[u.pathname]; if (wk !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', ...(u.pathname === '/wk/cshared.js' ? { 'content-security-policy': 'script-src \'self\'' } : {}) }); @@ -632,7 +633,7 @@ async function drawsServer() { } res.writeHead(200, { 'content-type': 'text/html' }); if (u.pathname === '/wk') { - const start = (s) => (s.startsWith('new Worker(') ? `${s}.onmessage = take;` : `{ const s = ${s}; s.port.onmessage = take; s.port.start(); }`); + const start = (s) => (s.startsWith('new Worker(') ? `{ const w = ${s}; w.onmessage = take; w.postMessage('early'); }` : `{ const s = ${s}; s.port.onmessage = take; s.port.start(); }`); res.end(page(``)); return; } @@ -1318,10 +1319,17 @@ for (const engine of ENGINES) { assert.match(w.mod.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); assert.match(w.mshared.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); for (const c of [w.c1, w.c2, w['c1+2']]) assert.match(c.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); - for (const k of ['ded', 'shared']) { - assert.match(w[k].stack, new RegExp(`/wk/${k === 'ded' ? 'ded' : 'shared'}\\.js`), w[k].stack); - assert.doesNotMatch(w[k].stack, /__keploy_worker/, w[k].stack); + // No code of keploy's is on a stack through the app's script, at its + // top level or in its handlers, held connections' too. + const stacks = [['ded', 'stack', 'ded'], ['shared', 'stack', 'shared'], ['mod', 'top', 'mod'], ['mshared', 'top', 'mshared'], + ...['c1', 'c2', 'c1+2'].flatMap((c) => [[c, 'top', 'cshared'], [c, 'stack', 'cshared']])]; + for (const [k, f, file] of stacks) { + assert.match(w[k][f], new RegExp(`/wk/${file}\\.js[?:]`), `${k}.${f}: ${w[k][f]}`); + assert.doesNotMatch(w[k][f], /__keploy_worker/, `${k}.${f}: ${w[k][f]}`); } + // A message the page posts as it starts a Web Worker reaches the app's + // handler: the app's script has run before the worker takes it. + assert.deepEqual([w['ded-early'], w['mod-early']], [{ kind: 'ded-early', data: 'early' }, { kind: 'mod-early', data: 'early' }]); assert.deepEqual([w.ded.dep, w.mod.dep, w.mshared.dep, w.c1.dep, w.c2.dep, w['c1+2'].dep], ['dep', 'lib', 'lib', 'lib', 'lib', 'lib']); // c1's second connection is c1's worker's: what it drew as it loaded. assert.deepEqual(w['c1+2'].draws, w.c1.draws); From ee271047b60d167113c4aa3ccba484e75a758354 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 10:13:28 +0530 Subject: [PATCH 25/73] fix(playwright): leave keploy's frames out of a worker's Firefox stacks Firefox keeps, in a stack, the frames of the code that queued what is running. So in a seeded classic Web Worker, an Error made at the top level of the app's script still showed the frame of keploy's answer that queued its import, at the worker's URL with keploy's mark. A worker's seeding now wraps the stack getter of Error.prototype and DOMException.prototype where the engine reads a stack through it (Firefox): a stack read in the worker leaves out each frame at a URL whose query has keploy's mark, and keeps the getter's setter and flags. Chromium and WebKit keep each error's stack on the error itself and show no such frame; their prototypes are left as they are. Rows: a unit row reads an Error's and a DOMException's stack through the prototype's getter and checks which frames are left out, that the setter, the flags and a value that is no string are kept, and that a stack that is no getter stays as it was; the browser row already checks the Firefox stack at the top level of the app's script. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/random.ts | 22 ++++++++++++++++++++-- playwright/test/random.test.mjs | 27 +++++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 3 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 765b026..ff40256 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server never sees the mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark, and the worker's `location` shows none. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server never sees the mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 131382f..0571e70 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1281,7 +1281,10 @@ export function markWorker(url: string, kind: WorkerMark['kind'], token: string, * instance of the token (a Shared Worker started again after its documents * closed is the next). Its location (href, search, toString) does not show * the mark kind.token the worker's URL carries; the app's script runs at - * its own URL (workerAnswer). + * its own URL (workerAnswer). An Error's or a DOMException's stack read + * through its prototype's getter (Firefox's, whose stacks go on through + * the code that queued what runs) leaves out each frame at a URL keploy + * marked: keploy's own code. */ export function workerScript(token: string, n: number, kind: 'w' | 'm' | 's' | 'x'): string { return `(() => { @@ -1317,6 +1320,19 @@ ${DRAW_JS} if (typeof t === 'function') define(L, 'toString', { toString() { return strip(t.call(this)); } }.toString); } catch (e) {} } + // Firefox reads a stack through its prototype's getter, and the stack + // goes on through the code that queued what runs: keploy's answer, at a + // URL keploy marked. A stack read here shows no frame at such a URL. + const marked = new RegExp(${JSON.stringify(`[?&]${WORKER_PARAM}=`)}); + for (const C of [g.Error, g.DOMException]) { + try { + const d = Object.getOwnPropertyDescriptor(C.prototype, 'stack'); + const get = d.get; + if (typeof get !== 'function') continue; + const v = { get stack() { const s = get.call(this); return typeof s === 'string' ? s.split('\\n').filter((l) => !marked.test(l)).join('\\n') : s; } }; + Object.defineProperty(C.prototype, 'stack', { get: Object.getOwnPropertyDescriptor(v, 'stack').get, set: d.set, enumerable: d.enumerable, configurable: true }); + } catch (e) {} + } })(); `; } @@ -1357,7 +1373,9 @@ export function cspAllowsData(headers: Record): boolean { * seeding, in a microtask the answer queues, which runs as the answer ends, * before the worker takes any message or connection, and reports what the * app's script throws as the worker's own error, as an uncaught one at its - * top level is. A module Web Worker imports its seeding (the route answers + * top level is (a Firefox stack, which goes on through the code that queued + * it, leaves the answer's frames out: workerScript). A module Web Worker + * imports its seeding (the route answers * that import, marked 'p') and then the app's script. Chromium and WebKit * let the route reach none of what a Shared Worker loads after its script, * so a module Shared Worker imports its seeding as a data: module, which diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 9189cff..dc6dadd 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1442,6 +1442,33 @@ test('a worker\'s seeding draws the same numbers for a token and instance every assert.equal(workerAt(other, workerScript(t, 0, 'w')).run('location.href'), other); }); +test('a worker\'s seeding: an Error\'s or a DOMException\'s stack read through its prototype\'s getter (Firefox) leaves out each frame at a URL whose query has keploy\'s mark, any kind; its setter, flags and a value that is no string are kept; a stack that is no getter stays as it was', () => { + const t = 'ab'.repeat(16); + const href = markWorker('http://h/w.js', 'w', t); + const kept = ['@http://h/w.js:1:2', `h@http://h/w.js?a__keploy_worker=w.${t}:7:8`]; + const frames = [kept[0], `VoidFunction*@${href}:114:15`, `g@http://h/w.js?b=1&__keploy_worker=o.${t}:5:6`, `f@${markWorker('http://h/m.js', 'p', t, 0)}:3:4`, kept[1], + `promise callback*@${markWorker('http://h/s.js?q=1', 's', 'ef'.repeat(16))}:9:1`, '']; + class DOMException {} + const w = workerAt(href, '', { DOMException }); + w.run('for (const C of [Error, DOMException]) Object.defineProperty(C.prototype, \'stack\', { get() { return this.s; }, set(v) { this.set = v; }, enumerable: false, configurable: true });'); + w.run(workerScript(t, 0, 'w')); + // V8 gives each error its own stack; an object made from the prototype + // reads it through the prototype's getter, as Firefox's errors do. + for (const C of ['Error', 'DOMException']) { + assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.s = ${JSON.stringify(frames.join('\n'))}; return e.stack; })()`), [...kept, ''].join('\n'), C); + assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.stack = 'x'; return e.set; })()`), 'x', C); + assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.s = 7; return e.stack; })()`), 7, C); + const d = w.run(`Object.getOwnPropertyDescriptor(${C}.prototype, 'stack')`); + assert.deepEqual([typeof d.get, typeof d.set, d.enumerable, d.configurable], ['function', 'function', false, true], C); + } + // A stack that is a value on the prototype, or none (V8, WebKit): as it was. + const v = workerAt(href, ''); + v.run('Object.defineProperty(Error.prototype, \'stack\', { value: \'v\', writable: true, enumerable: false, configurable: true });'); + v.run(workerScript(t, 0, 'w')); + assert.deepEqual(JSON.parse(v.run('JSON.stringify(Object.getOwnPropertyDescriptor(Error.prototype, \'stack\'))')), { value: 'v', writable: true, enumerable: false, configurable: true }); + assert.equal(workerAt(href, workerScript(t, 0, 'w')).run('Object.getOwnPropertyDescriptor(Error.prototype, \'stack\')'), undefined); +}); + // A context whose route keeps its matcher and handler, and a route of it // that records what the handler did. function routed() { From 96d5a731149b8e1d078456a5e073f8c426716e76 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 10:33:18 +0530 Subject: [PATCH 26/73] test(playwright): rows for a taken-over document's worker and a bad URL Two rows for the page's Worker wrapper: - A document that a second run of the seeding took over (WebKit runs it again in a document that takes over its window's first about:blank) starts a Web Worker: its script URL carries one mark, the same as at a document with no takeover, and nothing is named. The first run passes the worker on as the app asked. - A worker URL the browser cannot read throws the browser's own error, and nothing is named. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index dc6dadd..2e79ebb 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1330,6 +1330,27 @@ test('a document that a second run of the script takes over (WebKit runs it in a assert.deepEqual(drew, documentAt(initScript(S1), '/p', new Map(), 't0').draws()); }); +test('a Web Worker started in a document that a second run of the script took over is marked once, by that run, as at a document with no takeover; the first run passes it on as asked and names nothing', () => { + const made = []; + class Worker { constructor(url) { made.push(String(url)); } } + const told = []; + const blank = { baseURI: 'about:blank', querySelectorAll: () => [] }; + const d = documentAt(initScript(S1, 'tell'), 'blank', new Map(), 't0', { doc: blank, report: 'tell', told, globals: { URL, Worker } }); + d.ctx.document = { baseURI: 'http://h/p', querySelectorAll: () => [] }; + d.ctx.location.pathname = '/p'; + d.run(initScript(S1, 'tell')); + d.run(pageScript('t0')); + d.run('new Worker("/w.js")'); + assert.equal(made.length, 1); + assert.equal(made[0].split(`${WORKER_PARAM}=`).length, 2, made[0]); + assert.equal(workerMark(made[0]).url, 'http://h/w.js'); + const plain = []; + class Plain { constructor(url) { plain.push(String(url)); } } + documentAt(initScript(S1), '/p', new Map(), 't0', { globals: { URL, Worker: Plain } }).run('new Worker("/w.js")'); + assert.deepEqual(made, plain); + assert.deepEqual(told, []); +}); + // Workers: the page's Worker and SharedWorker mark the script URL with the // worker's token; the route answers the marked script with the worker's // seeding, then the app's own script. @@ -1383,6 +1404,12 @@ test('a worker keploy cannot mark starts as the app asked and is named once with d.run('new Worker("blob:http://h/0f0e"); new Worker("data:text/javascript,1"); new Worker("blob:http://h/0f0e");'); assert.deepEqual(w.made.map(([u]) => u), ['blob:http://h/0f0e', 'data:text/javascript,1', 'blob:http://h/0f0e']); assert.deepEqual(told, [['blob:', 'worker'], ['data:', 'worker'], ['blob:', 'worker']]); + // A URL the browser cannot read: the browser's own error, nothing named. + const told1 = []; + class Strict { constructor(url) { if (String(url) === 'http://[') throw new SyntaxError('the browser\'s own'); } } + const d1 = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told: told1, globals: { URL, Worker: Strict } }); + assert.throws(() => d1.run('new Worker("http://[")'), (e) => e.message === 'the browser\'s own'); + assert.deepEqual(told1, []); // The browser refuses the marked URL (a Trusted Types policy): as asked. const told2 = []; const made = []; From 7b740d40c7ae3a5b5369710fce9bb22b9146517b Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 10:33:18 +0530 Subject: [PATCH 27/73] fix(playwright): never wait on a Shared Worker's script before close Chromium reports no end for a Shared Worker's script request, since Playwright does not follow Shared Workers. A keploy test whose browser was not seeded had no route to answer that request, so settle waited on it until the test's timeout failed the test. A seeded Chromium test was fine, because keploy's route answered the script whole. Every keploy test's context, seeded or not, now runs a small init script. It wraps the page's SharedWorker: the worker starts as the app asked, with the same arguments, and then the test learns the script's URL (resolved against the document's base, no fragment) through a binding. The script takes the binding off the window. Settle does not wait on a request of that frame at that URL that is no navigation, whether the browser reports it before or after. Playwright reports none of a Shared Worker's own calls in any engine, so the test says once for each Shared Worker script address that its script and calls are not waited for, and what to do: wait in the test for what the worker does. The README says the same. Rows: unit rows for Inflight (before and after, fragment, another frame, URL or a navigation still waited for, two scripts of one frame) and for the init script (arguments as given, URL, binding removed, a refused worker throws the browser's own error and reports nothing, new, instanceof, constructor, name, length, prototype and flags kept, no binding leaves the browser's own); a browser row in each engine for an unseeded test whose page starts a Shared Worker, which passes under a 4 s timeout with no settle step and says the line once; the seeded workers row now expects the line for each Shared Worker script. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/index.ts | 19 ++++- playwright/src/settle.ts | 93 ++++++++++++++++++++++++- playwright/test/fixtures/settle.spec.ts | 10 +++ playwright/test/rows.test.mjs | 29 +++++++- playwright/test/unit.test.mjs | 82 +++++++++++++++++++++- 6 files changed, 227 insertions(+), 8 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index ff40256..f779bd9 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -43,7 +43,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. - **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped )`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. -- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. +- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see the limits below). diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 56a240c..7464536 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -60,7 +60,7 @@ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; -import { Inflight } from './settle'; +import { Inflight, sharedWorkerLine, sharedWorkerScript } from './settle'; export { expect, KeployError }; export { seedFaker } from './random'; @@ -89,6 +89,9 @@ const saidFaker = new Set(); // How many contexts this worker seeded: each one's report function has its // own name. let reports = 0; +// How many contexts of a keploy test this worker opened: each one's Shared +// Worker report function has its own name. +let sharedReports = 0; // How a test's browser draws its random numbers. type PageSeeding = { @@ -222,6 +225,20 @@ const keployTestType = base.extend({ restore = placing.restore; opened = placing.opened; } + // Seeded or not: each Shared Worker a page starts is one settle does not + // wait on, and the test says so once for each script address (origin + // and path). + const sharedReport = `__keploy_shared_worker_${++sharedReports}`; + const sharedTold = new Set(); + await context.exposeBinding(sharedReport, (source, url: unknown) => { + const u = String(url); + inflight.sharedWorker(source.frame, u); + const at = pageAddress(u.split('?')[0]); + if (sharedTold.has(at)) return; + sharedTold.add(at); + process.stdout.write(`${sharedWorkerLine(name, at)}\n`); + }); + await context.addInitScript({ content: sharedWorkerScript(sharedReport) }); const seeded = s.seed !== ''; const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; if (context.pages().length > 0) opened(); diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 28a3f18..d1985ac 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -13,6 +13,9 @@ export class Inflight { private waiters: (() => void)[] = []; // The calls keploy's own route answered whole (answered). private readonly ended = new WeakSet(); + // The script URLs, with no fragment, that each frame started a Shared + // Worker at, for the test's life (sharedWorker). + private readonly shared = new Map>(); constructor(context: BrowserContext) { context.on('request', (r) => { @@ -20,7 +23,9 @@ export class Inflight { let frame: Frame | null = null; // A service worker's call has no frame. try { frame = r.frame(); } catch { frame = null; } - this.open.set(r, { frame, nav: r.isNavigationRequest(), n: ++this.reported }); + const nav = r.isNavigationRequest(); + if (!nav && this.shared.get(frame)?.has(noFragment(r.url()))) return; + this.open.set(r, { frame, nav, n: ++this.reported }); }); context.on('requestfinished', (r) => this.done(r)); context.on('requestfailed', (r) => this.done(r)); @@ -106,14 +111,33 @@ export class Inflight { this.done(r); } + /** + * A document of frame started a Shared Worker whose script is at url + * (sharedWorkerScript). Settle never waits on a request of that frame at + * that URL (fragment aside) that is no navigation, reported before this or + * after, for the rest of the test: the worker's script is one, and Chromium + * reports no end for it, as Playwright does not follow Shared Workers. The + * test's line (sharedWorkerLine) names the URL. + */ + sharedWorker(frame: Frame | null, url: string): void { + const u = noFragment(url); + let s = this.shared.get(frame); + if (s === undefined) { + s = new Set(); + this.shared.set(frame, s); + } + s.add(u); + this.drop((o, r) => !o.nav && o.frame === frame && noFragment(r.url()) === u); + } + private done(r: Request): void { if (this.open.delete(r)) this.wake(); } - private drop(gone: (o: { frame: Frame | null; nav: boolean; n: number }) => boolean): void { + private drop(gone: (o: { frame: Frame | null; nav: boolean; n: number }, r: Request) => boolean): void { let n = 0; for (const [r, o] of this.open) { - if (gone(o)) { this.open.delete(r); n++; } + if (gone(o, r)) { this.open.delete(r); n++; } } if (n > 0) this.wake(); } @@ -129,3 +153,66 @@ export class Inflight { function pageOf(f: Frame): Page | null { try { return f.page(); } catch { return null; } } + +function noFragment(url: string): string { + const i = url.indexOf('#'); + return i < 0 ? url : url.slice(0, i); +} + +/** + * The init script every keploy test's context runs, seeded or not, after + * the seeding's (initScript): the page's SharedWorker, which starts each + * worker as the app asked, with the same arguments, then calls the binding + * report (exposed on the context, which this script takes off the window) + * with the script's URL, resolved against the document's base, with no + * fragment, so settle never waits on that script (Inflight.sharedWorker). + * A worker the browser refuses to start (a URL it cannot read) throws the + * browser's own error and reports nothing. It keeps new, instanceof, the + * constructor, name and length of the browser's own. + */ +export function sharedWorkerScript(report: string): string { + const r = JSON.stringify(report); + return `(() => { + const g = globalThis; + let tell = null; + try { const f = g[${r}]; if (typeof f === 'function') tell = f; delete g[${r}]; } catch (e) { tell = null; } + const Real = g.SharedWorker; + if (typeof Real !== 'function' || tell === null) return; + const U = g.URL; + const construct = Reflect.construct; + const slice = Array.prototype.slice; + const W = { SharedWorker: function (url) { + // Called without new, it throws as the browser's own does. + if (!new.target) return Real.apply(this, arguments); + const w = construct(Real, slice.call(arguments), new.target); + try { + const d = g.document; + const u = new U(String(url), d ? d.baseURI : g.location.href); + u.hash = ''; + const p = tell(u.href); + if (p && typeof p.catch === 'function') p.catch(() => {}); + } catch (e) {} + return w; + } }.SharedWorker; + try { + W.prototype = Real.prototype; + Object.defineProperty(Real.prototype, 'constructor', { value: W, writable: true, enumerable: false, configurable: true }); + Object.setPrototypeOf(W, Object.getPrototypeOf(Real)); + const d = Object.getOwnPropertyDescriptor(g, 'SharedWorker'); + Object.defineProperty(g, 'SharedWorker', { value: W, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); + } catch (e) { + // A window that takes no property: its Shared Workers are as the browser's. + } +})(); +`; +} + +/** + * The line a test shows, once for each Shared Worker script address (at, + * origin and path), when one of its pages started a Shared Worker. + */ +export function sharedWorkerLine(name: string, at: string): string { + return `keploy: in ${name}, a page started the Shared Worker at ${at}: keploy does not wait for its script or its calls before the test's browser context closes ` + + '(Playwright reports none of a Shared Worker\'s calls), so a call it still has in flight when the test ends is cut there; ' + + 'have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends.'; +} diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index 8540681..4a6b193 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -252,3 +252,13 @@ test('a call in flight across a lowercase attachment download', async ({ page }) await page.evaluate(() => { history.pushState({}, '', '/page?route'); }); await routed; }); +test('a page that starts a Shared Worker', async ({ page }) => { + await page.goto(`${base}/page`); + // The worker answers once its script has run; its URL's fragment is no + // part of the request, and its query no part of the address said. + const said = await page.evaluate(() => new Promise((resolve) => { + const w = new SharedWorker('/shared.js?v=1#f'); + w.port.onmessage = (e) => resolve(String(e.data)); + })); + if (said !== 'up') throw new Error(`the Shared Worker said ${said}`); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index f7817aa..77fd7f8 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -11,6 +11,7 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { LOADS_KEY, OFF_LINE, SEEDED_LINE, initScript, processRandom, unplacedLine, unseededLine, workerMark, workerScript } from '../dist/random.js'; +import { sharedWorkerLine } from '../dist/settle.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; const PKG = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); @@ -65,6 +66,8 @@ async function pageServer(j) { if (req.url === '/seen') { void seen.then(() => res.end('ok')); return; } // A service worker that calls /slow as it installs. if (req.url === '/sw.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end("self.addEventListener('install', () => { void fetch('/slow'); });"); return; } + // A Shared Worker that answers each page that connects. + if (req.url === '/shared.js?v=1') { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end("onconnect = (e) => { e.ports[0].postMessage('up'); };"); return; } if (req.url === '/nocontent') { res.writeHead(204); res.end(); return; } if (req.url === '/resetcontent') { res.writeHead(205); res.end(); return; } // A disposition's type is case-insensitive (RFC 6266): the browser downloads "Attachment" too. @@ -410,6 +413,26 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { }); } +// A Shared Worker's script: Chromium reports no end for it (Playwright does +// not follow Shared Workers), so settle never waits on it, in any engine, +// and the test says so, seeded or not. +for (const engine of ['chromium', 'firefox', 'webkit']) { + test(`settle, in ${engine}: a test whose page starts a Shared Worker ends at once, its browser unseeded, and says once that the worker's script and calls are not waited for`, async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = 'a page that starts a Shared Worker'; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'off', SLOW_BASE: p.url, PW_TIMEOUT: '4000', PW_BROWSER: engine }); + const t = tests[title]; + assert.equal(t.status, 'passed', out); + assert.ok(!t.steps.some((s) => s.startsWith('keploy: wait for')), t.steps.join('\n')); + const said = t.stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, `)).map((l) => l.replace(/:\d+\//, ':/')); + assert.deepEqual(said, [sharedWorkerLine(title, 'http://127.0.0.1:/shared.js')]); + } finally { await p.close(); await a.close(); } + }); +} + test('settle: a route change inside the page cancels nothing, so its call in flight is waited for', async () => { const j = journal(); const a = await fakeAgent(j); @@ -1340,16 +1363,18 @@ for (const engine of ENGINES) { const cmodel = [c1, c2].map((m) => workerAt('http://h/wk/cshared.js', workerScript(m.token, 0, 'x')).draws()); const said = r.tests.workers.stdout.split('\n').filter((l) => l.startsWith('keploy: in workers, ')).map((l) => l.replace(/:\d+\//, ':/')); const at = (p) => `http://127.0.0.1:/wk/${p}`; + // Every engine: each Shared Worker script's address, once, is not waited for. + const unwaited = ['shared.js', 'mshared.js', 'cshared.js'].map((p) => sharedWorkerLine('workers', at(p))); if (engine === 'firefox') { assert.notDeepEqual(w.shared.draws, model); assert.notDeepEqual(w.mshared.draws, mmodel); assert.notDeepEqual([w.c1.draws, w.c2.draws], cmodel); - assert.deepEqual(said.sort(), ['shared.js', 'mshared.js', 'cshared.js'].map((p) => unplacedLine('workers', at(p), 'shared')).sort()); + assert.deepEqual(said.sort(), [...['shared.js', 'mshared.js', 'cshared.js'].map((p) => unplacedLine('workers', at(p), 'shared')), ...unwaited].sort()); } else { assert.deepEqual(w.shared.draws, model); assert.deepEqual(w.mshared.draws, mmodel); assert.deepEqual([w.c1.draws, w.c2.draws], cmodel); - assert.deepEqual(said, []); + assert.deepEqual(said.sort(), [...unwaited].sort()); } // What the app's server was asked for: never keploy's mark, and each // worker's script and what it loads. diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index f7fcee9..e54a090 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -8,8 +8,9 @@ import os from 'node:os'; import path from 'node:path'; import { AgentClient, KeployError } from '../dist/agent.js'; import { decide } from '../dist/join.js'; -import { Inflight } from '../dist/settle.js'; +import { Inflight, sharedWorkerScript } from '../dist/settle.js'; import { EventEmitter } from 'node:events'; +import vm from 'node:vm'; import { testRef, outcomeOf, sourceOf, keployText, runAgain } from '../dist/index.js'; // One fake agent answering every call with answer(path, body). @@ -299,6 +300,85 @@ test('settle: a call keploy\'s route answered has ended, though the browser repo assert.equal(inflight.size, 0); }); +test('settle: a request of the frame that started a Shared Worker at its script\'s URL (fragment aside), reported before or after, is not waited for; one of another frame, another URL, or a navigation is', async () => { + const call = (frame, url, nav = false) => ({ frame: () => frame, isNavigationRequest: () => nav, method: () => 'GET', url: () => url }); + const ctx = fakeContext(); + const inflight = new Inflight(ctx); + const page = new EventEmitter(); + ctx.emit('page', page); + const f = { page: () => page }; + const g = { page: () => page }; + const before = call(f, 'http://h/s.js?q=1#a'); + const waited = [call(g, 'http://h/s.js?q=1'), call(f, 'http://h/s.js'), call(f, 'http://h/s.js?q=1', true)]; + ctx.emit('request', before); + for (const r of waited) ctx.emit('request', r); + let settled = false; + inflight.settled().then(() => { settled = true; }); + inflight.sharedWorker(f, 'http://h/s.js?q=1#b'); + await ticks(); + assert.equal(inflight.size, 3); + // A second Shared Worker of that frame, at another URL: both are kept. + inflight.sharedWorker(f, 'http://h/t.js'); + ctx.emit('request', call(f, 'http://h/s.js?q=1')); + ctx.emit('request', call(f, 'http://h/s.js?q=1#c')); + ctx.emit('request', call(f, 'http://h/t.js')); + assert.equal(inflight.size, 3); + for (const r of waited) { + assert.equal(settled, false); + ctx.emit('requestfinished', r); + await ticks(); + } + assert.equal(settled, true); + // With nothing else in flight, the frame's report wakes settle. + const ctx2 = fakeContext(); + const one = new Inflight(ctx2); + ctx2.emit('request', call(f, 'http://h/s.js')); + let woke = false; + one.settled().then(() => { woke = true; }); + one.sharedWorker(f, 'http://h/s.js'); + await ticks(); + assert.deepEqual([woke, one.size], [true, 0]); +}); + +test('sharedWorkerScript: the page\'s SharedWorker starts each worker as asked, then reports its script URL against the document\'s base with no fragment; the binding leaves the window; a worker the browser refuses throws its own error and reports nothing; new, instanceof, constructor, name and length kept; with no binding the browser\'s own stays', () => { + const made = []; + const told = []; + class Base {} + class SharedWorker extends Base { constructor(url, o) { super(); if (String(url) === 'http://[' || String(url).endsWith('/refused.js')) throw new SyntaxError('the browser\'s own'); made.push([url, o]); } } + const ctx = { URL, document: { baseURI: 'http://h/a/p' }, location: { href: 'http://h/b/p' }, __r: (u) => { told.push(u); return Promise.reject(new Error('gone')); } }; + // As a window's: not enumerable; the wrapper keeps the property's flags. + Object.defineProperty(ctx, 'SharedWorker', { value: SharedWorker, writable: true, enumerable: false, configurable: true }); + vm.createContext(ctx); + vm.runInContext(sharedWorkerScript('__r'), ctx); + assert.equal(vm.runInContext('typeof __r', ctx), 'undefined'); + const o = { name: 'n', type: 'module' }; + ctx.o = o; + vm.runInContext('globalThis.w = new SharedWorker("s.js?q=1#f", o)', ctx); + assert.deepEqual(made, [['s.js?q=1#f', o]]); + assert.equal(made[0][1], o); + assert.deepEqual(told, ['http://h/a/s.js?q=1']); + assert.equal(vm.runInContext('w instanceof SharedWorker && SharedWorker.prototype.constructor === SharedWorker && SharedWorker.name === "SharedWorker" && SharedWorker.length === 1', ctx), true); + assert.equal(vm.runInContext('const d = Object.getOwnPropertyDescriptor(globalThis, "SharedWorker"); JSON.stringify([d.enumerable, d.writable, d.configurable])', ctx), '[false,true,true]'); + assert.equal(Object.getPrototypeOf(ctx.SharedWorker), Base); + for (const u of ['http://[', '/refused.js']) assert.throws(() => vm.runInContext(`new SharedWorker(${JSON.stringify(u)})`, ctx), (e) => e.message === 'the browser\'s own', u); + assert.throws(() => vm.runInContext('SharedWorker("/s.js")', ctx), TypeError); + assert.equal(told.length, 1); + // A window with no document yet: its location. + delete ctx.document; + vm.runInContext('new SharedWorker("s.js")', ctx); + assert.deepEqual(told, ['http://h/a/s.js?q=1', 'http://h/b/s.js']); + // A report that throws: the worker is still the app's. + const c2 = { URL, SharedWorker, location: { href: 'http://h/p' }, __r: () => { throw new Error('gone'); } }; + vm.createContext(c2); + vm.runInContext(sharedWorkerScript('__r'), c2); + assert.equal(vm.runInContext('new SharedWorker("/s.js") instanceof SharedWorker', c2), true); + // No binding: the browser's own stays. + const c3 = { URL, SharedWorker, location: { href: 'http://h/p' } }; + vm.createContext(c3); + vm.runInContext(sharedWorkerScript('__r'), c3); + assert.equal(c3.SharedWorker, SharedWorker); +}); + test('settle: a cancel that comes while settle already waits wakes it (its page closes; then, for a service worker\'s call, the test closes its context)', async () => { for (const worker of [false, true]) { const c = worker ? 'a page\'s call and a service worker\'s' : 'a page\'s call'; From badc4c43fdce5fc4cb534063f8a7559f2e0822e3 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 10:40:03 +0530 Subject: [PATCH 28/73] test(playwright): row for a navigation after a Shared Worker's report Settle waits for a navigation of the frame that started a Shared Worker to its script's URL when the browser reports it after the frame's report too, as it does for one reported before. Signed-off-by: slayerjain --- playwright/test/unit.test.mjs | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index e54a090..b00f7b5 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -323,6 +323,11 @@ test('settle: a request of the frame that started a Shared Worker at its script\ ctx.emit('request', call(f, 'http://h/s.js?q=1#c')); ctx.emit('request', call(f, 'http://h/t.js')); assert.equal(inflight.size, 3); + // A navigation of that frame to that URL, reported after: waited for. + const nav = call(f, 'http://h/s.js?q=1', true); + ctx.emit('request', nav); + assert.equal(inflight.size, 4); + waited.push(nav); for (const r of waited) { assert.equal(settled, false); ctx.emit('requestfinished', r); From 1ed0f2866d36e3aae55cf483db4551325f62d789 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 10:49:54 +0530 Subject: [PATCH 29/73] fix(playwright): keep the seeding's frames off a worker's thrown errors In a seeded Web or Shared Worker, an error the browser's own call throws inside a seeded function or getter (crypto.getRandomValues of a float array or of more than 65536 bytes, crypto.randomUUID or the location's href, search or toString with a this of another kind) kept, in Chromium and WebKit, a stack made as it was thrown: with a frame of keploy's seeding, at the worker's URL with keploy's mark, or at the data: module a module Shared Worker imports its seeding as. The seeding now catches what those calls throw and leaves each frame at a URL keploy marked out of the error's own stack, through the stack's setter (V8) or as its value (WebKit), flags kept, then throws it on. A stack with no such frame, one that is no string, an error with no stack of its own (Firefox's, which its prototype's getter already filters) and a thrown value that is no object stay as they were. The data: module's media type now carries keploy's mark as a parameter, so its frames are found as the others are. A page's seeded functions let an error go on as it is: no page URL carries the mark. Rows: a unit row throws, from each seeded function and getter, errors whose stack is a setter's or a value, with frames at a query's mark, a 'p' import and a data: parameter, and checks what is left, that the flags are kept, and that the four kinds above and a page's errors are not touched; the workers browser row has each worker kind (classic and module Web Workers, classic and module Shared Workers, one served with a policy that refuses data:) throw six such errors and checks, in each engine, that no stack shows the mark or a data: frame and, in Chromium and WebKit, that each frame is the app's script's or the browser's own. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/random.ts | 52 ++++++++++++++++++++------ playwright/test/random.test.mjs | 66 ++++++++++++++++++++++++++++++++- playwright/test/rows.test.mjs | 33 ++++++++++++++--- 4 files changed, 135 insertions(+), 18 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index f779bd9..de994fe 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server never sees the mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server never sees the mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 0571e70..f68e619 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -635,7 +635,9 @@ const STREAM_JS = ` const imul = Math.imul; // A browser script's seeded Math.random, crypto.getRandomValues and // crypto.randomUUID, each drawing from from(name) ('random', 'grv', // 'uuid'): a stream, or null for the browser's own; and define, which -// keeps a property's flags. +// keeps a property's flags. What the browser's own call throws in them +// reaches their caller as thrown(e) gives it (workerScript's: with no frame +// of keploy's seeding on its stack). const DRAW_JS = ` const define = (o, name, value) => { const d = Object.getOwnPropertyDescriptor(o, name); Object.defineProperty(o, name, { value, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); @@ -659,7 +661,8 @@ const DRAW_JS = ` const define = (o, name, value) => { define(C, 'getRandomValues', { getRandomValues(array) { /* The browser's own call first: it refuses what the browser refuses (a float array, more than 65536 bytes, a this that is no Crypto). */ - const out = real.call(this, array); + let out; + try { out = real.call(this, array); } catch (e) { throw thrown(e); } const grv = from('grv'); if (grv !== null) fill(grv, new Uint8Array(out.buffer, out.byteOffset, out.byteLength)); return out; @@ -668,7 +671,8 @@ const DRAW_JS = ` const define = (o, name, value) => { if (C && typeof C.randomUUID === 'function') { const realUUID = C.randomUUID; define(C, 'randomUUID', { randomUUID() { - const own = realUUID.call(this); + let own; + try { own = realUUID.call(this); } catch (e) { throw thrown(e); } const uuid = from('uuid'); if (uuid === null) return own; const b = new Uint8Array(16); @@ -1154,6 +1158,8 @@ ${STREAM_JS} } return s.streams === false ? null : s.streams[name]; }; + // A page's stacks show no URL keploy marked: an error goes on as it is. + const thrown = (e) => e; ${DRAW_JS} const realOpen = g.open; if (typeof realOpen === 'function') { @@ -1284,7 +1290,10 @@ export function markWorker(url: string, kind: WorkerMark['kind'], token: string, * its own URL (workerAnswer). An Error's or a DOMException's stack read * through its prototype's getter (Firefox's, whose stacks go on through * the code that queued what runs) leaves out each frame at a URL keploy - * marked: keploy's own code. + * marked: keploy's own code. An error the browser's own call throws in a + * seeded function or in the location's getters (a float array, a this that + * is no Crypto or location) reaches its caller with none of the seeding's + * frames on the stack it keeps (V8's and WebKit's). */ export function workerScript(token: string, n: number, kind: 'w' | 'm' | 's' | 'x'): string { return `(() => { @@ -1293,6 +1302,25 @@ export function workerScript(token: string, n: number, kind: 'w' | 'm' | 's' | ' ${STREAM_JS} const streams = { random: stream(k + 'Math.random'), grv: stream(k + 'crypto.getRandomValues'), uuid: stream(k + 'crypto.randomUUID') }; const from = (name) => streams[name]; + // keploy's seeding runs at a URL keploy marked (a query's mark, or a + // data: module's media type parameter: workerAnswer). An error the + // browser's own call throws in a seeded function or getter keeps, in V8 + // and WebKit, a stack of its own, made as it was thrown, with the seeding's + // frames: they are left out of it before it reaches the caller, and a + // stack with none, or that is no string, stays as it was. + const marked = new RegExp(${JSON.stringify(`[?&;]${WORKER_PARAM}=`)}); + const unmark = (s) => s.split('\\n').filter((l) => !marked.test(l)).join('\\n'); + const thrown = (e) => { + try { + const d = Object.getOwnPropertyDescriptor(e, 'stack'); + const s = d.get ? d.get.call(e) : d.value; + if (typeof s === 'string' && marked.test(s)) { + if (d.get) d.set.call(e, unmark(s)); + else Object.defineProperty(e, 'stack', { value: unmark(s) }); + } + } catch (x) {} + return e; + }; ${DRAW_JS} const M = ${JSON.stringify(`${WORKER_PARAM}=${kind}.${token}`)}; const strip = (h) => { @@ -1309,27 +1337,27 @@ ${DRAW_JS} const d = Object.getOwnPropertyDescriptor(L, name); if (!d || typeof d.get !== 'function') continue; const get = d.get; + const read = (o) => { try { return get.call(o); } catch (e) { throw thrown(e); } }; const v = name === 'href' - ? { get href() { return strip(get.call(this)); } } - : { get search() { const s = get.call(this); const i = s.indexOf(M); if (i < 1) return s; const t = s.slice(0, i - 1); return t === '?' ? '' : t; } }; + ? { get href() { return strip(read(this)); } } + : { get search() { const s = read(this); const i = s.indexOf(M); if (i < 1) return s; const t = s.slice(0, i - 1); return t === '?' ? '' : t; } }; Object.defineProperty(L, name, { get: Object.getOwnPropertyDescriptor(v, name).get, set: d.set, enumerable: d.enumerable, configurable: true }); } catch (e) {} } try { const t = L.toString; - if (typeof t === 'function') define(L, 'toString', { toString() { return strip(t.call(this)); } }.toString); + if (typeof t === 'function') define(L, 'toString', { toString() { let h; try { h = t.call(this); } catch (e) { throw thrown(e); } return strip(h); } }.toString); } catch (e) {} } // Firefox reads a stack through its prototype's getter, and the stack // goes on through the code that queued what runs: keploy's answer, at a // URL keploy marked. A stack read here shows no frame at such a URL. - const marked = new RegExp(${JSON.stringify(`[?&]${WORKER_PARAM}=`)}); for (const C of [g.Error, g.DOMException]) { try { const d = Object.getOwnPropertyDescriptor(C.prototype, 'stack'); const get = d.get; if (typeof get !== 'function') continue; - const v = { get stack() { const s = get.call(this); return typeof s === 'string' ? s.split('\\n').filter((l) => !marked.test(l)).join('\\n') : s; } }; + const v = { get stack() { const s = get.call(this); return typeof s === 'string' ? unmark(s) : s; } }; Object.defineProperty(C.prototype, 'stack', { get: Object.getOwnPropertyDescriptor(v, 'stack').get, set: d.set, enumerable: d.enumerable, configurable: true }); } catch (e) {} } @@ -1378,7 +1406,9 @@ export function cspAllowsData(headers: Record): boolean { * imports its seeding (the route answers * that import, marked 'p') and then the app's script. Chromium and WebKit * let the route reach none of what a Shared Worker loads after its script, - * so a module Shared Worker imports its seeding as a data: module, which + * so a module Shared Worker imports its seeding as a data: module (its + * media type carries keploy's mark as a parameter, so a stack leaves the + * seeding's frames out: workerScript), which * runs before every module the app's script imports; where its policy * refuses data: scripts (dataOk false), its seeding runs in the answer * itself, which then imports the app's script (import()), and holds each @@ -1391,7 +1421,7 @@ export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | ' if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; if (kind === 'm') return `import ${JSON.stringify(markWorker(own, 'p', token, n))};\nimport ${url};\n`; const seeding = workerScript(token, n, 'x'); - if (dataOk) return `import ${JSON.stringify(`data:text/javascript;base64,${Buffer.from(seeding).toString('base64')}`)};\nimport ${url};\n`; + if (dataOk) return `import ${JSON.stringify(`data:text/javascript;${WORKER_PARAM}=x.${token};base64,${Buffer.from(seeding).toString('base64')}`)};\nimport ${url};\n`; return `${seeding}const held = []; const hold = (e) => { held.push(e); }; addEventListener('connect', hold); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 2e79ebb..805e8c0 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1496,6 +1496,69 @@ test('a worker\'s seeding: an Error\'s or a DOMException\'s stack read through i assert.equal(workerAt(href, workerScript(t, 0, 'w')).run('Object.getOwnPropertyDescriptor(Error.prototype, \'stack\')'), undefined); }); +test('a worker\'s seeding: an error the browser\'s own call throws in a seeded getRandomValues or randomUUID, or in the location\'s href, search or toString, reaches the caller with each frame at a URL keploy marked (a query\'s mark or a data: module\'s parameter) left out of its own stack, through its setter or as its value, flags kept; a stack with no such frame, one that is no string, none of its own and a thrown value that is no object stay as they were; a page\'s errors go on as they are', () => { + const t = 'cd'.repeat(16); + const href = markWorker('http://h/w.js', 'w', t); + const kept = ['TypeError: no', ' at f (http://h/w.js:1:2)', ` at g (http://h/w.js?a__keploy_worker=w.${t}:7:8)`, '']; + const stack = [kept[0], ` at Crypto.getRandomValues (${href}:30:5)`, kept[1], ` at x (data:text/javascript;__keploy_worker=x.${t};base64,QUJD:1:9)`, + ` at y (${markWorker('http://h/m.js', 'p', t, 0)}:3:4)`, kept[2], kept[3]].join('\n'); + // What the browser's own calls throw, in turn. + const throws = []; + const raise = () => { throw throws.shift(); }; + // A scope's own classes: its seeding wraps their prototypes' functions. + const natives = () => { + class Crypto { getRandomValues() { raise(); } randomUUID() { raise(); } } + class WorkerLocation { get href() { return raise(); } get search() { return raise(); } toString() { return raise(); } } + return { Crypto, crypto: new Crypto(), WorkerLocation, location: new WorkerLocation() }; + }; + const w = workerAt(href, workerScript(t, 0, 'w'), natives()); + const caught = (at, call) => [...at.run(`(() => { try { ${call}; } catch (e) { return [e]; } return []; })()`)]; + // An own stack read and written through a getter and a setter (V8's). + const accessor = (s0) => { + const e = new Error('no'); + let s = s0; + const set = []; + Object.defineProperty(e, 'stack', { get() { return s; }, set(v) { set.push(v); s = v; }, enumerable: false, configurable: true }); + return { e, set }; + }; + const calls = ['crypto.getRandomValues(new Uint8Array(1))', 'crypto.randomUUID()', 'location.href', 'location.search', 'String(location)']; + for (const call of calls) { + const a = accessor(stack); + throws.push(a.e); + assert.deepEqual(caught(w, call), [a.e], call); + assert.deepEqual(a.set, [kept.join('\n')], call); + const d = Object.getOwnPropertyDescriptor(a.e, 'stack'); + assert.deepEqual([typeof d.get, typeof d.set, d.enumerable], ['function', 'function', false], call); + // A stack that is the error's own value (WebKit's). + const v = new Error('no'); + Object.defineProperty(v, 'stack', { value: stack, writable: true, enumerable: false, configurable: true }); + throws.push(v); + assert.deepEqual(caught(w, call), [v], call); + assert.deepEqual(Object.getOwnPropertyDescriptor(v, 'stack'), { value: kept.join('\n'), writable: true, enumerable: false, configurable: true }, call); + } + // As they were: no frame keploy marked, a stack that is no string (never + // read as one), an error with no stack of its own (Firefox's), a value + // that is no object. + const plain = accessor(kept.join('\n')); + let read = 0; + const odd = accessor({ toString() { read++; return stack; } }); + const none = Object.create(Error.prototype); + for (const e of [plain.e, odd.e, none, 'thrown']) { + throws.push(e); + assert.deepEqual(caught(w, 'crypto.getRandomValues(new Uint8Array(1))'), [e]); + } + assert.deepEqual([plain.set, odd.set, read, Object.getOwnPropertyNames(none)], [[], [], 0, []]); + // A page's seeded functions: what the browser throws goes on as it is. + const { Crypto, crypto } = natives(); + const page = documentAt(initScript(S1), '/p', new Map(), 't0', { globals: { Crypto, crypto } }); + for (const call of calls.slice(0, 2)) { + const a = accessor(stack); + throws.push(a.e); + assert.deepEqual(caught(page, call), [a.e], call); + assert.deepEqual([a.set, a.e.stack], [[], stack], call); + } +}); + // A context whose route keeps its matcher and handler, and a route of it // that records what the handler did. function routed() { @@ -1580,7 +1643,8 @@ test('workerAnswer: a classic Web or Shared Worker runs its seeding, then import assert.equal(w.run('location.href'), 'http://h/s.js?q=1', kind); } const m = workerAnswer(t, 2, 'x', 'http://h/m.js?q=1'); - const data = /^import "data:text\/javascript;base64,([A-Za-z0-9+/=]+)";\nimport "http:\/\/h\/m\.js\?q=1";\n$/.exec(m); + // Its media type carries the mark, so a stack can leave its frames out. + const data = new RegExp(`^import "data:text/javascript;__keploy_worker=x\\.${t};base64,([A-Za-z0-9+/=]+)";\\nimport "http://h/m\\.js\\?q=1";\\n$`).exec(m); assert.ok(data, m.slice(0, 120)); assert.equal(Buffer.from(data[1], 'base64').toString(), workerScript(t, 2, 'x')); }); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 77fd7f8..cb62103 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -638,16 +638,16 @@ async function drawsServer() { // workers.spec.ts's workers, and the scripts they load. const wk = { // Each Web Worker answers the message its page posts as it starts it. - '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack) });`, + '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack), errs: ${ERRS} });`, '/wk/dep.js': 'self.dep = \'dep\';', - '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack) });`, + '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack), errs: ${ERRS} });`, '/wk/lib.mjs': 'export const dep = \'lib\';', - '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack) });`, + '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack), errs: ${ERRS} });`, // Module Shared Workers: one plain, one (started by two names, c1 // twice) served with a policy that refuses data: scripts. Each // connection after a worker's first is named by its number (c1+2). - '/wk/mshared.js': `import { dep } from './lib.mjs'; const top = String(new Error('at').stack); self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top });`, - '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; const top = String(new Error('at').stack); self.onconnect = (e) => { n++; e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url, top, stack: String(new Error('at').stack) }); };`, + '/wk/mshared.js': `import { dep } from './lib.mjs'; const top = String(new Error('at').stack); self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top, errs: ${ERRS} });`, + '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; const errs = ${ERRS}; const top = String(new Error('at').stack); self.onconnect = (e) => { n++; e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url, top, stack: String(new Error('at').stack), errs }); };`, }[u.pathname]; if (wk !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', ...(u.pathname === '/wk/cshared.js' ? { 'content-security-policy': 'script-src \'self\'' } : {}) }); @@ -1306,6 +1306,14 @@ const WK_STARTS = [ 'new SharedWorker(\'/wk/cshared.js\', { type: \'module\', name: \'c1\' })', ]; +// What each worker sends of the errors the browser's own calls throw in its +// seeded functions and its location's getters (a float array, more than +// 65536 bytes, a this that is no Crypto or location): each one's stack, or +// null for an error with none; 'none' if a call threw nothing. +const ERRS = '[() => crypto.getRandomValues(new Float64Array(1)), () => crypto.getRandomValues(new Uint8Array(65537)), () => crypto.randomUUID.call({}), ' + + '...[\'href\', \'search\'].map((k) => () => Object.getOwnPropertyDescriptor(WorkerLocation.prototype, k).get.call({})), () => WorkerLocation.prototype.toString.call({})]' + + '.map((f) => { try { f(); return \'none\'; } catch (e) { return e.stack === undefined ? null : String(e.stack); } })'; + // The marks a document at /wk of page t0 gives the workers workers.spec.ts // starts (WK_STARTS). function workerMarks(seed) { @@ -1350,6 +1358,21 @@ for (const engine of ENGINES) { assert.match(w[k][f], new RegExp(`/wk/${file}\\.js[?:]`), `${k}.${f}: ${w[k][f]}`); assert.doesNotMatch(w[k][f], /__keploy_worker/, `${k}.${f}: ${w[k][f]}`); } + // An error the browser's own call throws in a seeded function or + // getter shows no frame of keploy's seeding (at its marked URL or its + // data: module): each frame is the app's script's or the browser's own + // (Firefox, whose stacks go on through queued code: none of keploy's). + for (const [k, file] of [['ded', 'ded'], ['mod', 'mod'], ['shared', 'shared'], ['mshared', 'mshared'], ['c1', 'cshared'], ['c2', 'cshared']]) { + assert.equal(w[k].errs.length, 6, k); + for (const s of w[k].errs) { + assert.notEqual(s, 'none', k); + if (s === null) continue; + assert.doesNotMatch(s, /__keploy_worker|data:/, `${k}: ${s}`); + const frames = s.split('\n').filter((l) => (engine === 'chromium' ? /^\s+at /.test(l) : l !== '')); + assert.ok(frames.some((l) => l.includes(`/wk/${file}.js`)), `${k}: ${s}`); + if (engine !== 'firefox') for (const l of frames) assert.match(l, new RegExp(`/wk/${file}\\.js[?:]|\\[native code\\]|`), `${k}: ${s}`); + } + } // A message the page posts as it starts a Web Worker reaches the app's // handler: the app's script has run before the worker takes it. assert.deepEqual([w['ded-early'], w['mod-early']], [{ kind: 'ded-early', data: 'early' }, { kind: 'mod-early', data: 'early' }]); From 7b8321422ee09eb79c9bc62773d4050166c7ce00 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 11:36:29 +0530 Subject: [PATCH 30/73] fix(playwright): answer only a worker's script load with its seeding A worker's code resolves URLs against its script's URL, which carries keploy's mark. So fetch(''), a fragment alone or an XMLHttpRequest of '' in a seeded Web Worker reached the context's route and got keploy's seeding instead of the app's script, and counted as another start of the worker's token. The route now answers a marked URL only when the browser loads it as a worker's script: by its sec-fetch-dest (worker or sharedworker) where the route sees one (Firefox, WebKit), else by its type (script, Chromium). Any other marked request goes on to the test's other routes and the app's server with the mark taken off, and starts no instance. Rows: a unit row with each engine's request shapes, and the workers browser row, where each Web Worker's own fetch(''), fetch('#h') and XMLHttpRequest of '' get the app's script in Chromium, Firefox and WebKit. The README says where the mark still reaches the app's server: a seeded worker's Referer headers and, in Chromium and WebKit, a Shared Worker's own request at its script's URL. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/random.ts | 29 ++++++++++++++++++- playwright/test/fixtures/workers.spec.ts | 5 ++-- playwright/test/random.test.mjs | 36 +++++++++++++++++++++--- playwright/test/rows.test.mjs | 27 +++++++++++++++--- 5 files changed, 87 insertions(+), 12 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index de994fe..2cd7937 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server never sees the mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. The mark still reaches the app's server in two places: the `Referer` header of each request a seeded worker's code makes, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's own request at its script's URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index f68e619..d673f4c 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1434,6 +1434,24 @@ try { `; } +/** + * Whether the browser asks for r as a worker's script (a worker's own, or + * the seeding a module Web Worker's answer imports), not as a request the + * worker's code makes. A worker's code resolves URLs against its script's + * URL, so one that resolves to that URL (fetch(''), a fragment alone) + * carries keploy's mark too. Firefox and WebKit show the route the + * request's destination (sec-fetch-dest: worker or sharedworker for a + * worker's script, script for importScripts, empty for fetch and + * XMLHttpRequest; WebKit types a worker's script xhr). Chromium shows it + * none, and types a worker's script script, importScripts other, fetch + * fetch and XMLHttpRequest xhr. + */ +function workerScriptLoad(r: Request): boolean { + const dest = r.headers()['sec-fetch-dest']; + if (dest !== undefined) return dest === 'worker' || dest === 'sharedworker'; + return r.resourceType() === 'script'; +} + /** What the fixture hears from workerRoutes. */ export type WorkerHooks = { /** The route answered this request whole: none of it is in flight. */ @@ -1452,7 +1470,12 @@ export type WorkerHooks = { * app's and loads the app's script itself, as the browser would. Each * instance of a token (a Shared Worker started again) is the next n. Each * request it fulfills is answered: Chromium reports no end for a Shared - * Worker's script, as Playwright does not follow Shared Workers. + * Worker's script, as Playwright does not follow Shared Workers. Only the + * browser's load of a worker's script is answered so (workerScriptLoad): a + * request the worker's own code makes at its script's marked URL goes on + * to the test's other routes and the app's server with the mark taken off + * (the URL the app's code meant, as the worker's location shows it), and + * starts no instance of the token. */ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}): Promise { const counts = new Map(); @@ -1473,6 +1496,10 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await route.fallback(); return; } + if (!workerScriptLoad(route.request())) { + await route.fallback({ url: m.url }); + return; + } if (m.kind === 'p') { await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); return; diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index 3f76099..c94f5e9 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -7,12 +7,13 @@ import { test } from '../../dist'; // Worker and three module Shared Workers (each importing a module beside // it; one connected to twice); each sends what it drew, its location, what // it loaded and its stack, and each Web Worker answers the message the page -// posts as it starts it. The test keeps them. +// posts as it starts it and sends what its own code got at its script's URL. +// The test keeps them. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); test('workers', async ({ page }) => { await page.goto(`${base}/wk`); - await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 9, null, { timeout: 20000 }); + await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 11, null, { timeout: 20000 }); keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 805e8c0..1155100 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1560,18 +1560,19 @@ test('a worker\'s seeding: an error the browser\'s own call throws in a seeded g }); // A context whose route keeps its matcher and handler, and a route of it -// that records what the handler did. +// that records what the handler did. A request is a worker's script load as +// Chromium shows it (typed script, no sec-fetch-dest) unless as says. function routed() { const r = {}; const context = { route: async (match, handler) => { r.match = match; r.handler = handler; } }; - r.ask = async (url, answer) => { + r.ask = async (url, answer, as = { type: 'script', headers: {} }) => { const did = []; const route = { - request: () => ({ url: () => url }), + request: () => ({ url: () => url, resourceType: () => as.type, headers: () => as.headers }), fetch: async (o) => { did.push(['fetch', o.url]); if (answer instanceof Error) throw answer; return answer; }, fulfill: async (o) => { did.push(['fulfill', { ...o, body: String(o.body) }]); }, continue: async (o) => { did.push(['continue', o.url]); }, - fallback: async () => { did.push(['fallback']); }, + fallback: async (o) => { did.push(o === undefined ? ['fallback'] : ['fallback', o]); }, }; await r.handler(route); return did; @@ -1615,6 +1616,33 @@ test('workerRoutes answers a module worker script with an import of its seeding assert.equal(answered.length, 5); }); +test('workerRoutes answers a marked URL as a worker\'s script only when the browser loads it as one (sec-fetch-dest worker or sharedworker where the route sees it, else typed script): what the worker\'s own code asks for there (fetch(\'\'), XMLHttpRequest, importScripts) goes on with no mark and starts no instance of its token', async () => { + const { r, context } = routed(); + const answered = []; + await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); + const t = '67'.repeat(16); + const w = markWorker('http://h/w.js?a=1', 'w', t); + const as = (type, dest) => ({ type, headers: dest === undefined ? {} : { 'sec-fetch-dest': dest } }); + const own = [ + as('fetch'), as('xhr'), as('other'), // Chromium: fetch, XMLHttpRequest, importScripts + as('fetch', 'empty'), as('script', 'script'), // Firefox: fetch, importScripts + as('xhr', 'empty'), as('xhr', 'script'), // WebKit: XMLHttpRequest, importScripts + ]; + for (const a of own) assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), a), [['fallback', { url: 'http://h/w.js?a=1' }]], JSON.stringify(a)); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0), undefined, as('fetch', 'empty')), [['fallback', { url: 'http://h/m.js' }]]); + assert.deepEqual(answered, []); + // The worker's script, as each engine loads it: its first start is n 0. + const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; + assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), as('xhr', 'worker')), [['fetch', 'http://h/w.js?a=1'], + ['fulfill', { status: 200, headers, body: workerAnswer(t, 0, 'w', 'http://h/w.js?a=1') }]]); + assert.equal((await r.ask(w, answerOf(200, 'app()'), as('script', 'worker')))[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1')); + assert.equal((await r.ask(w, answerOf(200, 'app()'), as('script')))[1][1].body, workerAnswer(t, 2, 'w', 'http://h/w.js?a=1')); + const s = '89'.repeat(16); + assert.equal((await r.ask(markWorker('http://h/s.js', 's', s), answerOf(200, 'app()'), as('xhr', 'sharedworker')))[1][1].body, workerAnswer(s, 0, 's', 'http://h/s.js')); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0), undefined, as('script', 'worker')), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 0, 'm') }]]); + assert.equal(answered.length, 5); +}); + test('cspAllowsData: no policy, or each policy whose script or worker source list is named lists data:; a policy that names one without it refuses', () => { const csp = (v) => ({ 'content-security-policy': v }); assert.equal(cspAllowsData({}), true); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index cb62103..3eb7c47 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -637,10 +637,11 @@ async function drawsServer() { if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } // workers.spec.ts's workers, and the scripts they load. const wk = { - // Each Web Worker answers the message its page posts as it starts it. - '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack), errs: ${ERRS} });`, + // Each Web Worker answers the message its page posts as it starts it, + // and sends what its own code gets at its script's URL (OWN). + '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('ded')}`, '/wk/dep.js': 'self.dep = \'dep\';', - '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack), errs: ${ERRS} });`, + '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('mod')}`, '/wk/lib.mjs': 'export const dep = \'lib\';', '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack), errs: ${ERRS} });`, // Module Shared Workers: one plain, one (started by two names, c1 @@ -1314,6 +1315,13 @@ const ERRS = '[() => crypto.getRandomValues(new Float64Array(1)), () => crypto.g '...[\'href\', \'search\'].map((k) => () => Object.getOwnPropertyDescriptor(WorkerLocation.prototype, k).get.call({})), () => WorkerLocation.prototype.toString.call({})]' + '.map((f) => { try { f(); return \'none\'; } catch (e) { return e.stack === undefined ? null : String(e.stack); } })'; +// What a Web Worker's own code gets at its script's URL, which its script's +// marked URL is the base of: fetch(''), fetch('#h') and an XMLHttpRequest of +// '', each one's text (or 'error'), sent as kind -own. +const OWN = (k) => 'Promise.all([fetch(\'\'), fetch(\'#h\')].map((p) => p.then((r) => r.text(), () => \'error\')).concat(new Promise((ok) => { ' + + 'const x = new XMLHttpRequest(); x.open(\'GET\', \'\'); x.onload = () => ok(x.responseText); x.onerror = () => ok(\'error\'); x.send(); })))' + + `.then((own) => postMessage({ kind: '${k}-own', own }));`; + // The marks a document at /wk of page t0 gives the workers workers.spec.ts // starts (WK_STARTS). function workerMarks(seed) { @@ -1325,7 +1333,7 @@ function workerMarks(seed) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, and the app's server never sees keploy's mark; in Firefox each Shared Worker draws its own and is named`, async () => { + test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, a Web Worker's own requests at its script's URL getting the app's script, and the app's server never sees keploy's mark; in Firefox each Shared Worker draws its own and is named`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1376,6 +1384,17 @@ for (const engine of ENGINES) { // A message the page posts as it starts a Web Worker reaches the app's // handler: the app's script has run before the worker takes it. assert.deepEqual([w['ded-early'], w['mod-early']], [{ kind: 'ded-early', data: 'early' }, { kind: 'mod-early', data: 'early' }]); + // What a Web Worker's own code asks for at its script's URL (fetch(''), + // a fragment alone, an XMLHttpRequest of '') is the app's script, from + // the app's server (which never sees the mark, below), not keploy's + // answer for a worker's script. + for (const [k, first] of [['ded', 'importScripts(\'dep.js\');'], ['mod', 'import { dep } from \'./lib.mjs\';']]) { + assert.equal(w[`${k}-own`].own.length, 3, k); + for (const body of w[`${k}-own`].own) { + assert.ok(body.startsWith(first), `${k}: ${body.slice(0, 120)}`); + assert.doesNotMatch(body, /__keploy_worker/, k); + } + } assert.deepEqual([w.ded.dep, w.mod.dep, w.mshared.dep, w.c1.dep, w.c2.dep, w['c1+2'].dep], ['dep', 'lib', 'lib', 'lib', 'lib', 'lib']); // c1's second connection is c1's worker's: what it drew as it loaded. assert.deepEqual(w['c1+2'].draws, w.c1.draws); From 55f1b9c29c92ac02c0b41ea56098dd8a9f8bf9d9 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 13:03:18 +0530 Subject: [PATCH 31/73] fix(playwright): keep keploy's mark off a seeded worker's fetch A seeded worker's code resolves URLs against its script's URL, which carries keploy's mark, and the browser names that URL as the referrer of the worker's requests. So the app's server saw the mark in the Referer of every fetch a seeded worker made, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, in the URL of a Shared Worker's fetch(''). The worker's seeding now wraps fetch and Request: a fetch or a Request at the script's own URL (an empty URL, a fragment alone) asks for it with no mark, and a fetch whose referrer the browser would pick (no init, an init that names none, a Request made with no referrer and the default policy) names the script's URL with no mark. Any other URL, a referrer the caller gives, a Request with a referrer or policy of its own, and a worker with no mark go as before. What the browser's fetch or Request refuses reaches the caller with none of the seeding's frames on its stack. JavaScript cannot set the Referer of an XMLHttpRequest, of importScripts or of a module worker's import of the app's script, so the server still sees the mark there, and in the URL of a Shared Worker's XMLHttpRequest of its own script in Chromium and WebKit. The README says so: the mark comes from the test's seed, so it is the same at record and replay. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/random.ts | 42 +++++++++++-- playwright/test/fixtures/workers.spec.ts | 3 +- playwright/test/random.test.mjs | 76 ++++++++++++++++++++++++ playwright/test/rows.test.mjs | 38 +++++++++--- 5 files changed, 147 insertions(+), 14 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 2cd7937..49fb569 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. The mark still reaches the app's server in two places: the `Referer` header of each request a seeded worker's code makes, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's own request at its script's URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index d673f4c..0466692 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1287,7 +1287,8 @@ export function markWorker(url: string, kind: WorkerMark['kind'], token: string, * instance of the token (a Shared Worker started again after its documents * closed is the next). Its location (href, search, toString) does not show * the mark kind.token the worker's URL carries; the app's script runs at - * its own URL (workerAnswer). An Error's or a DOMException's stack read + * its own URL (workerAnswer), and its fetch and Request take the mark off + * the URLs and referrers they would send. An Error's or a DOMException's stack read * through its prototype's getter (Firefox's, whose stacks go on through * the code that queued what runs) leaves out each frame at a URL keploy * marked: keploy's own code. An error the browser's own call throws in a @@ -1330,6 +1331,36 @@ ${DRAW_JS} for (const c of ['?', '&']) if (b.endsWith(c + M)) return b.slice(0, b.length - M.length - 1) + (i < 0 ? '' : s.slice(i)); return s; }; + // The worker's code resolves a URL against its script's URL, keploy's + // mark and all, and the browser names that URL as the referrer of the + // worker's requests. A fetch or a Request at the script's own URL (an + // empty URL, a fragment alone) asks for it with no mark, and a fetch whose + // referrer the browser would pick (none given, by the init or by a + // Request made with the default policy) names the script's URL with no + // mark. What the browser's own fetch or Request refuses reaches the + // caller as thrown(e) gives it. + let at = ''; + try { at = String(g.location.href); } catch (e) {} + const own = strip(at); + const F = g.fetch; + const Q = g.Request; + if (own !== at && typeof F === 'function' && typeof Q === 'function') { + const U = g.URL; + const unmarked = (u) => { + if (typeof u !== 'string') return u; + try { const h = new U(u, at).href; const s = strip(h); return s === h ? u : s; } catch (e) { return u; } + }; + const named = (o) => (o === undefined || o === null ? { referrer: own } + : typeof o === 'object' || typeof o === 'function' ? new Proxy(o, { get: (t, k) => { const v = Reflect.get(t, k); return k === 'referrer' && v === undefined ? own : v; } }) : o); + const R = new Proxy(Q, { construct: (t, a, n) => { if (a.length > 0) a[0] = unmarked(a[0]); try { return Reflect.construct(t, a, n); } catch (e) { throw thrown(e); } } }); + define(g, 'Request', R); + define(Q.prototype, 'constructor', R); + define(g, 'fetch', { fetch(input, ...more) { + const pick = !(input instanceof Q) || (input.referrer === 'about:client' && input.referrerPolicy === ''); + const args = [unmarked(input), ...(pick ? [named(more[0]), ...more.slice(1)] : more)]; + return F.apply(this, args).then(undefined, (e) => { throw thrown(e); }); + } }.fetch); + } const L = typeof WorkerLocation === 'function' ? WorkerLocation.prototype : null; if (L) { for (const name of ['href', 'search']) { @@ -1472,10 +1503,11 @@ export type WorkerHooks = { * request it fulfills is answered: Chromium reports no end for a Shared * Worker's script, as Playwright does not follow Shared Workers. Only the * browser's load of a worker's script is answered so (workerScriptLoad): a - * request the worker's own code makes at its script's marked URL goes on - * to the test's other routes and the app's server with the mark taken off - * (the URL the app's code meant, as the worker's location shows it), and - * starts no instance of the token. + * request the worker's own code makes at its script's marked URL (an + * XMLHttpRequest's: its fetch and Request ask for no mark, workerScript) + * goes on to the test's other routes and the app's server with the mark + * taken off (the URL the app's code meant, as the worker's location shows + * it), and starts no instance of the token. */ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}): Promise { const counts = new Map(); diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index c94f5e9..b0530e3 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -8,12 +8,13 @@ import { test } from '../../dist'; // it; one connected to twice); each sends what it drew, its location, what // it loaded and its stack, and each Web Worker answers the message the page // posts as it starts it and sends what its own code got at its script's URL. +// Each worker sends what its fetch and Request sent. // The test keeps them. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); test('workers', async ({ page }) => { await page.goto(`${base}/wk`); - await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 11, null, { timeout: 20000 }); + await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 17, null, { timeout: 20000 }); keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 1155100..0270e27 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1763,3 +1763,79 @@ test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module await r.ask(`http://h/m.js?__keploy_worker=s.${t}.3`); assert.equal(answered.length, 5); }); + +test('a worker\'s seeding: its fetch and Request ask for its script\'s own URL with no mark, and a fetch whose referrer the browser would pick names that URL with no mark; another URL, a given referrer, a Request with a referrer or policy of its own, what is no string and a worker with no mark are left as they are; what they refuse reaches the caller with no frame at a marked URL', async () => { + const t = 'cd'.repeat(16); + const plain = 'http://h/w.js?a=1'; + const href = markWorker(plain, 'w', t); + // A Request that keeps what it was made with, and a fetch that keeps what + // it was given; each refuses 'bad' with an error whose stack has a frame + // at a marked URL. + const marked = (m) => { const e = {}; Object.defineProperty(e, 'stack', { value: `E: ${m}\n at f (${href}:1:2)\n at g (http://h/a.js:3:4)`, writable: true, configurable: true }); return e; }; + const given = []; + const fakes = () => ({ + URL, + Request: class Request { + constructor(u, init) { + if (u === 'bad') throw marked('request'); + this.n = arguments.length; + this.u = u; + this.referrer = init && init.referrer !== undefined ? init.referrer : 'about:client'; + this.referrerPolicy = init && init.referrerPolicy !== undefined ? init.referrerPolicy : ''; + } + }, + fetch: function fetch(...a) { given.push({ self: this, a }); return a[0] === 'bad' ? Promise.reject(marked('fetch')) : Promise.resolve('ok'); }, + }); + const w = workerAt(href, workerScript(t, 0, 'w'), fakes()); + const last = async (code) => { given.length = 0; assert.equal(await w.run(code), 'ok', code); assert.equal(given.length, 1, code); return given[0].a; }; + const ref = (a) => [a.length, a[0], a[1] && a[1].referrer]; + // At the script's own URL (an empty URL, a fragment alone): no mark, and + // the browser's referrer named with no mark. + assert.deepEqual(ref(await last('fetch(\'\')')), [2, plain, plain]); + assert.deepEqual(ref(await last('fetch(\'#f\', null)')), [2, `${plain}#f`, plain]); + // Another URL, one that is no URL, and what is no string go as given. + assert.deepEqual(ref(await last('fetch(\'x.js\')')), [2, 'x.js', plain]); + assert.deepEqual(ref(await last('fetch(\'http://[\')')), [2, 'http://[', plain]); + assert.equal((await last('globalThis.u = { toString() { return \'\'; } }; fetch(u)'))[0], w.run('u')); + // An init: its own referrer kept, its other members read from it whole + // (a getter's this is the init), what follows it kept; an init that is a + // function is one too; an init that is no object goes as given. + assert.deepEqual(ref(await last('fetch(\'\', { referrer: \'http://h/r\' })')), [2, plain, 'http://h/r']); + const m = await last('globalThis.o = { get method() { return this === o ? \'POST\' : \'other\'; } }; fetch(\'a\', o, \'more\')'); + assert.deepEqual([m.length, m[1].method, m[1].body, m[1].referrer, m[2]], [3, 'POST', undefined, plain, 'more']); + const f = await last('fetch(\'a\', Object.assign(function () {}, { method: \'PATCH\' }))'); + assert.deepEqual([f[1].method, f[1].referrer], ['PATCH', plain]); + assert.deepEqual(await last('fetch(\'a\', 5)'), ['a', 5]); + // fetch is called on what the caller called it on. + await last('globalThis.th = {}; fetch.call(th, \'a\')'); + assert.equal(given[0].self, w.run('th')); + // A Request: made at the script's own URL with no mark; given to fetch, + // its referrer named with no mark only when it left it to the browser + // with the default policy. + assert.deepEqual(['new Request(\'\').u', 'new Request(\'#f\').u', 'new Request(\'x.js\').u', 'new Request().n'].map((c) => w.run(c)), [plain, `${plain}#f`, 'x.js', 0]); + assert.equal(w.run('new Request(\'\') instanceof Request && Request.prototype.constructor === Request && Request.name'), 'Request'); + const r = await last('globalThis.r = new Request(\'a\'); fetch(r)'); + assert.deepEqual([r.length, r[0] === w.run('r'), r[1].referrer], [2, true, plain]); + assert.deepEqual([(await last('fetch(new Request(\'a\'), { method: \'PUT\' })'))[1].method, given[0].a[1].referrer], ['PUT', plain]); + for (const init of ['{ referrer: \'http://h/r\' }', '{ referrerPolicy: \'no-referrer\' }', '{ referrer: \'about:client\', referrerPolicy: \'origin\' }']) { + assert.equal((await last(`fetch(new Request('a', ${init}))`)).length, 1, init); + } + // What they refuse: no frame at a marked URL on its stack. + await assert.rejects(w.run('fetch(\'bad\')'), (e) => e.stack === 'E: fetch\n at g (http://h/a.js:3:4)'); + assert.equal(w.run('try { new Request(\'bad\'); } catch (e) { e.stack }'), 'E: request\n at g (http://h/a.js:3:4)'); + // A worker with no mark, or whose location cannot be read, keeps the + // browser's own fetch and Request; one with no fetch, or no Request, + // gets none. + const plainFakes = fakes(); + const p = workerAt(plain, workerScript(t, 0, 'w'), plainFakes); + assert.equal(p.run('fetch') === plainFakes.fetch && p.run('Request') === plainFakes.Request, true); + const qFakes = fakes(); + const q = workerAt(href, '', qFakes); + q.run('Object.defineProperty(WorkerLocation.prototype, \'href\', { get() { throw new Error(\'no\'); }, configurable: true });'); + q.run(workerScript(t, 0, 'w')); + assert.equal(q.run('fetch') === qFakes.fetch && q.run('Request') === qFakes.Request, true); + const { fetch: noFetch, ...rest } = fakes(); + const { Request: noRequest, ...rest2 } = fakes(); + assert.deepEqual([workerAt(href, workerScript(t, 0, 'w'), rest).run('typeof fetch + typeof Request'), workerAt(href, workerScript(t, 0, 'w'), rest2).run('typeof fetch + typeof Request')], + [`undefined${typeof noFetch}`, `${typeof noRequest}undefined`]); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 3eb7c47..81c3211 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -635,20 +635,22 @@ async function drawsServer() { // parse.spec.ts's removed: /rhold.js is answered at /rrelease. if (u.pathname === '/rhold.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); rheld.push(res); return; } if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } - // workers.spec.ts's workers, and the scripts they load. + // workers.spec.ts's workers, and the scripts they load; /wk/ref answers + // the Referer it was sent. + if (u.pathname === '/wk/ref') { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(req.headers.referer ?? 'none'); return; } const wk = { // Each Web Worker answers the message its page posts as it starts it, // and sends what its own code gets at its script's URL (OWN). - '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('ded')}`, + '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('ded')} ${REFS}.then((v) => postMessage({ kind: 'ded-ref', ...v }));`, '/wk/dep.js': 'self.dep = \'dep\';', - '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('mod')}`, + '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('mod')} ${REFS}.then((v) => postMessage({ kind: 'mod-ref', ...v }));`, '/wk/lib.mjs': 'export const dep = \'lib\';', - '/wk/shared.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack), errs: ${ERRS} });`, + '/wk/shared.js': `const ref = ${REFS}; onconnect = (e) => { e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack), errs: ${ERRS} }); ref.then((v) => e.ports[0].postMessage({ kind: 'shared-ref', ...v })); };`, // Module Shared Workers: one plain, one (started by two names, c1 // twice) served with a policy that refuses data: scripts. Each // connection after a worker's first is named by its number (c1+2). - '/wk/mshared.js': `import { dep } from './lib.mjs'; const top = String(new Error('at').stack); self.onconnect = (e) => e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top, errs: ${ERRS} });`, - '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; const errs = ${ERRS}; const top = String(new Error('at').stack); self.onconnect = (e) => { n++; e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url, top, stack: String(new Error('at').stack), errs }); };`, + '/wk/mshared.js': `import { dep } from './lib.mjs'; const top = String(new Error('at').stack); const ref = ${REFS}; self.onconnect = (e) => { e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top, errs: ${ERRS} }); ref.then((v) => e.ports[0].postMessage({ kind: 'mshared-ref', ...v })); };`, + '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; const errs = ${ERRS}; const top = String(new Error('at').stack); const ref = ${REFS}; self.onconnect = (e) => { n++; if (n === 1) ref.then((v) => e.ports[0].postMessage({ kind: self.name + '-ref', ...v })); e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url, top, stack: String(new Error('at').stack), errs }); };`, }[u.pathname]; if (wk !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', ...(u.pathname === '/wk/cshared.js' ? { 'content-security-policy': 'script-src \'self\'' } : {}) }); @@ -1322,6 +1324,15 @@ const OWN = (k) => 'Promise.all([fetch(\'\'), fetch(\'#h\')].map((p) => p.then(( 'const x = new XMLHttpRequest(); x.open(\'GET\', \'\'); x.onload = () => ok(x.responseText); x.onerror = () => ok(\'error\'); x.send(); })))' + `.then((own) => postMessage({ kind: '${k}-own', own }));`; +// What a worker's fetch and Request send, which /wk/ref answers with the +// request's Referer (or 'none'): a fetch with no init, one with an init, one +// of a Request made with no referrer, and one of a Request made with a +// policy of its own (no-referrer); the URL fetch('') was answered at; and the +// URLs of a Request at '' and at a fragment alone. +const REFS = 'Promise.all([fetch(\'ref\'), fetch(\'ref\', { cache: \'no-store\' }), fetch(new Request(\'ref\')), fetch(new Request(\'ref\', { referrerPolicy: \'no-referrer\' }))]' + + '.map((p) => p.then((r) => r.text(), () => \'error\')).concat(fetch(\'\').then((r) => r.url, () => \'error\')))' + + '.then((ref) => ({ ref, req: [new Request(\'\').url, new Request(\'#h\').url] }))'; + // The marks a document at /wk of page t0 gives the workers workers.spec.ts // starts (WK_STARTS). function workerMarks(seed) { @@ -1333,7 +1344,7 @@ function workerMarks(seed) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, a Web Worker's own requests at its script's URL getting the app's script, and the app's server never sees keploy's mark; in Firefox each Shared Worker draws its own and is named`, async () => { + test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, a Web Worker's own requests at its script's URL getting the app's script, each worker's fetch and Request asking for its script's own URL and naming it as the Referer with no mark, and the app's server never sees keploy's mark in a URL; in Firefox each Shared Worker draws its own and is named`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1395,6 +1406,19 @@ for (const engine of ENGINES) { assert.doesNotMatch(body, /__keploy_worker/, k); } } + // What each worker's fetch and Request send (REFS): its script's URL + // with no mark as the Referer the browser picks (no init, an init with + // no referrer, a Request made with no referrer), none for a Request + // with a policy of its own that sends none, and fetch('') and a Request + // at '' or a fragment alone at its script's URL with no mark. + const origin = new URL(w.ded.href).origin; + for (const [k, file] of [['ded', 'ded.js?x=1'], ['mod', 'mod.js'], ['shared', 'shared.js'], ['mshared', 'mshared.js'], ['c1', 'cshared.js'], ['c2', 'cshared.js']]) { + const url = `${origin}/wk/${file}`; + const v = w[`${k}-ref`]; + assert.deepEqual(v.ref, [url, url, url, 'none', url], k); + assert.equal(v.req[0], url, k); + assert.ok([url, `${url}#h`].includes(v.req[1]), `${k}: ${v.req[1]}`); + } assert.deepEqual([w.ded.dep, w.mod.dep, w.mshared.dep, w.c1.dep, w.c2.dep, w['c1+2'].dep], ['dep', 'lib', 'lib', 'lib', 'lib', 'lib']); // c1's second connection is c1's worker's: what it drew as it loaded. assert.deepEqual(w['c1+2'].draws, w.c1.draws); From 6a1ccc08048564f8b8a282c7bf29c32eb6ead44e Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 14:32:19 +0530 Subject: [PATCH 32/73] fix(playwright): keep keploy's worker route ahead of the test's own A test's own route ran before keploy's context route: Playwright runs a page's routes before its context's, and the last added first. A suite with a catch-all page.route('**/*', r => r.continue()) took a worker's marked script request itself. The worker then ran unseeded at its marked URL, the app's server got the mark, nothing said so, and a seeded Chromium test whose page started a Shared Worker hung until its timeout: settle waited on the marked request, which Chromium never ends. keploy's route now stays first on the context and on each page: it is added again ahead of every route the test adds (route, routeFromHAR) and after unrouteAll. It hands a worker's script load on to the test's routes at the URL the app asked for, with the route's continue answering it with the seeding; a route that falls back, or none, leaves it to keploy's last route on the context, which answers it so. A route that answers the script itself runs it unseeded, and the test names that worker once with the next step. Settle never waits on a request whose URL carries keploy's Shared Worker mark, whichever route answers it. Rows: the workers row runs again behind catch-all routes on the context and the page in each engine (seeded, the same at replay, no mark at the server), with a worker whose script the test's route answers; unit rows for the route order and for settle. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/index.ts | 15 ++- playwright/src/random.ts | 160 +++++++++++++++++++---- playwright/src/settle.ts | 14 +- playwright/test/fixtures/workers.spec.ts | 26 +++- playwright/test/random.test.mjs | 103 +++++++++++++-- playwright/test/rows.test.mjs | 36 +++-- playwright/test/unit.test.mjs | 18 +++ 8 files changed, 320 insertions(+), 54 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 49fb569..81c7767 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end does not wait on a script keploy answered. So a seeded test's requests go through that route (in Chromium the browser's cache is off while a context has one, as for any `context.route`), and a route or `waitForRequest` of the test's own sees a worker's script request with the mark: one that matches the URL whole needs a pattern that allows it (`**/worker.js*`). These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 7464536..631fc39 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -218,8 +218,19 @@ const keployTestType = base.extend({ // document; the page's own scripts do not know it. const mark = freshSeed(); // The workers' scripts: Firefox's route does not reach a Shared - // Worker's, so there it is not marked, and says so. - await workerRoutes(context, { answered: (r) => inflight.answered(r) }); + // Worker's, so there it is not marked, and says so. A worker whose + // script a route of the test's own answered says so too, once for each + // address. + await workerRoutes(context, { + answered: (r) => inflight.answered(r), + routed: (at) => { + const a = pageAddress(at); + const k = `routed\n${a}`; + if (told.has(k)) return; + told.add(k); + process.stdout.write(`${unplacedLine(name, a, 'routed')}\n`); + }, + }); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox') }); const placing = placePages(context, mark); restore = placing.restore; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 0466692..fa28db6 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -17,7 +17,7 @@ // On by default; KEPLOY_SEED_RANDOM=off or the fixture option // keploySeedRandom: false turns it off, and the worker says what that costs. -import type { BrowserContext, Frame, Page, Request } from '@playwright/test'; +import type { BrowserContext, Frame, Page, Request, Route } from '@playwright/test'; import crypto from 'node:crypto'; import fs from 'node:fs'; import { createRequire } from 'node:module'; @@ -96,19 +96,23 @@ export function pageAddress(at: unknown): string { * another origin that has no session storage to count its documents in, or * a frame inside one, 'nostorage' for a top document with no session * storage to count its documents in (other than a page's first document), - * or a frame of its origin inside one. + * or a frame of its origin inside one, 'routed' for a worker whose script + * a route of the test's own answered itself (workerRoutes). */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed'; // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared', string> = { +const UNREACHED: Record<'worker' | 'shared' | 'routed', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + 'record and replay the test in Chromium or WebKit to seed it.', + routed: 'a route of the test\'s own answered the worker\'s script itself (route.fulfill), so keploy cannot put its seeding ahead of the script, ' + + 'and the worker\'s URL, which its location shows, carries keploy\'s __keploy_worker mark; have that route call route.continue() or route.fallback() ' + + 'for the worker\'s script to seed it.', }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -130,8 +134,8 @@ const UNPLACED: Record, str * (othersNow). */ export function unplacedLine(name: string, at: string, why: Unplaced = 'popup', others = 0): string { - if (why === 'worker' || why === 'shared') { - return `keploy: in ${name}, the ${why === 'worker' ? 'worker' : 'Shared Worker'} at ${at} draws the browser's own random numbers, so its draws may differ at replay: ${UNREACHED[why]}`; + if (why === 'worker' || why === 'shared' || why === 'routed') { + return `keploy: in ${name}, the ${why === 'shared' ? 'Shared Worker' : 'worker'} at ${at} draws the browser's own random numbers, so its draws may differ at replay: ${UNREACHED[why]}`; } const n = Math.floor(Number(others)); const reason = why === 'uncounted' @@ -1487,8 +1491,52 @@ function workerScriptLoad(r: Request): boolean { export type WorkerHooks = { /** The route answered this request whole: none of it is in flight. */ answered?: (r: Request) => void; + /** + * A route of the test's own answered the script of the worker at at (its + * origin and path) itself: it runs unseeded, at its marked URL. + */ + routed?: (at: string) => void; }; +/** What keploy's routes go on: the test's context, or one of its pages. */ +type Routes = Pick; + +/** + * Keeps handler at url ahead of every route the test adds to target after + * it (route, and routeFromHAR, which adds its own through route): each time + * one is added, keploy's is added again, ahead of it, before the one it + * replaces is taken off, so no request finds neither. After the test's + * unrouteAll, again comes first, then keploy's. Playwright runs a page's + * routes before its context's, and the last added first. + */ +function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Route) => Promise, again: () => Promise = async () => {}): Promise { + const route = target.route; + const unrouteAll = target.unrouteAll; + let now: ((route: Route) => Promise) | null = null; + const add = async (): Promise => { + const h = (r: Route): Promise => handler(r); + const was = now; + now = h; + await route.call(target, url, h); + if (was !== null) await target.unroute(url, was); + }; + target.route = async function (this: Routes, ...args: Parameters) { + const added = route.apply(this, args); + const ahead = add(); + try { + return await added; + } finally { + await ahead.catch(() => {}); + } + } as Routes['route']; + target.unrouteAll = async function (this: Routes, ...args: Parameters) { + await unrouteAll.apply(this, args); + await again(); + await add(); + } as Routes['unrouteAll']; + return add(); +} + /** * Answers each marked worker script the test's context asks for with * workerAnswer, under the status and headers the app's server answered for @@ -1505,9 +1553,19 @@ export type WorkerHooks = { * browser's load of a worker's script is answered so (workerScriptLoad): a * request the worker's own code makes at its script's marked URL (an * XMLHttpRequest's: its fetch and Request ask for no mark, workerScript) - * goes on to the test's other routes and the app's server with the mark - * taken off (the URL the app's code meant, as the worker's location shows - * it), and starts no instance of the token. + * goes on to the test's routes and the app's server with the mark taken off + * (the URL the app's code meant, as the worker's location shows it), and + * starts no instance of the token. + * + * keploy's route comes first, ahead of every route the test adds to its + * context or to one of its pages (keepFirst), so no route of the test's sees + * the mark: it hands a worker's script load on to the test's routes at the + * URL the app asked for, with the route's continue answering it as above (a + * continued script is the app's server's, with keploy's seeding ahead of + * it). A route of the test's that answers it itself (fulfill) answers it so, + * unseeded, and the test hears of it (routed); one that aborts it aborts + * it. One that falls back, or none, leaves it to keploy's last route, the + * context's first, which answers it as above. */ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}): Promise { const counts = new Map(); @@ -1518,20 +1576,15 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = for (const [k, v] of Object.entries(h)) if (k !== 'content-encoding' && k !== 'content-length') out[k] = v; return out; }; - await context.route((u) => u.searchParams.has(WORKER_PARAM), async (route) => { - const fulfill = async (o: Parameters[0]): Promise => { - await route.fulfill(o); + // The worker script loads handed on to the test's routes: each one's mark, + // and the route's own fulfill; and their URLs, for the last route. + const handed = new WeakMap(); + const urls = new Set(); + const answer = async (route: Route, m: WorkerMark, own: Route['fulfill']): Promise => { + const fulfill = async (o: Parameters[0]): Promise => { + await own(o); hooks.answered?.(route.request()); }; - const m = workerMark(route.request().url()); - if (m === null) { - await route.fallback(); - return; - } - if (!workerScriptLoad(route.request())) { - await route.fallback({ url: m.url }); - return; - } if (m.kind === 'p') { await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); return; @@ -1541,16 +1594,65 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = let status = 200; let headers: Record = { 'content-type': 'text/javascript' }; try { - const answer = await route.fetch({ url: m.url }); - if (!answer.ok()) { - await fulfill({ status: answer.status(), headers: headersOf(answer.headers()), body: await answer.body() }); + const got = await route.fetch({ url: m.url }); + if (!got.ok()) { + await fulfill({ status: got.status(), headers: headersOf(got.headers()), body: await got.body() }); return; } - status = answer.status(); - headers = headersOf(answer.headers()); + status = got.status(); + headers = headersOf(got.headers()); } catch { // The worker loads the app's script itself, and fails as it would. } - await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind, m.url, cspAllowsData(headers)) }); - }); + await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers)) }); + }; + const first = async (route: Route): Promise => { + const r = route.request(); + const m = workerMark(r.url()); + if (m === null) { + await route.fallback(); + return; + } + if (!workerScriptLoad(r)) { + await route.fallback({ url: m.url }); + return; + } + const own = route.fulfill.bind(route); + if (m.kind === 'p') { + await answer(route, m, own); + return; + } + handed.set(r, { m, fulfill: own }); + urls.add(new URL(m.url).href); + route.continue = async () => answer(route, m, own); + route.fulfill = async (o) => { + const u = new URL(m.url); + hooks.routed?.(u.origin + u.pathname); + await own(o); + }; + await route.fallback({ url: m.url }); + }; + const last = async (route: Route): Promise => { + const h = handed.get(route.request()); + if (h === undefined) { + await route.fallback(); + return; + } + await answer(route, h.m, h.fulfill); + }; + const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM); + const lastUrl = (u: URL): boolean => urls.has(u.href); + const route = context.route; + const addLast = async (): Promise => { + await route.call(context, lastUrl, last); + }; + await addLast(); + await keepFirst(context, marked, first, addLast); + const onPage = (p: Page): void => { + keepFirst(p, marked, first).catch(() => { + // A page that closed as it opened: none of its requests is left. + }); + }; + context.pages().forEach(onPage); + context.on('page', onPage); } diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index d1985ac..3410f25 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -5,6 +5,7 @@ // browser's own request events; no time window decides it. import type { BrowserContext, Frame, Page, Request } from '@playwright/test'; +import { workerMark } from './random'; export class Inflight { // Each call's number, in the order the browser reported it. @@ -24,7 +25,7 @@ export class Inflight { // A service worker's call has no frame. try { frame = r.frame(); } catch { frame = null; } const nav = r.isNavigationRequest(); - if (!nav && this.shared.get(frame)?.has(noFragment(r.url()))) return; + if (!nav && (this.shared.get(frame)?.has(noFragment(r.url())) || sharedScript(r.url()))) return; this.open.set(r, { frame, nav, n: ++this.reported }); }); context.on('requestfinished', (r) => this.done(r)); @@ -154,6 +155,17 @@ function pageOf(f: Frame): Page | null { try { return f.page(); } catch { return null; } } +/** + * Whether url carries keploy's mark for a Shared Worker's script (workerMark: + * kind s or x). Settle never waits on such a request, whichever route + * answers it: Chromium reports no end for it, as Playwright does not follow + * Shared Workers. + */ +function sharedScript(url: string): boolean { + const k = workerMark(url)?.kind; + return k === 's' || k === 'x'; +} + function noFragment(url: string): string { const i = url.indexOf('#'); return i < 0 ? url : url.slice(0, i); diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index b0530e3..f069679 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -1,4 +1,5 @@ import fs from 'node:fs'; +import type { Page } from '@playwright/test'; import { test } from '../../dist'; // Run in Chromium, Firefox and WebKit (PW_BROWSER). The page at /wk starts @@ -9,12 +10,31 @@ import { test } from '../../dist'; // it loaded and its stack, and each Web Worker answers the message the page // posts as it starts it and sends what its own code got at its script's URL. // Each worker sends what its fetch and Request sent. -// The test keeps them. +// The test keeps them, under its title. const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); -test('workers', async ({ page }) => { +const workers = async (page: Page): Promise => { await page.goto(`${base}/wk`); await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 17, null, { timeout: 20000 }); - keep('workers', await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); + keep(test.info().title, await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w)); +}; + +test('workers', async ({ page }) => { + await workers(page); +}); + +// The same, behind the test's own catch-all routes, on its context and on +// its page, each continuing every request (as a suite that logs requests, +// or adds a header, has). Then a route of the test's own answers the +// script of a Web Worker the page starts twice at one path itself: each +// sends its location. +test('workers behind routes', async ({ page }) => { + await page.context().route('**/*', (route) => route.continue()); + await page.route('**/*', (route) => route.continue()); + await page.route('**/wk/own.js', (route) => route.fulfill({ contentType: 'text/javascript', body: 'postMessage(location.href);' })); + await workers(page); + keep('own', await page.evaluate(() => Promise.all([0, 1].map(() => new Promise((resolve) => { + new Worker('/wk/own.js').onmessage = (e) => resolve(e.data); + }))))); }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 0270e27..6f08006 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1562,19 +1562,60 @@ test('a worker\'s seeding: an error the browser\'s own call throws in a seeded g // A context whose route keeps its matcher and handler, and a route of it // that records what the handler did. A request is a worker's script load as // Chromium shows it (typed script, no sec-fetch-dest) unless as says. +// A fake browser context for workerRoutes, in Playwright's order: a page's +// routes before its context's, the last added first; a route that falls +// back hands the request on, at the URL its fallback gave, to the next +// route that matches it there. What answered it is what ask returns: the +// fetches and the fulfill, continue or abort that ended it, or, when no +// route did, the fallback it went on to the server with. function routed() { const r = {}; - const context = { route: async (match, handler) => { r.match = match; r.handler = handler; } }; - r.ask = async (url, answer, as = { type: 'script', headers: {} }) => { + const target = () => { + const t = { routes: [] }; + t.route = async (match, handler) => { t.routes.unshift({ match, handler }); }; + t.unroute = async (match, handler) => { t.routes = t.routes.filter((x) => !(x.match === match && (handler === undefined || x.handler === handler))); }; + t.unrouteAll = async () => { t.routes = []; }; + return t; + }; + const context = target(); + const pages = []; + const listeners = []; + context.pages = () => pages.slice(); + context.on = (event, f) => { if (event === 'page') listeners.push(f); }; + r.page = () => { + const p = target(); + pages.push(p); + for (const f of listeners) f(p); + return p; + }; + // keploy's first route's match. + r.match = (u) => context.routes[0].match(u); + r.ask = async (url, answer, as = { type: 'script', headers: {} }, page = null) => { const did = []; + let at = url; + let over; + let ended = false; + const request = { url: () => at, resourceType: () => as.type, headers: () => as.headers }; const route = { - request: () => ({ url: () => url, resourceType: () => as.type, headers: () => as.headers }), + request: () => request, fetch: async (o) => { did.push(['fetch', o.url]); if (answer instanceof Error) throw answer; return answer; }, - fulfill: async (o) => { did.push(['fulfill', { ...o, body: String(o.body) }]); }, - continue: async (o) => { did.push(['continue', o.url]); }, - fallback: async (o) => { did.push(o === undefined ? ['fallback'] : ['fallback', o]); }, + fulfill: async (o) => { ended = true; did.push(['fulfill', { ...o, body: String(o.body) }]); }, + continue: async (o) => { ended = true; did.push(['continue', o?.url ?? at]); }, + abort: async () => { ended = true; did.push(['abort']); }, + fallback: async (o) => { + if (o === undefined || Object.keys(o).length === 0) return; + over = { ...over, ...o }; + if (o.url !== undefined) at = o.url; + }, }; - await r.handler(route); + for (const t of page === null ? [context] : [page, context]) { + for (const x of t.routes.slice()) { + if (ended) break; + if (!t.routes.includes(x) || !x.match(new URL(at))) continue; + await x.handler(route, request); + } + } + if (!ended) did.push(over === undefined ? ['fallback'] : ['fallback', over]); return did; }; return { r, context }; @@ -1740,6 +1781,51 @@ test('workerAnswer: a module Shared Worker whose app script fails to load fails ch.port1.close(); ch.port2.close(); }); +test('workerRoutes keeps keploy\'s route ahead of every route the test adds to its context or its pages, after unrouteAll too: the test\'s route sees a worker\'s script load at the URL the app asked for; continuing it has keploy answer it with its seeding, answering it itself answers it unseeded and is heard, aborting it aborts it, falling back leaves it to keploy; what the worker\'s own code asks for at its marked URL reaches it with no mark; a module worker\'s seeding never reaches it', async () => { + const { r, context } = routed(); + const before = r.page(); + const heard = []; + const answered = []; + await workerRoutes(context, { answered: (q) => answered.push(q.url()), routed: (at) => heard.push(at) }); + const page = r.page(); + const t = 'ab'.repeat(16); + const w = markWorker('http://h/w.js?a=1', 'w', t); + const seen = []; + const pass = async (route, q) => { seen.push(q.url()); await route.continue(); }; + await page.route(() => true, pass); + const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; + const seeded = (n) => [['fetch', 'http://h/w.js?a=1'], ['fulfill', { status: 200, headers, body: workerAnswer(t, n, 'w', 'http://h/w.js?a=1') }]]; + assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), undefined, page), seeded(0)); + assert.deepEqual(seen, ['http://h/w.js?a=1']); + assert.deepEqual(answered, ['http://h/w.js?a=1']); + // The worker's own XMLHttpRequest at its marked URL, and a module + // worker's seeding, which keploy answers before any route of the test's. + assert.deepEqual(await r.ask(w, undefined, { type: 'xhr', headers: { 'sec-fetch-dest': 'empty' } }, page), [['continue', 'http://h/w.js?a=1']]); + assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0), undefined, undefined, page), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 0, 'm') }]]); + assert.deepEqual(seen, ['http://h/w.js?a=1', 'http://h/w.js?a=1']); + // A page that was open before: the test's route answers the script itself. + await before.route(() => true, (route) => route.fulfill({ status: 200, body: 'mine()' })); + assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), undefined, before), [['fulfill', { status: 200, body: 'mine()' }]]); + assert.deepEqual(heard, ['http://h/w.js']); + // The context, for a request of no page: aborted, then left to keploy. + await context.route(() => true, (route) => route.abort()); + assert.deepEqual(await r.ask(w, answerOf(200, 'app()')), [['abort']]); + await context.unrouteAll(); + await context.route(() => true, (route) => route.fallback()); + assert.deepEqual(await r.ask(w, answerOf(200, 'app()')), seeded(1)); + // After the page's unrouteAll, keploy's route is ahead of the next again. + await page.unrouteAll(); + await page.route(() => true, pass); + assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), undefined, page), seeded(2)); + assert.deepEqual(heard, ['http://h/w.js']); + // One route of keploy's ahead on each, the one it replaced taken off; + // the context's last route stays last. + for (const x of [context, page, before]) assert.equal(x.routes.filter((y) => y.match === x.routes[0].match).length, 1); + assert.equal(context.routes.length, 3); + assert.equal(context.routes[2].match(new URL('http://h/w.js?a=1')), true); + assert.equal(context.routes[2].match(new URL('http://h/v.js')), false); +}); + test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module one whose policy refuses data: with its seeding in the answer; each request it fulfills is answered', async () => { const { r, context } = routed(); const answered = []; @@ -1758,7 +1844,8 @@ test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module // A classic one is not refused by a policy: it imports the app's script. const d0 = markWorker('http://h/d.js', 's', t.replace('6', 'a')); assert.equal((await r.ask(d0, answerOf(200, 'app()', policy)))[1][1].body, workerAnswer(t.replace('6', 'a'), 0, 's', 'http://h/d.js')); - assert.deepEqual(answered, [s0, s0, x0, x1, d0]); + // Each at the URL the app asked for: keploy's first route handed it on so. + assert.deepEqual(answered, [s0, s0, x0, x1, d0].map((u) => workerMark(u).url)); // Not answered: what falls back. await r.ask(`http://h/m.js?__keploy_worker=s.${t}.3`); assert.equal(answered.length, 5); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 81c3211..f3ea60b 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1343,19 +1343,23 @@ function workerMarks(seed) { return made; } +// workers.spec.ts's tests that start the workers at /wk. +const WK_TITLES = ['workers', 'workers behind routes']; + for (const engine of ENGINES) { - test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, a Web Worker's own requests at its script's URL getting the app's script, each worker's fetch and Request asking for its script's own URL and naming it as the Referer with no mark, and the app's server never sees keploy's mark in a URL; in Firefox each Shared Worker draws its own and is named`, async () => { + test(`seeded, in ${engine}: a page's classic and module Web Workers and its classic and module Shared Workers draw from the test's seed, the same at replay, a module Shared Worker whose policy refuses data: too (each connection that came before its script ran given to it, in order), at the app's location, URLs and stack, loading what sits beside them, a Web Worker's own requests at its script's URL getting the app's script, each worker's fetch and Request asking for its script's own URL and naming it as the Referer with no mark, and the app's server never sees keploy's mark in a URL, behind the test's own catch-all routes on its context and page too; in Firefox each Shared Worker draws its own and is named; a worker whose script the test's own route answers runs it as given, unseeded, and is named once`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); - const byKind = (r) => Object.fromEntries(r.drew.workers.map((w) => [w.kind, w])); + const byKind = (r, title) => Object.fromEntries(r.drew[title].map((w) => [w.kind, w])); try { - const r = await drawRun(j, env(rec.url), ['workers.spec.ts'], ['workers']); - const w = byKind(r); - const [ded, mod, shared, mshared, c1, c2, c1again] = workerMarks(r.seeds.workers); + const r = await drawRun(j, env(rec.url), ['workers.spec.ts'], [...WK_TITLES, 'own']); + for (const title of WK_TITLES) { + const w = byKind(r, title); + const [ded, mod, shared, mshared, c1, c2, c1again] = workerMarks(r.seeds[title]); assert.deepEqual([ded, mod, shared, mshared, c1, c2, c1again].map((m) => m.kind), ['w', 'm', 's', 'x', 'x', 'x', 'x']); assert.notEqual(c1.token, c2.token); assert.equal(c1again.token, c1.token); @@ -1427,28 +1431,39 @@ for (const engine of ENGINES) { const model = workerAt('http://h/wk/shared.js', workerScript(shared.token, 0, 's')).draws(); const mmodel = workerAt('http://h/wk/mshared.js', workerScript(mshared.token, 0, 'x')).draws(); const cmodel = [c1, c2].map((m) => workerAt('http://h/wk/cshared.js', workerScript(m.token, 0, 'x')).draws()); - const said = r.tests.workers.stdout.split('\n').filter((l) => l.startsWith('keploy: in workers, ')).map((l) => l.replace(/:\d+\//, ':/')); + const said = r.tests[title].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, `)).map((l) => l.replace(/:\d+\//, ':/')); const at = (p) => `http://127.0.0.1:/wk/${p}`; // Every engine: each Shared Worker script's address, once, is not waited for. - const unwaited = ['shared.js', 'mshared.js', 'cshared.js'].map((p) => sharedWorkerLine('workers', at(p))); + // Behind the test's routes: the worker whose script its own route + // answered, once for its address (it started two there). + const unwaited = ['shared.js', 'mshared.js', 'cshared.js'].map((p) => sharedWorkerLine(title, at(p))) + .concat(title === 'workers' ? [] : [unplacedLine(title, at('own.js'), 'routed')]); if (engine === 'firefox') { assert.notDeepEqual(w.shared.draws, model); assert.notDeepEqual(w.mshared.draws, mmodel); assert.notDeepEqual([w.c1.draws, w.c2.draws], cmodel); - assert.deepEqual(said.sort(), [...['shared.js', 'mshared.js', 'cshared.js'].map((p) => unplacedLine('workers', at(p), 'shared')), ...unwaited].sort()); + assert.deepEqual(said.sort(), [...['shared.js', 'mshared.js', 'cshared.js'].map((p) => unplacedLine(title, at(p), 'shared')), ...unwaited].sort()); } else { assert.deepEqual(w.shared.draws, model); assert.deepEqual(w.mshared.draws, mmodel); assert.deepEqual([w.c1.draws, w.c2.draws], cmodel); assert.deepEqual(said.sort(), [...unwaited].sort()); } + } + // The worker whose script the test's own route answered runs that + // script, at its marked URL: each start of it is the next. + assert.equal(r.drew.own.length, 2); + for (const h of r.drew.own) assert.match(h, /^http:\/\/127\.0\.0\.1:\d+\/wk\/own\.js\?__keploy_worker=w\.[0-9a-f]{32}$/); + assert.notEqual(r.drew.own[0], r.drew.own[1]); // What the app's server was asked for: never keploy's mark, and each // worker's script and what it loads. assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); for (const u of ['/wk/ded.js?x=1', '/wk/dep.js', '/wk/mod.js', '/wk/lib.mjs', '/wk/shared.js', '/wk/mshared.js', '/wk/cshared.js']) assert.ok(r.seen.includes(u), `${u}: ${r.seen.join(' ')}`); kept = r.seeds; - const q = await drawRun(j, env(rep.url), ['workers.spec.ts'], ['workers']); - const p = byKind(q); + const q = await drawRun(j, env(rep.url), ['workers.spec.ts'], [...WK_TITLES, 'own']); + for (const title of WK_TITLES) { + const w = byKind(r, title); + const p = byKind(q, title); assert.deepEqual([p.ded.draws, p.mod.draws], [w.ded.draws, w.mod.draws]); for (const k of ['shared', 'mshared']) { if (engine === 'firefox') assert.notDeepEqual(p[k].draws, w[k].draws, k); @@ -1458,6 +1473,7 @@ for (const engine of ENGINES) { if (engine === 'firefox') assert.notDeepEqual(p[k].draws, w[k].draws, k); else assert.deepEqual(p[k].draws, w[k].draws, k); } + } assert.deepEqual(q.seen.filter((u) => u.includes('__keploy_worker')), []); } finally { await rep.close(); await rec.close(); } }); diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index b00f7b5..eb8644f 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -9,6 +9,7 @@ import path from 'node:path'; import { AgentClient, KeployError } from '../dist/agent.js'; import { decide } from '../dist/join.js'; import { Inflight, sharedWorkerScript } from '../dist/settle.js'; +import { markWorker } from '../dist/random.js'; import { EventEmitter } from 'node:events'; import vm from 'node:vm'; import { testRef, outcomeOf, sourceOf, keployText, runAgain } from '../dist/index.js'; @@ -300,6 +301,23 @@ test('settle: a call keploy\'s route answered has ended, though the browser repo assert.equal(inflight.size, 0); }); +test('settle: a request whose URL carries keploy\'s mark for a Shared Worker\'s script, classic or module, is not waited for, whichever route answers it (a test\'s own that continues it: no answered); a Web Worker\'s, its seeding\'s, a navigation and a malformed mark are', async () => { + const ctx = fakeContext(); + const inflight = new Inflight(ctx); + const page = new EventEmitter(); + ctx.emit('page', page); + const f = { page: () => page }; + const call = (url, nav = false) => ({ frame: () => f, isNavigationRequest: () => nav, method: () => 'GET', url: () => url }); + const t = 'cd'.repeat(16); + ctx.emit('request', call(markWorker('http://h/s.js', 's', t))); + ctx.emit('request', call(markWorker('http://h/x.js?q=1#a', 'x', t))); + assert.equal(inflight.size, 0); + const waited = [markWorker('http://h/w.js', 'w', t), markWorker('http://h/m.js', 'm', t), markWorker('http://h/m.js', 'p', t, 0), 'http://h/s.js?__keploy_worker=s.zz'] + .map((u) => call(u)).concat(call(markWorker('http://h/s.js', 's', t), true)); + for (const r of waited) ctx.emit('request', r); + assert.equal(inflight.size, 5); +}); + test('settle: a request of the frame that started a Shared Worker at its script\'s URL (fragment aside), reported before or after, is not waited for; one of another frame, another URL, or a navigation is', async () => { const call = (frame, url, nav = false) => ({ frame: () => frame, isNavigationRequest: () => nav, method: () => 'GET', url: () => url }); const ctx = fakeContext(); From beb079bc29ea15c5ff8bc8435dfd4cd201d16041 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 14:43:51 +0530 Subject: [PATCH 33/73] test(playwright): keploy routes return after the context unrouteAll A row for a test that calls its context's unrouteAll and adds no route after it: keploy's first route is back, so a worker's script load of no page is still answered with the seeding. Signed-off-by: slayerjain --- playwright/test/random.test.mjs | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 6f08006..e897426 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1810,13 +1810,16 @@ test('workerRoutes keeps keploy\'s route ahead of every route the test adds to i // The context, for a request of no page: aborted, then left to keploy. await context.route(() => true, (route) => route.abort()); assert.deepEqual(await r.ask(w, answerOf(200, 'app()')), [['abort']]); + // After the context's unrouteAll, with no route added after it, keploy's + // routes are back: its first answers a request of no page. await context.unrouteAll(); - await context.route(() => true, (route) => route.fallback()); assert.deepEqual(await r.ask(w, answerOf(200, 'app()')), seeded(1)); + await context.route(() => true, (route) => route.fallback()); + assert.deepEqual(await r.ask(w, answerOf(200, 'app()')), seeded(2)); // After the page's unrouteAll, keploy's route is ahead of the next again. await page.unrouteAll(); await page.route(() => true, pass); - assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), undefined, page), seeded(2)); + assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), undefined, page), seeded(3)); assert.deepEqual(heard, ['http://h/w.js']); // One route of keploy's ahead on each, the one it replaced taken off; // the context's last route stays last. From c02cbffb8dffe2ef2b39be2eea08ccaef780bc8e Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 15:27:29 +0530 Subject: [PATCH 34/73] test(playwright): the faker line is said once for each reason A row for a worker whose seeded tests import a faker keploy cannot seed: two folders, one whose faker exports no instance and one whose faker throws as it loads, hold four tests in three files that one worker runs. The worker says each reason's line once, and both reasons are said. The rows' run helper takes a config so a row can run test files it writes. Signed-off-by: slayerjain --- playwright/test/rows.test.mjs | 59 ++++++++++++++++++++++++++++++++--- 1 file changed, 54 insertions(+), 5 deletions(-) diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index f3ea60b..428d3a1 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -10,7 +10,7 @@ import http from 'node:http'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; -import { LOADS_KEY, OFF_LINE, SEEDED_LINE, initScript, processRandom, unplacedLine, unseededLine, workerMark, workerScript } from '../dist/random.js'; +import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, workerMark, workerScript } from '../dist/random.js'; import { sharedWorkerLine } from '../dist/settle.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; @@ -105,15 +105,15 @@ async function pageServer(j) { return { url: `http://127.0.0.1:${srv.address().port}`, close: () => new Promise((r) => { srv.closeAllConnections(); srv.close(r); }) }; } -// Runs one Playwright run of the fixture specs, from the folder cwd; -// returns each test's result. -async function playwright(specs, env, cwd = PKG) { +// Runs one Playwright run of the fixture specs (or of config's), from the +// folder cwd; returns each test's result. +async function playwright(specs, env, cwd = PKG, config = CONFIG) { const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'kpw-')); const report = path.join(tmp, 'report.json'); const stepsFile = path.join(tmp, 'steps.jsonl'); const childEnv = { ...process.env, PLAYWRIGHT_JSON_OUTPUT_NAME: report, PW_OUTPUT_DIR: path.join(tmp, 'out'), MARKER_DIR: tmp, STEPS_FILE: stepsFile, ...env }; for (const k of Object.keys(childEnv)) if (childEnv[k] === undefined) delete childEnv[k]; - const child = spawn(process.execPath, [CLI, 'test', '-c', CONFIG, ...specs], { cwd, env: childEnv, stdio: ['ignore', 'pipe', 'pipe'] }); + const child = spawn(process.execPath, [CLI, 'test', '-c', config, ...specs], { cwd, env: childEnv, stdio: ['ignore', 'pipe', 'pipe'] }); let out = ''; child.stdout.on('data', (d) => { out += d; }); child.stderr.on('data', (d) => { out += d; }); @@ -1676,6 +1676,55 @@ test('seeded: faker\'s instances in the test process make from the test\'s seed } finally { await rep.close(); await rec.close(); } }); +test('a worker says once for each reason that it cannot seed faker, however many seeded tests and test files that reason meets', async () => { + // Two folders whose tests import a faker keploy cannot seed: one that + // exports no instance, one that throws as it loads. The first folder's + // two files hold three tests; the second's one file, one. One worker + // runs them all, seeded, and a fifth test that imports a faker keploy + // seeds. + fs.mkdirSync(path.join(PKG, 'test-results'), { recursive: true }); + const dir = fs.mkdtempSync(path.join(PKG, 'test-results', 'faker-said-')); + const j = journal(); + const a = await fakeAgent(j); + try { + const fakerIn = (sub, main) => { + const d = path.join(dir, sub, 'node_modules', '@faker-js', 'faker'); + fs.mkdirSync(d, { recursive: true }); + fs.writeFileSync(path.join(d, 'package.json'), JSON.stringify({ name: '@faker-js/faker', main: 'm.cjs' })); + fs.writeFileSync(path.join(d, 'm.cjs'), main); + }; + fakerIn('none', 'exports.x = 1;\n'); + fakerIn('throws', 'throw new Error(\'this faker cannot load\');\n'); + const spec = (sub, file, names) => fs.writeFileSync(path.join(dir, sub, file), + `import { test } from ${JSON.stringify(path.join(PKG, 'dist'))};\n` + + names.map((n) => `test(${JSON.stringify(n)}, () => {});\n`).join('')); + spec('none', 'one.spec.ts', ['none one a', 'none one b']); + spec('none', 'two.spec.ts', ['none two']); + spec('throws', 'three.spec.ts', ['throws three']); + // A folder whose tests import the package's own faker, which keploy + // seeds: no line. + fs.mkdirSync(path.join(dir, 'seeds')); + spec('seeds', 'four.spec.ts', ['seeds four']); + const config = path.join(dir, 'playwright.config.mjs'); + fs.writeFileSync(config, 'export default { testDir: \'.\', workers: 1, retries: 0, reporter: [[\'json\']], ' + + `outputDir: ${JSON.stringify(path.join(dir, 'out'))}, projects: [{ name: 'p1' }] };\n`); + const { tests, out } = await playwright([], keployEnv(a.url), PKG, config); + assert.deepEqual(Object.keys(tests).sort(), ['none one a', 'none one b', 'none two', 'seeds four', 'throws three'], out); + for (const [k, t] of Object.entries(tests)) assert.equal(t.status, 'passed', `${k}: ${t.error}\n${out}`); + const seeds = calls(j).filter((c) => c.path === '/agent/scope/begin').map((c) => c.body.seed); + assert.equal(seeds.length, 5); + for (const sd of seeds) assert.match(sd, /^[0-9a-f]{32}$/); + const said = Object.values(tests).map((t) => t.stdout).join('').split('\n').filter((l) => l.includes('cannot seed it')).sort(); + assert.deepEqual(said, [ + fakerLine('it exports no instance keploy knows how to seed (faker 8, 9 and 10 do)'), + fakerLine('this faker cannot load'), + ].sort(), out); + } finally { + await a.close(); + fs.rmSync(dir, { recursive: true, force: true }); + } +}); + test('the test process draws its own Math.random when keploy answers no seed, each test saying so, or with KEPLOY_SEED_RANDOM=off, the worker saying once what that costs', async () => { const j = journal(); const none = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); From f27d04f1f176cbe990ba867e41486d70af403234 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 16:39:05 +0530 Subject: [PATCH 35/73] fix(playwright): name Service Workers and the workers they may answer A Service Worker draws the browser's own random numbers: keploy cannot put its seeding ahead of its script. A seeded test now says so for each Service Worker the browser registers, once for each script address, with the next step (serviceWorkers: 'block' when the tests do not need it). A registration Playwright blocks says nothing. A worker whose script a Service Worker may answer would show that Service Worker keploy's __keploy_worker mark in the script's URL. Such a worker now starts as the app asked, unseeded, and the test names it with the Service Worker: a Web Worker of a controlled page, and a Shared Worker of a page that is controlled, registered one, or found one registered for its origin as it began. The worker still takes its place at its path, so the next worker there draws what it drew at record. Rows: the unit rows for the register wrap, the controlled and registered cases and the lines; a browser row in each engine for a page whose Service Worker controls it, and the same page with serviceWorkers: 'block' (seeded, the same at replay). Signed-off-by: slayerjain --- playwright/README.md | 4 +- playwright/src/index.ts | 6 +- playwright/src/random.ts | 92 +++++++++++-- playwright/test/draws.mjs | 2 +- .../test/fixtures/service-blocked.spec.ts | 10 ++ playwright/test/fixtures/service-helper.ts | 29 +++++ playwright/test/fixtures/service.spec.ts | 7 + playwright/test/random.test.mjs | 123 +++++++++++++++++- playwright/test/rows.test.mjs | 83 ++++++++++++ 9 files changed, 339 insertions(+), 17 deletions(-) create mode 100644 playwright/test/fixtures/service-blocked.spec.ts create mode 100644 playwright/test/fixtures/service-helper.ts create mode 100644 playwright/test/fixtures/service.spec.ts diff --git a/playwright/README.md b/playwright/README.md index 81c7767..2e1f6fa 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. Service Workers draw the browser's own numbers. The first test a worker seeds prints one line saying so, and how to turn it off. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page that a Service Worker controls, that registered one, or for whose origin one was registered when the page began. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -95,7 +95,7 @@ Any answer from the agent that this package cannot trust fails the test. That co - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: - a faker instance the suite makes itself with `new Faker(...)` as a test file loads and does not give to `seedFaker` (see above): it seeds its own generator from the worker's `Math.random` then; - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - - Service Workers, and a worker that a worker starts; + - a Service Worker, and a worker that a Service Worker may answer (each says so as the test runs, above); a worker that a worker starts; - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; - `crypto.subtle`'s keys. diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 631fc39..bc1dc9d 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -182,14 +182,16 @@ const keployTestType = base.extend({ let opened: (page: Page | null | undefined) => Promise = async () => false; await context.exposeBinding(report, async (source, at: unknown, why: unknown, above: unknown) => { const a = pageAddress(at); - const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' || why === 'worker' || why === 'shared' ? why : (await opened(source.page)) ? 'own' : 'popup'; + const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' || why === 'worker' || why === 'shared' || why === 'service' || why === 'controlled' + ? why : (await opened(source.page)) ? 'own' : 'popup'; // Once for each address and reason: two documents at one address // (about:blank) may have no place for two reasons. const k = `${w}\n${a}`; if (told.has(k)) return; told.add(k); if (w !== 'uncounted') { - process.stdout.write(`${unplacedLine(name, a, w)}\n`); + // A worker a Service Worker may answer: that Service Worker's address. + process.stdout.write(`${unplacedLine(name, a, w, 0, pageAddress(above))}\n`); return; } // Its line says how many frames are beside its frame once their diff --git a/playwright/src/random.ts b/playwright/src/random.ts index fa28db6..512cd1d 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -52,7 +52,7 @@ export const SEEDED_LINE = 'keploy: each test\'s random numbers are seeded from 'and in the test process Math.random and the instances ' + `${FAKER} makes as it loads (faker, fakerDE, ...), from the test's begin to its end ` + '(its beforeEach and afterEach hooks too). ' + - 'Service Workers keep the browser\'s own, and so do a page the app opens and a worker that keploy cannot place or reach (each one is named); ' + + 'A Service Worker, a page the app opens and a worker that keploy cannot place or reach keep the browser\'s own (each one is named); ' + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + 'Own Faker instances, those the suite makes itself as a file loads (new Faker(...)), draw their own: seed them with seedFaker(instance, name) ' + @@ -97,12 +97,17 @@ export function pageAddress(at: unknown): string { * a frame inside one, 'nostorage' for a top document with no session * storage to count its documents in (other than a page's first document), * or a frame of its origin inside one, 'routed' for a worker whose script - * a route of the test's own answered itself (workerRoutes). + * a route of the test's own answered itself (workerRoutes), 'service' for a + * Service Worker a document registered, 'controlled' for a worker whose + * script a Service Worker may answer (initScript). */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'service' | 'controlled'; + +// The next step for a Service Worker the tests do not need. +const BLOCK = 'if the tests do not need it, set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered)'; // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared' | 'routed', string> = { +const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + @@ -110,9 +115,11 @@ const UNREACHED: Record<'worker' | 'shared' | 'routed', string> = { routed: 'a route of the test\'s own answered the worker\'s script itself (route.fulfill), so keploy cannot put its seeding ahead of the script, ' + 'and the worker\'s URL, which its location shows, carries keploy\'s __keploy_worker mark; have that route call route.continue() or route.fallback() ' + 'for the worker\'s script to seed it.', + service: 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + + `${BLOCK}.`, }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -131,11 +138,18 @@ const UNPLACED: Record hash(s).map((w) => (w + 4294967296).toString(16).slice(1)).join(''); const addressOf = (u) => (u.protocol === 'http:' || u.protocol === 'https:' ? u.origin + u.pathname : u.protocol); + // Service Workers. keploy cannot put its seeding ahead of a Service + // Worker's script, so each registration the browser makes says so; one + // Playwright blocks (serviceWorkers: 'block' answers register with no + // registration) says nothing. A worker's script request reaches a Service + // Worker, which would see keploy's mark: a Web Worker's through its + // document's controller, a Shared Worker's through the Service Worker + // whose scope holds its URL, which keploy takes to be the controller, one + // this document registered or one registered for its origin as it began. + // Such a worker starts as the app asked and says so (servedBy). A Service + // Worker is named by its script's address ('' when keploy cannot read it): + // the controller; else the one this document registered last; else, of a + // registration found as it began, the one that answers (active), else the + // one next. + let registered = null; + const swAddress = (w) => { try { return addressOf(new U(String(w.scriptURL))); } catch (e) { return ''; } }; + try { + const proto = g.ServiceWorkerContainer.prototype; + const realRegister = proto.register; + if (typeof realRegister === 'function') { + define(proto, 'register', { register(url) { + const p = realRegister.apply(this, arguments); + let at = ''; + try { const d = docNow(); at = addressOf(new U(String(url), d ? d.baseURI : location.href)); } catch (e) { at = ''; } + try { + p.then((r) => { + if (!r) return; + registered = at; + if (tell !== null) { const t = tell(at, 'service', 0); if (t && typeof t.catch === 'function') t.catch(() => {}); } + }, () => {}); + } catch (e) { + // A register that returned no promise: nothing to hear. + } + return p; + } }.register); + } + } catch (e) { + // No Service Workers here (an insecure context), or a prototype that + // takes no property: its registrations say nothing. + } + try { + g.navigator.serviceWorker.getRegistrations().then((rs) => { + if (registered === null && rs.length > 0) registered = swAddress(rs[0].active || rs[0].waiting || rs[0].installing); + }, () => {}); + } catch (e) { + // No Service Worker here (an opaque origin): none can answer. + } + const servedBy = (name) => { + let c = null; + try { c = g.navigator.serviceWorker.controller; } catch (e) { c = null; } + if (c) return swAddress(c); + return name === 'SharedWorker' ? registered : null; + }; const wrap = (name, tokenOf) => { const Real = g[name]; if (typeof Real !== 'function') return; @@ -1209,7 +1275,11 @@ ${DRAW_JS} const module = o !== null && typeof o === 'object' && o.type === 'module'; let at = ''; try { at = addressOf(u); } catch (e) { at = '?'; } - const why = u.protocol !== 'http:' && u.protocol !== 'https:' ? 'worker' : tokenOf(u, o); + // Its token is taken whoever answers it: the next worker's place does + // not hang on when a Service Worker came. + let why = u.protocol !== 'http:' && u.protocol !== 'https:' ? 'worker' : tokenOf(u, o); + const by = why !== null && why.length === 32 ? servedBy(name) : null; + if (by !== null) why = 'controlled'; if (why !== null && why.length === 32) { const h = u.href; const i = h.indexOf('#'); @@ -1221,7 +1291,7 @@ ${DRAW_JS} } const w = construct(Real, args, new.target); if (why !== '' && tell !== null) { - try { const r = tell(at, why === 'shared' ? 'shared' : 'worker', 0); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} + try { const r = tell(at, why === 'shared' || why === 'controlled' ? why : 'worker', why === 'controlled' ? by : 0); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} } return w; } }[name]; diff --git a/playwright/test/draws.mjs b/playwright/test/draws.mjs index 7ec969c..aa32711 100644 --- a/playwright/test/draws.mjs +++ b/playwright/test/draws.mjs @@ -47,7 +47,7 @@ export function documentAt(script, path = '/p', storage = new Map(), page = 't0' } else { ctx.sessionStorage = { getItem: (k) => (storage.has(k) ? storage.get(k) : null), setItem: (k, v) => { storage.set(k, String(v)); } }; } - if (report) ctx[report] = (at, why, above) => { told.push(why === 'uncounted' ? [at, why, above] : why ? [at, why] : at); return Promise.resolve(); }; + if (report) ctx[report] = (at, why, above) => { told.push(why === 'uncounted' || why === 'controlled' ? [at, why, above] : why ? [at, why] : at); return Promise.resolve(); }; if (element !== undefined) Object.defineProperty(ctx, 'frameElement', { get() { return element; }, configurable: true, enumerable: true }); if (doc !== undefined) { ctx.document = doc; ctx.URL = URL; } if (opener !== undefined) ctx.opener = opener; diff --git a/playwright/test/fixtures/service-blocked.spec.ts b/playwright/test/fixtures/service-blocked.spec.ts new file mode 100644 index 0000000..62ad72d --- /dev/null +++ b/playwright/test/fixtures/service-blocked.spec.ts @@ -0,0 +1,10 @@ +import { test } from '../../dist'; +import { serviceRun } from './service-helper'; + +// The page at /sw (service-helper.ts), with Playwright blocking every +// Service Worker. +test.use({ serviceWorkers: 'block' }); + +test('service worker blocked', async ({ page }) => { + await serviceRun(page); +}); diff --git a/playwright/test/fixtures/service-helper.ts b/playwright/test/fixtures/service-helper.ts new file mode 100644 index 0000000..df6c4e4 --- /dev/null +++ b/playwright/test/fixtures/service-helper.ts @@ -0,0 +1,29 @@ +import fs from 'node:fs'; +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// service.spec.ts's and service-blocked.spec.ts's run, in Chromium, Firefox +// and WebKit (PW_BROWSER). The page at /sw registers the Service Worker +// /sw.js, waits until it controls the page (when the browser registered +// it), and starts the Web Worker /swk/w.js and the Shared Worker /swk/s.js; +// each sends what it drew and its location. The Service Worker then sends +// the URL of each request it was asked. The test keeps them, under its title. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +type Took = { __w: { kind: string }[] }; + +export async function serviceRun(page: Page): Promise { + await page.goto(`${base}/sw`); + await page.waitForFunction(() => (window as unknown as Took).__w.filter((m) => m.kind === 'sw-ded' || m.kind === 'sw-shared').length >= 2, null, { timeout: 20000 }); + const seen = await page.evaluate(() => { + const c = navigator.serviceWorker; + if (!c || !c.controller) return null; + const asked = c.controller; + return new Promise((ok) => { + c.addEventListener('message', (e) => ok(e.data), { once: true }); + asked.postMessage('seen'); + }); + }); + keep(test.info().title, { w: await page.evaluate(() => (window as unknown as Took).__w), seen }); +} diff --git a/playwright/test/fixtures/service.spec.ts b/playwright/test/fixtures/service.spec.ts new file mode 100644 index 0000000..3fcb539 --- /dev/null +++ b/playwright/test/fixtures/service.spec.ts @@ -0,0 +1,7 @@ +import { test } from '../../dist'; +import { serviceRun } from './service-helper'; + +// The page at /sw with its Service Worker (service-helper.ts). +test('service worker', async ({ page }) => { + await serviceRun(page); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index e897426..f50f628 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -37,7 +37,7 @@ test('the lines say what is seeded, what is not, where the seed lives, and how t 'and in the test process Math.random and the instances ' + '@faker-js/faker makes as it loads (faker, fakerDE, ...), from the test\'s begin to its end ' + '(its beforeEach and afterEach hooks too). ' + - 'Service Workers keep the browser\'s own, and so do a page the app opens and a worker that keploy cannot place or reach (each one is named); ' + + 'A Service Worker, a page the app opens and a worker that keploy cannot place or reach keep the browser\'s own (each one is named); ' + 'the test process keeps its own crypto, and its own Math.random and faker outside every test (beforeAll, afterAll, a file as it loads) ' + 'and while a worker fixture registered with this package\'s test.extend or test.use is set up. ' + 'Own Faker instances, those the suite makes itself as a file loads (new Faker(...)), draw their own: seed them with seedFaker(instance, name) ' + @@ -1441,6 +1441,16 @@ test('a Shared Worker\'s token is the same in every document of the test, by scr assert.deepEqual([off, told], [[[null, undefined]], [['http://h/s.js', 'shared']]]); }); +test('the lines for a Service Worker and a worker one may answer say what it is, why, and the next step', () => { + const block = 'if the tests do not need it, set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered).'; + assert.equal(unplacedLine('t', 'http://h/sw.js', 'service'), 'keploy: in t, the Service Worker at http://h/sw.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + block); + assert.equal(unplacedLine('t', 'http://h/w.js', 'controlled', 0, 'http://h/sw.js'), 'keploy: in t, the worker at http://h/w.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + 'a Service Worker (at http://h/sw.js) may answer the worker\'s script, and it would see keploy\'s __keploy_worker mark in the script\'s URL, so keploy starts the worker as the app asked; ' + block); + assert.equal(unplacedLine('t', 'http://h/w.js', 'controlled'), 'keploy: in t, the worker at http://h/w.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + 'a Service Worker may answer the worker\'s script, and it would see keploy\'s __keploy_worker mark in the script\'s URL, so keploy starts the worker as the app asked; ' + block); +}); + test('the lines for a worker keploy cannot reach say what it is, why, and the next step', () => { assert.equal(unplacedLine('t', 'blob:', 'worker'), 'keploy: in t, the worker at blob: draws the browser\'s own random numbers, so its draws may differ at replay: ' + 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + @@ -1450,6 +1460,117 @@ test('the lines for a worker keploy cannot reach say what it is, why, and the ne 'record and replay the test in Chromium or WebKit to seed it.'); }); +// A document's ServiceWorkerContainer: register answers what answer gives +// (a registration of its script, by default), getRegistrations the +// registrations regs, and the controller is state.controller. +const swGlobals = ({ answer = (u) => Promise.resolve({ active: { scriptURL: `http://h${u}` } }), regs = [], late = false, state = { controller: null } } = {}) => { + class ServiceWorkerContainer { + register(u) { return answer(u); } + // late: the registrations come 20 ms after. + getRegistrations() { return late ? new Promise((r) => setTimeout(() => r(regs), 20)) : Promise.resolve(regs); } + get controller() { return state.controller; } + } + return { state, globals: { ServiceWorkerContainer, navigator: { serviceWorker: new ServiceWorkerContainer() } } }; +}; +const turn = () => new Promise((r) => setImmediate(r)); + +test('a Service Worker a document registers is named once the browser registers it, at its script\'s address; one Playwright blocks (serviceWorkers: \'block\'), or a refused one, is not; register answers as the browser\'s does', async () => { + const run = async (sw, script) => { + const told = []; + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { URL, ...sw.globals } }); + const v = await d.run(script); + await turn(); + return { told, v }; + }; + const a = await run(swGlobals(), 'navigator.serviceWorker.register("/sw.js?v=2", { scope: "/" })'); + assert.deepEqual(a.told, [['http://h/sw.js', 'service']]); + assert.deepEqual(a.v, { active: { scriptURL: 'http://h/sw.js?v=2' } }); + // Playwright's block: register answers no registration. + const blocked = swGlobals(); + blocked.globals.ServiceWorkerContainer.prototype.register = async () => undefined; + assert.deepEqual((await run(blocked, 'navigator.serviceWorker.register("/sw.js")')).told, []); + // A refused registration rejects as the browser's does, and is not named. + const refused = swGlobals({ answer: () => Promise.reject(new TypeError('the browser\'s own')) }); + await assert.rejects(run(refused, 'navigator.serviceWorker.register("/sw.js")'), (e) => e.message === 'the browser\'s own'); + // A container with no register keeps none. + const none = swGlobals(); + delete none.globals.ServiceWorkerContainer.prototype.register; + assert.equal((await run(none, 'typeof navigator.serviceWorker.register')).v, 'undefined'); + // Without the fixture's function: registered, nothing said. + const quiet = swGlobals(); + const d = documentAt(initScript(S1), '/p', new Map(), 't0', { globals: { URL, ...quiet.globals } }); + assert.deepEqual(await d.run('navigator.serviceWorker.register("/sw.js")'), { active: { scriptURL: 'http://h/sw.js' } }); +}); + +test('a worker whose script a Service Worker may answer starts as the app asked, unmarked, and is named with that Service Worker: a Web Worker of a controlled document, a Shared Worker of a document that is controlled, registered one or found one registered as it began; the next worker at its path keeps its place', async () => { + const run = async (sw, script, before = '') => { + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...sw.globals } }); + await turn(); + if (before) { await d.run(before); await turn(); } + d.run(script); + return { told, made: w.made.map(([u]) => [u, workerMark(u)]) }; + }; + // Controlled: neither kind is marked, each is named with the controller. + const c = swGlobals({ state: { controller: { scriptURL: 'http://h/sw.js?v=1' } } }); + const a = await run(c, 'new Worker("/w.js"); new SharedWorker("/s.js"); new Worker("/w.js");'); + assert.deepEqual(a.made.map(([u, m]) => [u, m]), [['/w.js', null], ['/s.js', null], ['/w.js', null]]); + assert.deepEqual(a.told, [['http://h/w.js', 'controlled', 'http://h/sw.js'], ['http://h/s.js', 'controlled', 'http://h/sw.js'], ['http://h/w.js', 'controlled', 'http://h/sw.js']]); + // Registered by this document but not its controller: its Web Worker is + // marked (its script goes to the network), its Shared Worker is not. + const r = await run(swGlobals(), 'new Worker("/w.js"); new SharedWorker("/s.js");', 'navigator.serviceWorker.register("/sw.js")'); + assert.equal(r.made[0][1].kind, 'w'); + assert.deepEqual(r.made[1], ['/s.js', null]); + assert.deepEqual(r.told, [['http://h/sw.js', 'service'], ['http://h/s.js', 'controlled', 'http://h/sw.js']]); + // A registration of its origin found as the document began. + const f = await run(swGlobals({ regs: [{ active: null, waiting: null, installing: { scriptURL: 'http://h/old.js?x' } }] }), 'new Worker("/w.js"); new SharedWorker("/s.js");'); + assert.equal(f.made[0][1].kind, 'w'); + assert.deepEqual([f.made[1], f.told], [['/s.js', null], [['http://h/s.js', 'controlled', 'http://h/old.js']]]); + // A registration found is named by its active Service Worker, else its + // waiting one, else its installing one. + const sws = { a: { scriptURL: 'http://h/a.js' }, b: { scriptURL: 'http://h/b.js' }, c: { scriptURL: 'http://h/c.js' } }; + for (const [reg, named] of [[{ active: sws.a, waiting: sws.b, installing: sws.c }, 'http://h/a.js'], [{ active: null, waiting: sws.b, installing: sws.c }, 'http://h/b.js']]) { + assert.deepEqual((await run(swGlobals({ regs: [reg] }), 'new SharedWorker("/s.js");')).told, [['http://h/s.js', 'controlled', named]]); + } + // The one this document registered is named, even when one found as + // the document began comes in after it. + { + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...swGlobals({ regs: [{ active: sws.a }], late: true }).globals } }); + await d.run('navigator.serviceWorker.register("/sw.js")'); + await new Promise((r) => setTimeout(r, 60)); + d.run('new SharedWorker("/s.js");'); + assert.deepEqual(told, [['http://h/sw.js', 'service'], ['http://h/s.js', 'controlled', 'http://h/sw.js']]); + } + // A controller whose script keploy cannot read: named with no address. + const u = await run(swGlobals({ state: { controller: { scriptURL: 'http://[' } } }), 'new Worker("/w.js");'); + assert.deepEqual([u.made, u.told], [[['/w.js', null]], [['http://h/w.js', 'controlled', '']]]); + // A worker keploy cannot mark is named for that, controlled or not. + const b = await run(c, 'new Worker("blob:http://h/0f0e");'); + assert.deepEqual([b.made, b.told], [[['blob:http://h/0f0e', null]], [['blob:', 'worker']]]); + // None: both are marked, nothing said. + const n = await run(swGlobals(), 'new Worker("/w.js"); new SharedWorker("/s.js");'); + assert.deepEqual([n.made.map(([, m]) => m.kind), n.told], [['w', 's'], []]); + // No Service Worker can be read here (an opaque origin's throws). + const opaque = { globals: { navigator: Object.defineProperty({}, 'serviceWorker', { get() { throw new Error('SecurityError: sandboxed'); } }) } }; + const o = await run(opaque, 'new Worker("/w.js");'); + assert.deepEqual([o.made.map(([, m]) => m.kind), o.told], [['w'], []]); + // A worker a Service Worker may answer still takes its place at its path: + // the next one there, once no Service Worker controls the document, is + // marked as the second at that path, as in a document with none. + const state = { controller: { scriptURL: 'http://h/sw.js' } }; + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...swGlobals({ state }).globals } }); + d.run('new Worker("/w.js");'); + state.controller = null; + d.run('new Worker("/w.js");'); + const plain = await run(swGlobals(), 'new Worker("/w.js"); new Worker("/w.js");'); + assert.deepEqual([workerMark(w.made[0][0]), workerMark(w.made[1][0])], [null, plain.made[1][1]]); +}); + test('a worker\'s seeding draws the same numbers for a token and instance every run, others for another; its location shows no mark, its own query and fragment kept', () => { const t = 'cd'.repeat(16); const a = workerAt(markWorker('http://h/w.js', 'w', t), workerScript(t, 0, 'w')).draws(); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 428d3a1..3410a42 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -657,12 +657,32 @@ async function drawsServer() { res.end(wk); return; } + // service-helper.ts's: the Service Worker /sw.js claims its pages, keeps + // the URL of each request it is asked (and leaves it to the network), and + // answers a message with them; the workers /swk/w.js and /swk/s.js send + // what they drew and their location. + const sw = { + '/sw.js': 'const seen = []; self.addEventListener(\'install\', () => self.skipWaiting()); self.addEventListener(\'activate\', (e) => e.waitUntil(self.clients.claim())); ' + + 'self.addEventListener(\'fetch\', (e) => { seen.push(e.request.url); }); self.addEventListener(\'message\', (e) => e.source.postMessage({ kind: \'sw-seen\', seen }));', + '/swk/w.js': `postMessage({ kind: 'sw-ded', href: location.href, draws: ${DRAWS} });`, + '/swk/s.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'sw-shared', href: location.href, draws: ${DRAWS} });`, + }[u.pathname]; + if (sw !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end(sw); return; } res.writeHead(200, { 'content-type': 'text/html' }); if (u.pathname === '/wk') { const start = (s) => (s.startsWith('new Worker(') ? `{ const w = ${s}; w.onmessage = take; w.postMessage('early'); }` : `{ const s = ${s}; s.port.onmessage = take; s.port.start(); }`); res.end(page(``)); return; } + // /sw registers /sw.js and, once the browser registered it, waits until + // it controls the page; then it starts /swk/w.js and /swk/s.js. + if (u.pathname === '/sw') { + res.end(page('')); + return; + } if (u.pathname === '/draws') { res.end(page(``)); return; } if (u.pathname === '/frame') { res.end(page('frame')); return; } // /nest holds /frame of its own origin. @@ -1479,6 +1499,69 @@ for (const engine of ENGINES) { }); } +// service-helper.ts's tests: one with its Service Worker, one with +// serviceWorkers: 'block'. +const SW_TITLES = ['service worker', 'service worker blocked']; + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a Service Worker a page registers draws the browser's own and is named once; a Web Worker and a Shared Worker that it may answer start as the app asked, so neither the Service Worker nor the app's server sees keploy's mark, and each is named with it; with serviceWorkers: 'block' neither is named and the Web Worker draws from the test's seed, the same at replay`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); + const byKind = (r, title) => Object.fromEntries(r.drew[title].w.map((m) => [m.kind, m])); + const specs = ['service.spec.ts', 'service-blocked.spec.ts']; + try { + const r = await drawRun(j, env(rec.url), specs, SW_TITLES); + const facts = JSON.stringify(r.drew); + const said = (title) => r.tests[title].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, `)).map((l) => l.replace(/127\.0\.0\.1:\d+\//g, '127.0.0.1:/')).sort(); + const at = (p) => `http://127.0.0.1:/${p}`; + const on = byKind(r, 'service worker'); + const off = byKind(r, 'service worker blocked'); + const origin = new URL(on['sw-ded'].href).origin; + // With its Service Worker: registered, controlling the page, and named; + // each worker named with it (in Firefox, whose route does not reach a + // Shared Worker's script, the Shared Worker is named for that). + assert.deepEqual(on.reg, { kind: 'reg', sw: true, ok: true, controlled: true }, facts); + assert.deepEqual(said('service worker'), [ + unplacedLine('service worker', at('sw.js'), 'service'), + unplacedLine('service worker', at('swk/w.js'), 'controlled', 0, at('sw.js')), + engine === 'firefox' ? unplacedLine('service worker', at('swk/s.js'), 'shared') : unplacedLine('service worker', at('swk/s.js'), 'controlled', 0, at('sw.js')), + sharedWorkerLine('service worker', at('swk/s.js')), + ].sort(), facts); + // Each worker runs at the URL the app asked for, and the Service Worker + // was asked for the Web Worker's script with no mark. + assert.deepEqual([on['sw-ded'].href, on['sw-shared'].href], [`${origin}/swk/w.js`, `${origin}/swk/s.js`], facts); + assert.equal(r.drew['service worker'].seen.kind, 'sw-seen', facts); + assert.ok(r.drew['service worker'].seen.seen.includes(`${origin}/swk/w.js`), facts); + assert.deepEqual(r.drew['service worker'].seen.seen.filter((u) => u.includes('__keploy_worker')), [], facts); + // Blocked: Playwright registers none, nothing but the Shared Worker's + // settle line is said, and the Web Worker draws from the test's seed. + assert.deepEqual(off.reg, { kind: 'reg', sw: true, ok: false, controlled: false }, facts); + assert.equal(r.drew['service worker blocked'].seen, null, facts); + assert.deepEqual(said('service worker blocked'), [ + ...(engine === 'firefox' ? [unplacedLine('service worker blocked', at('swk/s.js'), 'shared')] : []), + sharedWorkerLine('service worker blocked', at('swk/s.js')), + ].sort(), facts); + assert.deepEqual([off['sw-ded'].href, off['sw-shared'].href], [`${origin}/swk/w.js`, `${origin}/swk/s.js`], facts); + const made = []; + class Worker { constructor(url) { made.push(workerMark(String(url))); } } + documentAt(initScript(r.seeds['service worker blocked']), '/sw', new Map(), 't0', { globals: { URL, Worker } }).run('new Worker("/swk/w.js");'); + assert.deepEqual(off['sw-ded'].draws, workerAt('http://h/swk/w.js', workerScript(made[0].token, 0, 'w')).draws(), facts); + // The app's server never sees keploy's mark. + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); + kept = r.seeds; + const q = await drawRun(j, env(rep.url), specs, SW_TITLES); + assert.deepEqual(byKind(q, 'service worker blocked')['sw-ded'].draws, off['sw-ded'].draws); + if (engine !== 'firefox') assert.deepEqual(byKind(q, 'service worker blocked')['sw-shared'].draws, off['sw-shared'].draws); + assert.notDeepEqual(byKind(q, 'service worker')['sw-ded'].draws, on['sw-ded'].draws); + assert.deepEqual(q.seen.filter((u) => u.includes('__keploy_worker')), []); + } finally { await rep.close(); await rec.close(); } + }); +} + test('seeded: keploy answers no seed (an older recording, or an older keploy): the browser draws its own, each test says so, and its end says own', async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); From 68659382f704a8d25b766a0f5164ddf4bf87dc82 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 17:50:22 +0530 Subject: [PATCH 36/73] fix(playwright): seed or name the Web Workers a seeded Web Worker starts A worker that a worker started drew the browser's own random numbers, with no line, so its draws could differ at replay. A seeded Web Worker's seeding now wraps its Worker. In WebKit, whose route reaches a nested worker's script, each Web Worker it starts (classic or module, and the ones that one starts) is seeded as a page's is: named by the worker that started it, its script's path and how many workers it started at that path before it, its script URL resolved against the URL the app asked for and marked for keploy's route. Chromium's and Firefox's routes reach no such script, so there it starts as the app asked, with no mark, as does one keploy cannot mark (a blob: or data: URL, or a marked URL the browser refuses). For each of these the worker posts why and the address to its own script URL marked t: keploy's route answers it with no body and prints the worker's line, so neither a route of the test's nor the app's server sees it. The line for a worker keploy cannot reach now says that each worker it starts draws the browser's own numbers too. Rows: the seeding's marks, tokens, refusals and words, the t answer, the lines; a browser row per engine for record and replay. Signed-off-by: slayerjain --- playwright/README.md | 4 +- playwright/src/index.ts | 12 ++- playwright/src/random.ts | 128 ++++++++++++++++++++---- playwright/test/fixtures/nested.spec.ts | 21 ++++ playwright/test/random.test.mjs | 105 +++++++++++++++++-- playwright/test/rows.test.mjs | 85 +++++++++++++++- 6 files changed, 324 insertions(+), 31 deletions(-) create mode 100644 playwright/test/fixtures/nested.spec.ts diff --git a/playwright/README.md b/playwright/README.md index 2e1f6fa..d5b5c72 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page that a Service Worker controls, that registered one, or for whose origin one was registered when the page began. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page that a Service Worker controls, that registered one, or for whose origin one was registered when the page began. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -95,7 +95,7 @@ Any answer from the agent that this package cannot trust fails the test. That co - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: - a faker instance the suite makes itself with `new Faker(...)` as a test file loads and does not give to `seedFaker` (see above): it seeds its own generator from the worker's `Math.random` then; - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - - a Service Worker, and a worker that a Service Worker may answer (each says so as the test runs, above); a worker that a worker starts; + - a Service Worker, a worker that a Service Worker may answer, a worker that a worker starts in Chromium and Firefox, and the workers that one of these starts (each says so as the test runs, above); in WebKit and Firefox, a worker that a Shared Worker starts (in Firefox the Shared Worker's line names it; in WebKit, whose route reaches nothing a Shared Worker asks for after its script, nothing is said of it yet); - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; - `crypto.subtle`'s keys. diff --git a/playwright/src/index.ts b/playwright/src/index.ts index bc1dc9d..4ba8cd4 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -232,7 +232,17 @@ const keployTestType = base.extend({ told.add(k); process.stdout.write(`${unplacedLine(name, a, 'routed')}\n`); }, - }); + // A worker a seeded Web Worker started that keploy cannot mark, or + // whose script the route does not reach: said as a page's is, once + // for each address and reason. WebKit's route reaches such a script. + unmarked: (at, why) => { + const a = pageAddress(at); + const k = `${why}\n${a}`; + if (told.has(k)) return; + told.add(k); + process.stdout.write(`${unplacedLine(name, a, why)}\n`); + }, + }, browserName === 'webkit'); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox') }); const placing = placePages(context, mark); restore = placing.restore; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 512cd1d..f0d2815 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -97,17 +97,23 @@ export function pageAddress(at: unknown): string { * a frame inside one, 'nostorage' for a top document with no session * storage to count its documents in (other than a page's first document), * or a frame of its origin inside one, 'routed' for a worker whose script - * a route of the test's own answered itself (workerRoutes), 'service' for a + * a route of the test's own answered itself (workerRoutes), 'nested' for a + * Web Worker a seeded Web Worker started where the route does not reach its + * script (NESTED_JS), 'service' for a * Service Worker a document registered, 'controlled' for a worker whose * script a Service Worker may answer (initScript). */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'service' | 'controlled'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'service' | 'controlled' | 'nested'; // The next step for a Service Worker the tests do not need. const BLOCK = 'if the tests do not need it, set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered)'; +// A worker keploy cannot reach starts its own workers unseeded too: its +// line says so (unplacedLine). +const KIDS = 'and so does each worker it starts, so their draws may differ at replay'; + // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service', string> = { +const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'nested', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + @@ -117,9 +123,11 @@ const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service', string> = { 'for the worker\'s script to seed it.', service: 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + `${BLOCK}.`, + nested: 'it is a worker that a worker started, and Chromium and Firefox do not let keploy reach its script to put its seeding ahead of it; ' + + 'record and replay the test in WebKit to seed it.', }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -143,13 +151,13 @@ const UNPLACED: Record { const g = globalThis; const k = ${JSON.stringify(`${token}\n${n}\n`)}; @@ -1466,10 +1475,72 @@ ${DRAW_JS} Object.defineProperty(C.prototype, 'stack', { get: Object.getOwnPropertyDescriptor(v, 'stack').get, set: d.set, enumerable: d.enumerable, configurable: true }); } catch (e) {} } -})(); +${kind === 'w' || kind === 'm' ? NESTED_JS(token, reach) : ''}})(); `; } +/** + * A Web Worker's seeding of the Web Workers it starts (workerScript). Where + * the context's route reaches their scripts (reach: WebKit), each is named by this worker's key, its script's path and how many workers + * this worker started at that path before it, and its script URL, resolved + * against the URL the app asked for, carries keploy's mark, which the + * context's route answers with its seeding (workerAnswer), as a page's + * workers are. Chromium and Firefox let the route reach none of them, so + * there each starts as the app asked, with no mark ('nested'), as does one + * keploy cannot mark (a blob: or data: URL, or a marked URL the browser + * refuses: 'worker'). For each of these this worker tells keploy why and + * the address: it posts them to its own script's URL marked t (kind t), a + * request keploy's route answers itself (workerRoutes), so the app's server + * never sees it. A Shared Worker's scope is left as it is. + */ +const NESTED_JS = (token: string, reach: boolean): string => ` const RW = g.Worker; + if (typeof RW === 'function') { + const U = g.URL; + const hex = (s) => hash(s).map((w) => (w + 4294967296).toString(16).slice(1)).join(''); + const started = new Map(); + const mark = (h, kind, t) => { + const i = h.indexOf('#'); + const b = i < 0 ? h : h.slice(0, i); + return b + (b.indexOf('?') < 0 ? '?' : '&') + ${JSON.stringify(`${WORKER_PARAM}=`)} + kind + '.' + t + (i < 0 ? '' : h.slice(i)); + }; + const tell = (why, where) => { + try { F.call(g, mark(own, 't', ${JSON.stringify(token)}), { method: 'POST', body: JSON.stringify([why, where]) }).then(undefined, () => {}); } catch (e) {} + }; + const W = { Worker: function (url) { + // Called without new, it throws as the browser's own does. + if (!new.target) return RW.apply(this, arguments); + const args = Array.prototype.slice.call(arguments); + let u = null; + try { u = new U(String(url), own); } catch (e) { u = null; } + // A URL the browser cannot read: its own error. + if (u === null) return Reflect.construct(RW, args, new.target); + const web = u.protocol === 'http:' || u.protocol === 'https:'; + if (web) { + const n = started.get(u.pathname) || 0; + started.set(u.pathname, n + 1); + const o = args[1]; + const m = ${reach ? 'true' : 'false'} && mark(u.href, o !== null && typeof o === 'object' && o.type === 'module' ? 'm' : 'w', hex(k + 'worker\\n' + u.pathname + '\\n' + n)); + if (m) { + try { return Reflect.construct(RW, [m].concat(args.slice(1)), new.target); } catch (e) { + // Refused with the mark (a Trusted Types policy): as the app asked. + } + } + } + const w = Reflect.construct(RW, args, new.target); + tell(web && !${reach ? 'true' : 'false'} ? 'nested' : 'worker', web ? u.origin + u.pathname : u.protocol); + return w; + } }.Worker; + try { + W.prototype = RW.prototype; + Object.defineProperty(RW.prototype, 'constructor', { value: W, writable: true, enumerable: false, configurable: true }); + Object.setPrototypeOf(W, Object.getPrototypeOf(RW)); + define(g, 'Worker', W); + } catch (e) { + // A scope that takes no property: its workers draw their own. + } + } +`; + /** * Whether a module Shared Worker answered with these headers may import a * data: module: each Content-Security-Policy that names a source list for @@ -1521,9 +1592,9 @@ export function cspAllowsData(headers: Record): boolean { * order, each in a microtask of its own, to the handlers the app's script * sets. */ -export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true): string { +export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true, reach = false): string { const url = JSON.stringify(own); - if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; + if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind, reach)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; if (kind === 'm') return `import ${JSON.stringify(markWorker(own, 'p', token, n))};\nimport ${url};\n`; const seeding = workerScript(token, n, 'x'); if (dataOk) return `import ${JSON.stringify(`data:text/javascript;${WORKER_PARAM}=x.${token};base64,${Buffer.from(seeding).toString('base64')}`)};\nimport ${url};\n`; @@ -1566,6 +1637,12 @@ export type WorkerHooks = { * origin and path) itself: it runs unseeded, at its marked URL. */ routed?: (at: string) => void; + /** + * A seeded Web Worker started a worker at at (its origin and path, or its + * scheme: blob:, data:) that keploy cannot mark ('worker') or whose script + * the route does not reach ('nested'), which so runs unseeded. + */ + unmarked?: (at: string, why: 'worker' | 'nested') => void; }; /** What keploy's routes go on: the test's context, or one of its pages. */ @@ -1625,7 +1702,10 @@ function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Ro * XMLHttpRequest's: its fetch and Request ask for no mark, workerScript) * goes on to the test's routes and the app's server with the mark taken off * (the URL the app's code meant, as the worker's location shows it), and - * starts no instance of the token. + * starts no instance of the token. A seeded Web Worker's word of a worker it + * started that keploy cannot mark (marked t: NESTED_JS) is answered here, with + * no body, and heard (unmarked); it reaches no route of the test's and not + * the app's server. * * keploy's route comes first, ahead of every route the test adds to its * context or to one of its pages (keepFirst), so no route of the test's sees @@ -1637,7 +1717,7 @@ function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Ro * it. One that falls back, or none, leaves it to keploy's last route, the * context's first, which answers it as above. */ -export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}): Promise { +export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}, reach = false): Promise { const counts = new Map(); // The answer's headers, but its encoding and length: the body given is // keploy's, of another length. @@ -1656,7 +1736,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = hooks.answered?.(route.request()); }; if (m.kind === 'p') { - await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm') }); + await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm', reach) }); return; } const n = counts.get(m.token) ?? 0; @@ -1674,7 +1754,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } catch { // The worker loads the app's script itself, and fails as it would. } - await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers)) }); + await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers), reach) }); }; const first = async (route: Route): Promise => { const r = route.request(); @@ -1683,6 +1763,16 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await route.fallback(); return; } + // A seeded Web Worker's word of a worker keploy cannot mark: heard, then + // answered here, so it never leaves the browser. + if (m.kind === 't') { + let said: unknown = null; + try { said = JSON.parse(r.postData() ?? ''); } catch { said = null; } + if (Array.isArray(said) && (said[0] === 'worker' || said[0] === 'nested') && typeof said[1] === 'string' && said[1] !== '') hooks.unmarked?.(said[1], said[0]); + await route.fulfill({ status: 204 }); + hooks.answered?.(r); + return; + } if (!workerScriptLoad(r)) { await route.fallback({ url: m.url }); return; diff --git a/playwright/test/fixtures/nested.spec.ts b/playwright/test/fixtures/nested.spec.ts new file mode 100644 index 0000000..e0177cb --- /dev/null +++ b/playwright/test/fixtures/nested.spec.ts @@ -0,0 +1,21 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). The page at /nw starts a +// classic and a module Web Worker, each of which starts Web Workers beside +// it (the classic one: a classic one twice at one path, one of which starts +// another, a module one, and one from a blob: URL twice), and a Shared Worker, +// which says whether its scope has Worker. Each worker sends what it drew +// and its location, which the worker that started it passes on. The test +// keeps them, under its title. +const base = process.env.DRAWS_BASE ?? ''; + +test('nested', async ({ page }) => { + await page.goto(`${base}/nw`); + await page.waitForFunction(() => { + const w = (window as unknown as { __w: { kind: string; worker?: string }[] }).__w; + const a = w.find((x) => x.kind === 'a'); + return a !== undefined && w.length >= (a.worker === 'function' ? 10 : 3); + }, null, { timeout: 20000 }); + fs.writeFileSync(`${process.env.MARKER_DIR}/nested.json`, JSON.stringify(await page.evaluate(() => (window as unknown as { __w: unknown[] }).__w))); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index f50f628..dd0ea7b 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -1365,7 +1365,7 @@ const workerGlobals = () => { test('a worker\'s mark: what markWorker makes, workerMark reads back, the URL as the app asked for it, query and fragment kept; any other URL has none', () => { const t = 'ab'.repeat(16); for (const url of ['http://h/w.js', 'http://h/w.js?a=1', 'http://h/w.js?', 'http://h/w.js#f', 'http://h/w.js?a=1#f?x']) { - for (const [kind, n] of [['w', -1], ['m', -1], ['s', -1], ['x', -1], ['p', 0], ['p', 7]]) { + for (const [kind, n] of [['w', -1], ['m', -1], ['s', -1], ['x', -1], ['t', -1], ['p', 0], ['p', 7]]) { const m = markWorker(url, kind, t, n); assert.ok(new URL(m).searchParams.has(WORKER_PARAM), m); assert.deepEqual(workerMark(m), { kind, token: t, n, url }, m); @@ -1373,7 +1373,7 @@ test('a worker\'s mark: what markWorker makes, workerMark reads back, the URL as } assert.equal(markWorker('http://h/w.js?a=1#f', 'w', t), `http://h/w.js?a=1&__keploy_worker=w.${t}#f`); for (const u of ['http://h/w.js', `http://h/w.js?__keploy_worker=w.${t}&a=1`, `http://h/w.js?__keploy_worker=z.${t}`, `http://h/w.js?__keploy_worker=w.${t.toUpperCase()}`, - `http://h/w.js?__keploy_worker=w.${t}.0`, `http://h/w.js?__keploy_worker=o.${t}`, `http://h/w.js?__keploy_worker=o.${t}.0`, `http://h/w.js?__keploy_worker=p.${t}`, `http://h/w.js?__keploy_worker=w.${t.slice(1)}`]) { + `http://h/w.js?__keploy_worker=w.${t}.0`, `http://h/w.js?__keploy_worker=t.${t}.0`, `http://h/w.js?__keploy_worker=o.${t}`, `http://h/w.js?__keploy_worker=o.${t}.0`, `http://h/w.js?__keploy_worker=p.${t}`, `http://h/w.js?__keploy_worker=w.${t.slice(1)}`]) { assert.equal(workerMark(u), null, u); } }); @@ -1441,25 +1441,32 @@ test('a Shared Worker\'s token is the same in every document of the test, by scr assert.deepEqual([off, told], [[[null, undefined]], [['http://h/s.js', 'shared']]]); }); +// What a line for a worker keploy cannot reach says of the workers it starts. +const KIDS = 'and so does each worker it starts, so their draws may differ at replay: '; + test('the lines for a Service Worker and a worker one may answer say what it is, why, and the next step', () => { const block = 'if the tests do not need it, set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered).'; assert.equal(unplacedLine('t', 'http://h/sw.js', 'service'), 'keploy: in t, the Service Worker at http://h/sw.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + block); - assert.equal(unplacedLine('t', 'http://h/w.js', 'controlled', 0, 'http://h/sw.js'), 'keploy: in t, the worker at http://h/w.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + assert.equal(unplacedLine('t', 'http://h/w.js', 'controlled', 0, 'http://h/sw.js'), 'keploy: in t, the worker at http://h/w.js draws the browser\'s own random numbers, ' + KIDS + 'a Service Worker (at http://h/sw.js) may answer the worker\'s script, and it would see keploy\'s __keploy_worker mark in the script\'s URL, so keploy starts the worker as the app asked; ' + block); - assert.equal(unplacedLine('t', 'http://h/w.js', 'controlled'), 'keploy: in t, the worker at http://h/w.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + assert.equal(unplacedLine('t', 'http://h/w.js', 'controlled'), 'keploy: in t, the worker at http://h/w.js draws the browser\'s own random numbers, ' + KIDS + 'a Service Worker may answer the worker\'s script, and it would see keploy\'s __keploy_worker mark in the script\'s URL, so keploy starts the worker as the app asked; ' + block); }); -test('the lines for a worker keploy cannot reach say what it is, why, and the next step', () => { - assert.equal(unplacedLine('t', 'blob:', 'worker'), 'keploy: in t, the worker at blob: draws the browser\'s own random numbers, so its draws may differ at replay: ' + +test('the lines for a worker keploy cannot reach say what it is, that each worker it starts is unseeded too, why, and the next step', () => { + assert.equal(unplacedLine('t', 'blob:', 'worker'), 'keploy: in t, the worker at blob: draws the browser\'s own random numbers, ' + KIDS + 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.'); - assert.equal(unplacedLine('t', 'http://h/s.js', 'shared'), 'keploy: in t, the Shared Worker at http://h/s.js draws the browser\'s own random numbers, so its draws may differ at replay: ' + + assert.equal(unplacedLine('t', 'http://h/s.js', 'shared'), 'keploy: in t, the Shared Worker at http://h/s.js draws the browser\'s own random numbers, ' + KIDS + 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + 'record and replay the test in Chromium or WebKit to seed it.'); + assert.equal(unplacedLine('t', 'http://h/b.js', 'nested'), 'keploy: in t, the worker at http://h/b.js draws the browser\'s own random numbers, ' + KIDS + + 'it is a worker that a worker started, and Chromium and Firefox do not let keploy reach its script to put its seeding ahead of it; record and replay the test in WebKit to seed it.'); + assert.match(unplacedLine('t', 'http://h/w.js', 'routed'),/^keploy: in t, the worker at http:\/\/h\/w\.js draws the browser's own random numbers, and so does each worker it starts, so their draws may differ at replay: a route of the test's own /); }); + // A document's ServiceWorkerContainer: register answers what answer gives // (a registration of its script, by default), getRegistrations the // registrations regs, and the controller is state.controller. @@ -1590,6 +1597,68 @@ test('a worker\'s seeding draws the same numbers for a token and instance every assert.equal(workerAt(other, workerScript(t, 0, 'w')).run('location.href'), other); }); +// A worker's seeding (token, instance n, kind; reach: whether the route +// reaches the scripts of the workers it starts) at /d/a.js?x=1, with a Worker +// that keeps what it is given (or W) and a fetch that keeps what it posts; +// code then runs in it. +const nestedAt = (token, n, kind, code, W, reach = true) => { + const made = []; + const posted = []; + class Worker { constructor(url, opts) { made.push([String(url), opts === undefined ? undefined : JSON.parse(JSON.stringify(opts))]); } } + const fetch = (u, o) => { posted.push([String(u), o === undefined ? undefined : JSON.parse(JSON.stringify(o))]); return Promise.resolve(); }; + const w = workerAt(markWorker('http://h/d/a.js?x=1', kind, token), workerScript(token, n, kind, reach), { URL, Worker: W ?? Worker, fetch, Request: class Request {} }); + return { made, posted, v: w.run(code) }; +}; + +test('a Web Worker\'s seeding starts each Web Worker it starts at its script URL, resolved against the URL the app asked for, marked with a token of its own: the same each run, another for the next worker at that path, another path, token or instance; a module one is marked so; it keeps new, instanceof and its constructor; a Shared Worker\'s seeding leaves Worker as it is', () => { + const t = 'cd'.repeat(16); + const code = 'new Worker("b.js", { name: "b0" }); new Worker("b.js", null); new Worker("/e/m.js", { type: "module" }); new Worker("?q=1#f"); new Worker("#f"); new Worker("https://h/s.js"); 0'; + for (const kind of ['w', 'm']) { + const a = nestedAt(t, 0, kind, code); + const marks = a.made.map(([u]) => workerMark(u)); + assert.deepEqual(a.made.map(([, o], i) => [marks[i].kind, marks[i].url, o]), [['w', 'http://h/d/b.js', { name: 'b0' }], ['w', 'http://h/d/b.js', null], + ['m', 'http://h/e/m.js', { type: 'module' }], ['w', 'http://h/d/a.js?q=1#f', undefined], ['w', 'http://h/d/a.js?x=1#f', undefined], ['w', 'https://h/s.js', undefined]], kind); + for (const m of marks) assert.match(m.token, TOKEN); + assert.equal(new Set(marks.map((m) => m.token)).size, 6); + assert.deepEqual(nestedAt(t, 0, kind, code).made, a.made); + assert.deepEqual(a.posted, []); + for (const [tt, n] of [['ef'.repeat(16), 0], [t, 1]]) assert.notEqual(workerMark(nestedAt(tt, n, kind, code).made[0][0]).token, marks[0].token); + assert.equal(nestedAt(t, 0, kind, 'const x = new Worker("b.js"); x instanceof Worker && Worker.prototype.constructor === Worker && Worker.name === "Worker" && Worker.length === 1').v, true); + assert.throws(() => nestedAt(t, 0, kind, 'Worker("b.js")'), TypeError); + } + // Each worker counts its own: the same for the same worker. + assert.equal(workerMark(nestedAt(t, 0, 'w', 'new Worker("b.js"); 0').made[0][0]).token, workerMark(nestedAt(t, 0, 'm', 'new Worker("b.js"); 0').made[0][0]).token); + for (const kind of ['s', 'x']) assert.deepEqual(nestedAt(t, 0, kind, 'new Worker("b.js"); 0').made, [['b.js', undefined]], kind); + // Where the route does not reach their scripts (Chromium, Firefox): each + // starts as the app asked, and its address is posted, with why. + const to = markWorker('http://h/d/a.js?x=1', 't', t); + for (const kind of ['w', 'm']) { + const a = nestedAt(t, 0, kind, 'new Worker("b.js", { name: "b0" }); new Worker("https://h/s.js?q=1"); new Worker("blob:http://h/0f"); 0', undefined, false); + assert.deepEqual(a.made, [['b.js', { name: 'b0' }], ['https://h/s.js?q=1', undefined], ['blob:http://h/0f', undefined]], kind); + assert.deepEqual(a.posted, [[to, { method: 'POST', body: '["nested","http://h/d/b.js"]' }], [to, { method: 'POST', body: '["nested","https://h/s.js"]' }], + [to, { method: 'POST', body: '["worker","blob:"]' }]], kind); + } + assert.deepEqual(workerAnswer(t, 0, 'w', 'http://h/w.js', true, true).split('queueMicrotask(')[0], workerScript(t, 0, 'w', true)); + assert.notEqual(workerScript(t, 0, 'w', true), workerScript(t, 0, 'w')); +}); + +test('a Web Worker\'s seeding starts a Web Worker keploy cannot mark as the app asked and posts its address to the worker\'s own URL marked t: a blob: or data: URL, or a marked URL the browser refuses; a URL the browser cannot read throws its own error and posts nothing', () => { + const t = 'cd'.repeat(16); + const to = markWorker('http://h/d/a.js?x=1', 't', t); + const post = (at) => [to, { method: 'POST', body: JSON.stringify(['worker', at]) }]; + const a = nestedAt(t, 0, 'w', 'new Worker("blob:http://h/0f0e"); new Worker("data:text/javascript,1"); 0'); + assert.deepEqual([a.made.map(([u]) => u), a.posted], [['blob:http://h/0f0e', 'data:text/javascript,1'], [post('blob:'), post('data:')]]); + const made = []; + class Picky { constructor(url) { if (workerMark(String(url)) !== null) throw new TypeError('refused'); made.push(String(url)); } } + assert.deepEqual(nestedAt(t, 0, 'm', 'new Worker("/e/b.js?q=1"); 0', Picky).posted, [post('http://h/e/b.js')]); + assert.deepEqual(made, ['/e/b.js?q=1']); + class Strict { constructor(url) { if (String(url) === 'http://[') throw new SyntaxError('the browser\'s own'); } } + const posted = []; + assert.throws(() => workerAt(markWorker('http://h/d/a.js', 'w', t), workerScript(t, 0, 'w'), { URL, Worker: Strict, fetch: (u) => { posted.push(u); return Promise.resolve(); }, Request: class Request {} }) + .run('new Worker("http://[")'), (e) => e.message === 'the browser\'s own'); + assert.deepEqual(posted, []); +}); + test('a worker\'s seeding: an Error\'s or a DOMException\'s stack read through its prototype\'s getter (Firefox) leaves out each frame at a URL whose query has keploy\'s mark, any kind; its setter, flags and a value that is no string are kept; a stack that is no getter stays as it was', () => { const t = 'ab'.repeat(16); const href = markWorker('http://h/w.js', 'w', t); @@ -1716,7 +1785,7 @@ function routed() { let at = url; let over; let ended = false; - const request = { url: () => at, resourceType: () => as.type, headers: () => as.headers }; + const request = { url: () => at, resourceType: () => as.type, headers: () => as.headers, postData: () => as.body ?? null }; const route = { request: () => request, fetch: async (o) => { did.push(['fetch', o.url]); if (answer instanceof Error) throw answer; return answer; }, @@ -1805,6 +1874,26 @@ test('workerRoutes answers a marked URL as a worker\'s script only when the brow assert.equal(answered.length, 5); }); +test('workerRoutes answers a seeded Web Worker\'s word of a worker keploy cannot mark (marked t) itself, with no body, and hears its address; one with no address is answered and names none; neither reaches a route of the test\'s', async () => { + const { r, context } = routed(); + const answered = []; + const unmarked = []; + await workerRoutes(context, { answered: (q) => answered.push(q.url()), unmarked: (at, why) => unmarked.push([why, at]) }); + let theirs = 0; + await context.route('**/*', async (route) => { theirs++; await route.continue(); }); + const t = 'ab'.repeat(16); + const to = markWorker('http://h/d/a.js?x=1', 't', t); + const one = [['worker', 'blob:']]; + const two = [...one, ['nested', 'http://h/e/b.js']]; + const words = [['["worker","blob:"]', one], ['["nested","http://h/e/b.js"]', two], [undefined, two], ['blob:', two], ['["routed","http://h/x.js"]', two], + ['["worker",""]', two], ['["worker",7]', two], ['{"0":"worker","1":"blob:"}', two]]; + for (const [body, said] of words) { + assert.deepEqual(await r.ask(to, undefined, { type: 'fetch', headers: { 'sec-fetch-dest': 'empty' }, body }), [['fulfill', { status: 204, body: 'undefined' }]]); + assert.deepEqual(unmarked, said); + } + assert.deepEqual([answered.length, new Set(answered).size, theirs], [words.length, 1, 0]); +}); + test('cspAllowsData: no policy, or each policy whose script or worker source list is named lists data:; a policy that names one without it refuses', () => { const csp = (v) => ({ 'content-security-policy': v }); assert.equal(cspAllowsData({}), true); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 3410a42..affed4d 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -10,7 +10,7 @@ import http from 'node:http'; import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; -import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, workerMark, workerScript } from '../dist/random.js'; +import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, markWorker, workerMark, workerScript } from '../dist/random.js'; import { sharedWorkerLine } from '../dist/settle.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; @@ -650,6 +650,18 @@ async function drawsServer() { // twice) served with a policy that refuses data: scripts. Each // connection after a worker's first is named by its number (c1+2). '/wk/mshared.js': `import { dep } from './lib.mjs'; const top = String(new Error('at').stack); const ref = ${REFS}; self.onconnect = (e) => { e.ports[0].postMessage({ kind: 'mshared', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top, errs: ${ERRS} }); ref.then((v) => e.ports[0].postMessage({ kind: 'mshared-ref', ...v })); };`, + // nested.spec.ts's: the page at /nw starts the classic Web Worker + // /nw/a.js?x=1 and the module one /nw/am.js, which start the workers + // beside them and pass on what each sends; /nw/b.js named b0 starts + // /nw/c.js. /nw/s.js, a Shared Worker, says whether its scope has Worker. + '/nw/a.js': `postMessage({ kind: 'a', draws: ${DRAWS}, href: location.href, worker: typeof Worker }); const start = (u, o) => { new Worker(u, o).onmessage = (e) => postMessage(e.data); }; ` + + 'if (typeof Worker === \'function\') { start(\'b.js\', { name: \'b0\' }); start(\'b.js\', { name: \'b1\' }); start(\'bm.js\', { type: \'module\' }); ' + + 'const blob = URL.createObjectURL(new Blob([\'postMessage({ kind: "blob" });\'], { type: \'text/javascript\' })); start(blob); start(blob); }', + '/nw/am.js': `postMessage({ kind: 'am', draws: ${DRAWS}, href: location.href, meta: import.meta.url }); if (typeof Worker === 'function') new Worker('b.js', { name: 'mb' }).onmessage = (e) => postMessage(e.data);`, + '/nw/b.js': `postMessage({ kind: self.name, draws: ${DRAWS}, href: location.href }); if (self.name === 'b0') new Worker('c.js').onmessage = (e) => postMessage(e.data);`, + '/nw/bm.js': `postMessage({ kind: 'bm', draws: ${DRAWS}, href: location.href, meta: import.meta.url });`, + '/nw/c.js': `postMessage({ kind: 'c', draws: ${DRAWS}, href: location.href });`, + '/nw/s.js': 'onconnect = (e) => e.ports[0].postMessage({ kind: \'sh\', worker: typeof Worker });', '/wk/cshared.js': `import { dep } from './lib.mjs'; let n = 0; const draws = ${DRAWS}; const errs = ${ERRS}; const top = String(new Error('at').stack); const ref = ${REFS}; self.onconnect = (e) => { n++; if (n === 1) ref.then((v) => e.ports[0].postMessage({ kind: self.name + '-ref', ...v })); e.ports[0].postMessage({ kind: self.name + (n > 1 ? \`+\${n}\` : ''), draws, href: location.href, dep, meta: import.meta.url, top, stack: String(new Error('at').stack), errs }); };`, }[u.pathname]; if (wk !== undefined) { @@ -674,6 +686,11 @@ async function drawsServer() { res.end(page(``)); return; } + if (u.pathname === '/nw') { + res.end(page('')); + return; + } // /sw registers /sw.js and, once the browser registered it, waits until // it controls the page; then it starts /swk/w.js and /swk/s.js. if (u.pathname === '/sw') { @@ -1499,6 +1516,72 @@ for (const engine of ENGINES) { }); } +// The marks a document at /nw of page t0 gives the workers nested.spec.ts's +// page starts, and the marks those Web Workers' seedings give theirs. +function nestedMarks(seed) { + const made = []; + class Worker { constructor(url) { made.push(workerMark(String(url))); } } + documentAt(initScript(seed), '/nw', new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }) + .run('new Worker(\'/nw/a.js?x=1\'); new Worker(\'/nw/am.js\', { type: \'module\' }); new SharedWorker(\'/nw/s.js\');'); + const kids = (m, code) => { + const got = []; + class W { constructor(url) { got.push(workerMark(String(url))); } } + workerAt(markWorker(m.url, m.kind, m.token), workerScript(m.token, 0, m.kind, true), { URL, Worker: W, fetch: () => Promise.resolve(), Request: class Request {} }).run(code); + return got; + }; + const [a, am] = made; + const [b0, b1, bm] = kids(a, 'new Worker(\'b.js\', { name: \'b0\' }); new Worker(\'b.js\', { name: \'b1\' }); new Worker(\'bm.js\', { type: \'module\' });'); + const [mb] = kids(am, 'new Worker(\'b.js\', { name: \'mb\' });'); + const [c] = kids(b0, 'new Worker(\'c.js\');'); + return { a, am, b0, b1, bm, mb, c }; +} + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: the Web Workers a Web Worker starts, classic and module, and one that one of them starts, draw from the test's seed in WebKit, each its own numbers, the same at replay, at the app's location; in Chromium and Firefox, whose route does not reach their scripts, they start as the app asked and each address is named once; the app's server never sees keploy's mark; one it starts from a blob: URL draws its own and is named once`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); + const SEEDED = [['a', 'a.js?x=1'], ['am', 'am.js'], ['b0', 'b.js'], ['b1', 'b.js'], ['bm', 'bm.js'], ['mb', 'b.js'], ['c', 'c.js']]; + try { + const r = await drawRun(j, env(rec.url), ['nested.spec.ts'], ['nested']); + const w = Object.fromEntries(r.drew.nested.map((x) => [x.kind, x])); + const facts = JSON.stringify(r.drew.nested); + assert.deepEqual(Object.keys(w).sort(), ['a', 'am', 'b0', 'b1', 'blob', 'bm', 'c', 'mb', 'sh'], facts); + // Chromium has no Worker in a Shared Worker's scope. + assert.equal(w.sh.worker, engine === 'chromium' ? 'undefined' : 'function', facts); + const seeds = engine === 'webkit'; + const m = nestedMarks(r.seeds.nested); + assert.deepEqual(SEEDED.map(([k]) => m[k].kind), ['w', 'm', 'w', 'w', 'm', 'w', 'w']); + assert.equal(new Set(SEEDED.map(([k]) => m[k].token)).size, SEEDED.length); + for (const [k, file] of SEEDED) { + assert.match(w[k].href, new RegExp(`^http://127\\.0\\.0\\.1:\\d+/nw/${file.replace(/[.?]/g, '\\$&')}$`), `${k} ${facts}`); + const model = workerAt(`http://h/nw/${file}`, workerScript(m[k].token, 0, m[k].kind)).draws(); + if (seeds || k === 'a' || k === 'am') assert.deepEqual(w[k].draws, model, `${k} ${facts}`); + else assert.notDeepEqual(w[k].draws, model, `${k} ${facts}`); + } + for (const k of ['am', 'bm']) assert.equal(w[k].meta, w[k].href); + const said = r.tests.nested.stdout.split('\n').filter((l) => l.startsWith('keploy: in nested, ')).map((l) => l.replace(/:\d+\//, ':/')); + const at = (p) => `http://127.0.0.1:/nw/${p}`; + assert.deepEqual(said.sort(), [unplacedLine('nested', 'blob:', 'worker'), sharedWorkerLine('nested', at('s.js')), + ...(seeds ? [] : ['b.js', 'bm.js'].map((f) => unplacedLine('nested', at(f), 'nested'))), + ...(engine === 'firefox' ? [unplacedLine('nested', at('s.js'), 'shared')] : [])].sort(), facts); + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); + for (const u of ['/nw/a.js?x=1', '/nw/am.js', '/nw/b.js', '/nw/bm.js', '/nw/c.js', '/nw/s.js']) assert.ok(r.seen.includes(u), `${u}: ${r.seen.join(' ')}`); + kept = r.seeds; + const q = await drawRun(j, env(rep.url), ['nested.spec.ts'], ['nested']); + const p = Object.fromEntries(q.drew.nested.map((x) => [x.kind, x])); + for (const [k] of SEEDED) { + if (seeds || k === 'a' || k === 'am') assert.deepEqual(p[k].draws, w[k].draws, k); + else assert.notDeepEqual(p[k].draws, w[k].draws, k); + } + assert.deepEqual(q.seen.filter((u) => u.includes('__keploy_worker')), []); + } finally { await rep.close(); await rec.close(); } + }); +} + // service-helper.ts's tests: one with its Service Worker, one with // serviceWorkers: 'block'. const SW_TITLES = ['service worker', 'service worker blocked']; From 50e14df0409cd3d4757c28cc6e5a20db87287618 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 21:16:26 +0530 Subject: [PATCH 37/73] fix(playwright): mark a Shared Worker only when no scope holds it A Shared Worker a page started before the browser answered which Service Workers its origin has was marked, and so was one whose URL the scope of a registration another page made after this page began holds: a Service Worker could then see keploy's mark in the worker's script URL. keploy now marks a Shared Worker only when it knows that no registration's scope holds the worker's URL. Each document asks getRegistrations as it begins and keeps the answer; the documents of the test's context tell each other, on a BroadcastChannel of keploy's, the scope of each register they call (before the browser registers it) and of each registration that answer finds. A register Playwright's block answers with none is taken back, unless the browser found that registration. A Shared Worker started before the answer, or in a document that cannot read its Service Workers, starts as the app asked and is named with a line whose next step is serviceWorkers: 'block': keploy knows at once that a context Playwright blocks has none. A document with no Service Workers (an insecure context) is known to have none. The registration whose scope is the longest that holds the worker's URL names the Service Worker. Rows: the decision before and after the answer, never answered, no Service Workers, scopes, the page's own register and its options, Playwright's block, the channel between documents and messages of any other shape; a browser row per engine for record and replay. The worker and nested-worker pages run with serviceWorkers: 'block', since they start their Shared Workers as they load. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/index.ts | 2 +- playwright/src/random.ts | 151 +++++++++++++---- playwright/test/draws.mjs | 6 +- playwright/test/fixtures/nested.spec.ts | 5 + .../test/fixtures/service-known.spec.ts | 73 ++++++++ playwright/test/fixtures/workers.spec.ts | 5 + playwright/test/random.test.mjs | 158 +++++++++++++++++- playwright/test/rows.test.mjs | 63 ++++++- 9 files changed, 426 insertions(+), 39 deletions(-) create mode 100644 playwright/test/fixtures/service-known.spec.ts diff --git a/playwright/README.md b/playwright/README.md index d5b5c72..4f90353 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page that a Service Worker controls, that registered one, or for whose origin one was registered when the page began. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), or in a page that cannot read its Service Workers (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 4ba8cd4..d6d5548 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -182,7 +182,7 @@ const keployTestType = base.extend({ let opened: (page: Page | null | undefined) => Promise = async () => false; await context.exposeBinding(report, async (source, at: unknown, why: unknown, above: unknown) => { const a = pageAddress(at); - const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' || why === 'worker' || why === 'shared' || why === 'service' || why === 'controlled' + const w: Unplaced = why === 'frame' || why === 'uncounted' || why === 'nostorage' || why === 'worker' || why === 'shared' || why === 'service' || why === 'controlled' || why === 'early' ? why : (await opened(source.page)) ? 'own' : 'popup'; // Once for each address and reason: two documents at one address // (about:blank) may have no place for two reasons. diff --git a/playwright/src/random.ts b/playwright/src/random.ts index f0d2815..8184891 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -101,9 +101,11 @@ export function pageAddress(at: unknown): string { * Web Worker a seeded Web Worker started where the route does not reach its * script (NESTED_JS), 'service' for a * Service Worker a document registered, 'controlled' for a worker whose - * script a Service Worker may answer (initScript). + * script a Service Worker may answer, 'early' for a Shared Worker started + * before keploy knew whether a Service Worker may answer its script + * (initScript). */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'service' | 'controlled' | 'nested'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'service' | 'controlled' | 'early' | 'nested'; // The next step for a Service Worker the tests do not need. const BLOCK = 'if the tests do not need it, set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered)'; @@ -113,7 +115,7 @@ const BLOCK = 'if the tests do not need it, set serviceWorkers: \'block\' in the const KIDS = 'and so does each worker it starts, so their draws may differ at replay'; // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'nested', string> = { +const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'early' | 'nested', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + @@ -123,11 +125,14 @@ const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'nested', s 'for the worker\'s script to seed it.', service: 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + `${BLOCK}.`, + early: 'it is a Shared Worker started before the browser told keploy which Service Workers the page\'s origin has (keploy asks as each document begins, ' + + 'and a document that cannot read its Service Workers, as a sandboxed one, is never told), and a Service Worker whose scope holds the worker\'s URL ' + + `would see keploy's __keploy_worker mark in the script's URL, so keploy starts the worker as the app asked; ${BLOCK}.`, nested: 'it is a worker that a worker started, and Chromium and Firefox do not let keploy reach its script to put its seeding ahead of it; ' + 'record and replay the test in WebKit to seed it.', }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -155,8 +160,8 @@ export function unplacedLine(name: string, at: string, why: Unplaced = 'popup', `a Service Worker${by === '' ? '' : ` (at ${by})`} may answer the worker's script, and it would see keploy's __keploy_worker mark in the script's URL, so keploy starts the worker as the app asked; ` + `${BLOCK}.`; } - if (why === 'worker' || why === 'shared' || why === 'routed' || why === 'service' || why === 'nested') { - const what = why === 'shared' ? 'Shared Worker' : why === 'service' ? 'Service Worker' : 'worker'; + if (why === 'worker' || why === 'shared' || why === 'routed' || why === 'service' || why === 'early' || why === 'nested') { + const what = why === 'shared' || why === 'early' ? 'Shared Worker' : why === 'service' ? 'Service Worker' : 'worker'; return `keploy: in ${name}, the ${what} at ${at} draws the browser's own random numbers, ${why === 'service' ? 'so its draws may differ at replay' : KIDS}: ${UNREACHED[why]}`; } const n = Math.floor(Number(others)); @@ -621,6 +626,11 @@ export const DOC_KEY = '__keploy_random_doc'; // off before the request leaves the browser. export const WORKER_PARAM = '__keploy_worker'; +// The BroadcastChannel, by the context's mark, on which the documents of a +// context say the scopes of the Service Worker registrations they make and +// find (initScript). +export const SW_CHANNEL = '__keploy_service_workers_'; + // A browser script's streams: hash, a string's 128 bits (cyrb128), and // stream, sfc32 over them, 32 random bits a call. A replay draws what its // recording drew only while these stay the same. @@ -1217,28 +1227,89 @@ ${DRAW_JS} // Playwright blocks (serviceWorkers: 'block' answers register with no // registration) says nothing. A worker's script request reaches a Service // Worker, which would see keploy's mark: a Web Worker's through its - // document's controller, a Shared Worker's through the Service Worker - // whose scope holds its URL, which keploy takes to be the controller, one - // this document registered or one registered for its origin as it began. - // Such a worker starts as the app asked and says so (servedBy). A Service + // document's controller, a Shared Worker's through the Service Worker of + // the registration whose scope its URL begins with. So a Shared Worker is + // marked only when keploy knows that no registration's scope holds its + // URL; one started before keploy knows, or one a known registration's + // scope holds, starts as the app asked and says so (servedBy), as does a + // worker of a controlled document. keploy knows the registrations of the + // document's origin once the browser answers the getRegistrations it asks + // as the document begins, and keeps them current: each document of the + // context says on keploy's channel the scope of each register it calls + // (before the browser registers it) and of each registration that answer + // finds, and hears what the others say. A context Playwright blocks + // registers none, and keploy knows that as it begins (the context's first + // document finds none: the fixture's context is a new one). A document with + // no Service Workers (an insecure context) has none to know. A Service // Worker is named by its script's address ('' when keploy cannot read it): - // the controller; else the one this document registered last; else, of a - // registration found as it began, the one that answers (active), else the - // one next. - let registered = null; + // the controller; else that of the registration with the longest scope + // that holds the worker's URL: its active one, else its waiting one, else + // its installing one, or the script this document's register named. const swAddress = (w) => { try { return addressOf(new U(String(w.scriptURL))); } catch (e) { return ''; } }; + // The registrations keploy knows of, by scope, and the scopes of those this + // document's getRegistrations found (the browser's own word). + const regs = new Map(); + const found = new Set(); + let known = false; + let channel = null; + let post = null; + const apply = Reflect.apply; + // With no channel, nothing is said. + const say = (m) => { try { apply(post, channel, [m]); } catch (e) {} }; + let realRegister = null; + try { realRegister = g.ServiceWorkerContainer.prototype.register; } catch (e) { realRegister = null; } + try { + if (/Service Worker registration blocked by Playwright/.test(Function.prototype.toString.call(realRegister))) known = true; + } catch (e) { + // No register to read: as the browser's own. + } + if (shared) { + try { + const BC = g.BroadcastChannel; + post = BC.prototype.postMessage; + channel = new BC(${JSON.stringify(SW_CHANNEL)} + mark); + // A message of another shape (another script's on this channel) is + // let be: nothing it holds is read but a string scope and address. + channel.onmessage = (e) => { + const m = e.data; + const scope = m !== null && typeof m === 'object' ? m.scope : undefined; + if (typeof scope !== 'string') return; + if (m.gone === true) { + if (!found.has(scope)) regs.delete(scope); + } else if (typeof m.at === 'string' && !regs.has(scope)) regs.set(scope, m.at); + }; + } catch (e) { + // No channel here: this document knows only its own. + } + } try { const proto = g.ServiceWorkerContainer.prototype; - const realRegister = proto.register; if (typeof realRegister === 'function') { - define(proto, 'register', { register(url) { - const p = realRegister.apply(this, arguments); + define(proto, 'register', { register(url, options) { let at = ''; - try { const d = docNow(); at = addressOf(new U(String(url), d ? d.baseURI : location.href)); } catch (e) { at = ''; } + let scope = ''; + try { + const d = docNow(); + const base = d ? d.baseURI : location.href; + const su = new U(String(url), base); + at = addressOf(su); + const sc = options !== null && typeof options === 'object' && options.scope !== undefined ? new U(String(options.scope), base) : new U('./', su); + sc.hash = ''; + scope = sc.href; + } catch (e) { scope = ''; } + // Known before the browser registers it: here, and in each other + // document of the context once it hears it. + const had = regs.has(scope); + if (!had) { regs.set(scope, at); say({ scope, at }); } + const p = realRegister.apply(this, arguments); try { p.then((r) => { - if (!r) return; - registered = at; + if (!r) { + // Playwright's block registered none: what this register held + // goes, unless the browser found that registration. + if (!found.has(scope)) { regs.delete(scope); say({ scope, gone: true }); } + return; + } if (tell !== null) { const t = tell(at, 'service', 0); if (t && typeof t.catch === 'function') t.catch(() => {}); } }, () => {}); } catch (e) { @@ -1252,17 +1323,39 @@ ${DRAW_JS} // takes no property: its registrations say nothing. } try { - g.navigator.serviceWorker.getRegistrations().then((rs) => { - if (registered === null && rs.length > 0) registered = swAddress(rs[0].active || rs[0].waiting || rs[0].installing); - }, () => {}); + const c = g.navigator.serviceWorker; + if (c === undefined) known = true; + else { + c.getRegistrations().then((rs) => { + for (const r of rs) { + const s = String(r.scope); + found.add(s); + if (!regs.has(s)) { + const a = swAddress(r.active || r.waiting || r.installing); + regs.set(s, a); + say({ scope: s, at: a }); + } + } + known = true; + }, () => {}); + } } catch (e) { - // No Service Worker here (an opaque origin): none can answer. + // No Service Worker can be read here (an opaque origin): keploy does not + // know. } - const servedBy = (name) => { + // The address of the Service Worker that may answer the script of a worker + // at href ('' when keploy cannot read it); null when none may; false for + // a Shared Worker started before keploy knows. + const servedBy = (name, href) => { let c = null; try { c = g.navigator.serviceWorker.controller; } catch (e) { c = null; } if (c) return swAddress(c); - return name === 'SharedWorker' ? registered : null; + if (name !== 'SharedWorker') return null; + let best = ''; + let by = null; + for (const [s, a] of regs) if (href.startsWith(s) && s.length > best.length) { best = s; by = a; } + if (by !== null) return by; + return known ? null : false; }; const wrap = (name, tokenOf) => { const Real = g[name]; @@ -1286,8 +1379,8 @@ ${DRAW_JS} // Its token is taken whoever answers it: the next worker's place does // not hang on when a Service Worker came. let why = u.protocol !== 'http:' && u.protocol !== 'https:' ? 'worker' : tokenOf(u, o); - const by = why !== null && why.length === 32 ? servedBy(name) : null; - if (by !== null) why = 'controlled'; + const by = why !== null && why.length === 32 ? servedBy(name, u.href) : null; + if (by !== null) why = by === false ? 'early' : 'controlled'; if (why !== null && why.length === 32) { const h = u.href; const i = h.indexOf('#'); @@ -1299,7 +1392,7 @@ ${DRAW_JS} } const w = construct(Real, args, new.target); if (why !== '' && tell !== null) { - try { const r = tell(at, why === 'shared' || why === 'controlled' ? why : 'worker', why === 'controlled' ? by : 0); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} + try { const r = tell(at, why === 'shared' || why === 'controlled' || why === 'early' ? why : 'worker', why === 'controlled' ? by : 0); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} } return w; } }[name]; diff --git a/playwright/test/draws.mjs b/playwright/test/draws.mjs index aa32711..0c5a337 100644 --- a/playwright/test/draws.mjs +++ b/playwright/test/draws.mjs @@ -32,7 +32,7 @@ export const BLANK_DRAWS = '({ random: [Math.random(), Math.random()], bytes: Ar * is the window's frameElement, read through a getter. doc is its document * (with URL beside it), for a document whose frames the row makes. opener * is the window's opener (none when undefined). globals: more of the - * window's globals (its URL, Worker, SharedWorker). + * window's globals (its URL, Worker, SharedWorker, navigator). */ export const OTHER = Symbol('a parent of another origin'); export function documentAt(script, path = '/p', storage = new Map(), page = 't0', { report = '', told = [], origin = 'http://h', href = undefined, parent = undefined, doc = undefined, frames = undefined, element = undefined, opener = undefined, globals = {} } = {}) { @@ -41,7 +41,9 @@ export function documentAt(script, path = '/p', storage = new Map(), page = 't0' getRandomValues(a) { if (!(this instanceof Crypto)) throw new TypeError('Illegal invocation'); return webcrypto.getRandomValues(a); } randomUUID() { if (!(this instanceof Crypto)) throw new TypeError('Illegal invocation'); return webcrypto.randomUUID(); } } - const ctx = { location: { origin, pathname: path, href: href ?? `${origin}${path}` }, Crypto, crypto: new Crypto() }; + // Its navigator has no Service Workers, as an insecure context's, unless + // globals give one. + const ctx = { location: { origin, pathname: path, href: href ?? `${origin}${path}` }, Crypto, crypto: new Crypto(), navigator: {} }; if (storage === null) { Object.defineProperty(ctx, 'sessionStorage', { get() { throw new Error('SecurityError: no session storage here'); } }); } else { diff --git a/playwright/test/fixtures/nested.spec.ts b/playwright/test/fixtures/nested.spec.ts index e0177cb..be8f6c8 100644 --- a/playwright/test/fixtures/nested.spec.ts +++ b/playwright/test/fixtures/nested.spec.ts @@ -10,6 +10,11 @@ import { test } from '../../dist'; // keeps them, under its title. const base = process.env.DRAWS_BASE ?? ''; +// The page starts its Shared Workers as it loads, before the browser can +// answer which Service Workers its origin has: with Playwright blocking +// them, keploy knows at once that none can answer, and seeds them. +test.use({ serviceWorkers: 'block' }); + test('nested', async ({ page }) => { await page.goto(`${base}/nw`); await page.waitForFunction(() => { diff --git a/playwright/test/fixtures/service-known.spec.ts b/playwright/test/fixtures/service-known.spec.ts new file mode 100644 index 0000000..3af7cbd --- /dev/null +++ b/playwright/test/fixtures/service-known.spec.ts @@ -0,0 +1,73 @@ +import fs from 'node:fs'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). Each test starts the +// Shared Worker /swk/s.js (or /swr/s.js), which sends what it drew and its +// location, and keeps what it got under its title: +// - 'shared worker early': a script of the test's own, run as the page's +// document begins (after keploy's, before the browser can answer which +// Service Workers the origin has), starts it. +// - 'shared worker announced': the page b, open before the page a registers +// the Service Worker /swr/sw.js (scope /swr/, which b's own address is +// not in), starts /swr/s.js once that Service Worker is active; the Service +// Worker then sends the URL of each request it was asked. +// - 'shared worker known': the page starts it once the browser has answered +// that its origin has no Service Worker. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +type Took = { __w: unknown[] }; + +test('shared worker early', async ({ page }) => { + await page.context().addInitScript(() => { + if (location.pathname !== '/long/early') return; + const w: unknown[] = []; + (window as unknown as Took).__w = w; + const s = new SharedWorker('/swk/s.js'); + s.port.onmessage = (e) => w.push(e.data); + s.port.start(); + }); + await page.goto(`${base}/long/early`); + await page.waitForFunction(() => (window as unknown as Took).__w.length >= 1, null, { timeout: 20000 }); + keep(test.info().title, { w: await page.evaluate(() => (window as unknown as Took).__w) }); +}); + +test('shared worker announced', async ({ context }) => { + const b = await context.newPage(); + await b.goto(`${base}/long/b`); + await b.evaluate(() => navigator.serviceWorker.getRegistrations()); + const a = await context.newPage(); + await a.goto(`${base}/swr/p`); + await a.evaluate(async () => { + await navigator.serviceWorker.register('/swr/sw.js'); + await navigator.serviceWorker.ready; + }); + const got = await b.evaluate(() => new Promise((ok) => { + const s = new SharedWorker('/swr/s.js'); + s.port.onmessage = (e) => ok(e.data); + s.port.start(); + })); + const seen = await a.evaluate(async () => { + const r = await navigator.serviceWorker.getRegistration('/swr/'); + const asked = r?.active; + if (!asked) return null; + return new Promise((ok) => { + navigator.serviceWorker.addEventListener('message', (e) => ok(e.data), { once: true }); + asked.postMessage('seen'); + }); + }); + keep(test.info().title, { w: [got], seen }); +}); + +test('shared worker known', async ({ page }) => { + await page.goto(`${base}/long/known`); + const got = await page.evaluate(async () => { + await navigator.serviceWorker.getRegistrations(); + return new Promise((ok) => { + const s = new SharedWorker('/swk/s.js'); + s.port.onmessage = (e) => ok(e.data); + s.port.start(); + }); + }); + keep(test.info().title, { w: [got] }); +}); diff --git a/playwright/test/fixtures/workers.spec.ts b/playwright/test/fixtures/workers.spec.ts index f069679..3684a4d 100644 --- a/playwright/test/fixtures/workers.spec.ts +++ b/playwright/test/fixtures/workers.spec.ts @@ -14,6 +14,11 @@ import { test } from '../../dist'; const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +// The page starts its Shared Workers as it loads, before the browser can +// answer which Service Workers its origin has: with Playwright blocking +// them, keploy knows at once that none can answer, and seeds them. +test.use({ serviceWorkers: 'block' }); + const workers = async (page: Page): Promise => { await page.goto(`${base}/wk`); await page.waitForFunction(() => (window as unknown as { __w: unknown[] }).__w.length >= 17, null, { timeout: 20000 }); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index dd0ea7b..263063f 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, markWorker, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, markWorker, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -1463,6 +1463,11 @@ test('the lines for a worker keploy cannot reach say what it is, that each worke 'record and replay the test in Chromium or WebKit to seed it.'); assert.equal(unplacedLine('t', 'http://h/b.js', 'nested'), 'keploy: in t, the worker at http://h/b.js draws the browser\'s own random numbers, ' + KIDS + 'it is a worker that a worker started, and Chromium and Firefox do not let keploy reach its script to put its seeding ahead of it; record and replay the test in WebKit to seed it.'); + assert.equal(unplacedLine('t', 'http://h/s.js', 'early'), 'keploy: in t, the Shared Worker at http://h/s.js draws the browser\'s own random numbers, ' + KIDS + + 'it is a Shared Worker started before the browser told keploy which Service Workers the page\'s origin has (keploy asks as each document begins, ' + + 'and a document that cannot read its Service Workers, as a sandboxed one, is never told), and a Service Worker whose scope holds the worker\'s URL ' + + 'would see keploy\'s __keploy_worker mark in the script\'s URL, so keploy starts the worker as the app asked; if the tests do not need it, ' + + 'set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered).'); assert.match(unplacedLine('t', 'http://h/w.js', 'routed'),/^keploy: in t, the worker at http:\/\/h\/w\.js draws the browser's own random numbers, and so does each worker it starts, so their draws may differ at replay: a route of the test's own /); }); @@ -1509,7 +1514,7 @@ test('a Service Worker a document registers is named once the browser registers assert.deepEqual(await d.run('navigator.serviceWorker.register("/sw.js")'), { active: { scriptURL: 'http://h/sw.js' } }); }); -test('a worker whose script a Service Worker may answer starts as the app asked, unmarked, and is named with that Service Worker: a Web Worker of a controlled document, a Shared Worker of a document that is controlled, registered one or found one registered as it began; the next worker at its path keeps its place', async () => { +test('a worker whose script a Service Worker may answer starts as the app asked, unmarked, and is named with that Service Worker: a Web Worker of a controlled document, a Shared Worker of a document that is controlled, or whose URL the scope of a registration it made or found as it began holds; the next worker at its path keeps its place', async () => { const run = async (sw, script, before = '') => { const told = []; const w = workerGlobals(); @@ -1531,13 +1536,13 @@ test('a worker whose script a Service Worker may answer starts as the app asked, assert.deepEqual(r.made[1], ['/s.js', null]); assert.deepEqual(r.told, [['http://h/sw.js', 'service'], ['http://h/s.js', 'controlled', 'http://h/sw.js']]); // A registration of its origin found as the document began. - const f = await run(swGlobals({ regs: [{ active: null, waiting: null, installing: { scriptURL: 'http://h/old.js?x' } }] }), 'new Worker("/w.js"); new SharedWorker("/s.js");'); + const f = await run(swGlobals({ regs: [{ scope: 'http://h/', active: null, waiting: null, installing: { scriptURL: 'http://h/old.js?x' } }] }), 'new Worker("/w.js"); new SharedWorker("/s.js");'); assert.equal(f.made[0][1].kind, 'w'); assert.deepEqual([f.made[1], f.told], [['/s.js', null], [['http://h/s.js', 'controlled', 'http://h/old.js']]]); // A registration found is named by its active Service Worker, else its // waiting one, else its installing one. const sws = { a: { scriptURL: 'http://h/a.js' }, b: { scriptURL: 'http://h/b.js' }, c: { scriptURL: 'http://h/c.js' } }; - for (const [reg, named] of [[{ active: sws.a, waiting: sws.b, installing: sws.c }, 'http://h/a.js'], [{ active: null, waiting: sws.b, installing: sws.c }, 'http://h/b.js']]) { + for (const [reg, named] of [[{ scope: 'http://h/', active: sws.a, waiting: sws.b, installing: sws.c }, 'http://h/a.js'], [{ scope: 'http://h/', active: null, waiting: sws.b, installing: sws.c }, 'http://h/b.js'], [{ scope: 'http://h/', active: null, waiting: null, installing: sws.c }, 'http://h/c.js']]) { assert.deepEqual((await run(swGlobals({ regs: [reg] }), 'new SharedWorker("/s.js");')).told, [['http://h/s.js', 'controlled', named]]); } // The one this document registered is named, even when one found as @@ -1545,7 +1550,7 @@ test('a worker whose script a Service Worker may answer starts as the app asked, { const told = []; const w = workerGlobals(); - const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...swGlobals({ regs: [{ active: sws.a }], late: true }).globals } }); + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...swGlobals({ regs: [{ scope: 'http://h/', active: sws.a }], late: true }).globals } }); await d.run('navigator.serviceWorker.register("/sw.js")'); await new Promise((r) => setTimeout(r, 60)); d.run('new SharedWorker("/s.js");'); @@ -1578,6 +1583,149 @@ test('a worker whose script a Service Worker may answer starts as the app asked, assert.deepEqual([workerMark(w.made[0][0]), workerMark(w.made[1][0])], [null, plain.made[1][1]]); }); +// The BroadcastChannels of one browser context: a message reaches each other +// channel of its name, a turn later, as a structured clone. +const channels = () => { + const all = []; + // What a handler threw: a page's error event would see it. + const thrown = []; + class BroadcastChannel { + constructor(name) { this.name = String(name); this.onmessage = null; all.push(this); } + postMessage(m) { + const d = m === undefined ? undefined : JSON.parse(JSON.stringify(m)); + for (const c of all) { + if (c !== this && c.name === this.name) setImmediate(() => { try { if (c.onmessage) c.onmessage({ data: d }); } catch (e) { thrown.push(e); } }); + } + } + } + return { BroadcastChannel, all, thrown }; +}; + +test('a Shared Worker is marked only once keploy knows no registration\'s scope holds its URL: one started before the browser answers, or where it never does, starts as the app asked and is named; the longest scope that holds it names its Service Worker; one with no Service Workers, or in a context Playwright blocks, is marked at once', async () => { + const start = (sw, extra = {}) => { + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...sw.globals, ...extra } }); + return { d, told, made: () => w.made.map(([u]) => [u, workerMark(u)]) }; + }; + // Before the answer: unmarked, named; once it has come with none, the next + // is marked, and a Web Worker is marked either way. + const e = start(swGlobals({ late: true })); + e.d.run('new SharedWorker("/s.js"); new Worker("/w.js");'); + await new Promise((r) => setTimeout(r, 60)); + e.d.run('new SharedWorker("/s.js");'); + assert.deepEqual(e.made().map(([u, m]) => [m ? m.url : u, m && m.kind]), [['/s.js', null], ['http://h/w.js', 'w'], ['http://h/s.js', 's']]); + assert.deepEqual(e.told, [['http://h/s.js', 'early']]); + // A browser that never answers (getRegistrations refused), or a document + // that cannot read its Service Workers: never known. + const refused = swGlobals(); + refused.globals.ServiceWorkerContainer.prototype.getRegistrations = () => Promise.reject(new Error('SecurityError')); + const r = start(refused); + await turn(); + r.d.run('new SharedWorker("/s.js");'); + assert.deepEqual([r.made(), r.told], [[['/s.js', null]], [['http://h/s.js', 'early']]]); + const o = start({ globals: { navigator: Object.defineProperty({}, 'serviceWorker', { get() { throw new Error('SecurityError: sandboxed'); } }) } }); + await turn(); + o.d.run('new SharedWorker("/s.js");'); + assert.deepEqual([o.made(), o.told], [[['/s.js', null]], [['http://h/s.js', 'early']]]); + // No Service Workers here (an insecure context): marked at once. + const n = start({ globals: { navigator: {} } }); + n.d.run('new SharedWorker("/s.js");'); + assert.deepEqual([n.made()[0][1].kind, n.told], ['s', []]); + // Scopes: one that does not hold the URL leaves it marked; of two that + // do, the longest names the Service Worker. + const regs = [{ scope: 'http://h/a/b/', active: { scriptURL: 'http://h/ab.js' } }, { scope: 'http://h/a/', active: { scriptURL: 'http://h/a/sw.js' } }]; + const s = start(swGlobals({ regs })); + await turn(); + s.d.run('new SharedWorker("/s.js"); new SharedWorker("/a/s.js"); new SharedWorker("/a/b/s.js"); new SharedWorker("/a/bs.js");'); + assert.deepEqual(s.made().map(([u, m]) => [m ? m.url : u, m && m.kind]), [['http://h/s.js', 's'], ['/a/s.js', null], ['/a/b/s.js', null], ['/a/bs.js', null]]); + assert.deepEqual(s.told, [['http://h/a/s.js', 'controlled', 'http://h/a/sw.js'], ['http://h/a/b/s.js', 'controlled', 'http://h/ab.js'], ['http://h/a/bs.js', 'controlled', 'http://h/a/sw.js']]); + // A register of this document's own, before the answer, at the scope it + // names (else its script's directory): known at once. + const own = start(swGlobals({ late: true })); + own.d.run('const c = navigator.serviceWorker; c.register("/x/sw.js"); c.register("/sw2.js", { scope: "/y/#f" }); c.register("/v/sw.js", { type: "module" }); c.register("/n/sw.js", null); ' + + 'new SharedWorker("/x/s.js"); new SharedWorker("/y/s.js"); new SharedWorker("/v/s.js"); new SharedWorker("/n/s.js"); new SharedWorker("/z/s.js");'); + assert.deepEqual(own.made().map(([, m]) => m), [null, null, null, null, null]); + assert.deepEqual(own.told.slice(0, 5), [['http://h/x/s.js', 'controlled', 'http://h/x/sw.js'], ['http://h/y/s.js', 'controlled', 'http://h/sw2.js'], + ['http://h/v/s.js', 'controlled', 'http://h/v/sw.js'], ['http://h/n/s.js', 'controlled', 'http://h/n/sw.js'], ['http://h/z/s.js', 'early']]); + // A register at a URL keploy cannot read holds no scope that can hold a worker's URL. + const bad = start(swGlobals()); + await turn(); + bad.d.run('navigator.serviceWorker.register("http://["); new SharedWorker("/s.js");'); + assert.equal(bad.made()[0][1].kind, 's'); + // Playwright's block (its own register, which registers none): known at + // once, so marked before the answer; its register says nothing, and its + // scope is not kept. + const blocked = swGlobals({ late: true }); + blocked.globals.ServiceWorkerContainer.prototype.register = async () => { void 'Service Worker registration blocked by Playwright'; }; + const b = start(blocked); + b.d.run('new SharedWorker("/s.js"); navigator.serviceWorker.register("/sw.js");'); + await turn(); + b.d.run('new SharedWorker("/t.js");'); + assert.deepEqual([b.made().map(([, m]) => m.kind), b.told], [['s', 's'], []]); + // A register that answers none after the browser found a registration + // at its scope: that one is kept. + const slow = swGlobals({ late: true, regs: [{ scope: 'http://h/', active: { scriptURL: 'http://h/old.js' } }] }); + slow.globals.ServiceWorkerContainer.prototype.register = () => new Promise((ok) => setTimeout(() => ok(undefined), 40)); + const k = start(slow); + k.d.run('navigator.serviceWorker.register("/sw.js");'); + await new Promise((r) => setTimeout(r, 80)); + k.d.run('new SharedWorker("/s.js");'); + assert.deepEqual([k.made(), k.told], [[['/s.js', null]], [['http://h/s.js', 'controlled', 'http://h/sw.js']]]); +}); + +test('the documents of a context say on keploy\'s channel each registration they make or find: another document\'s next Shared Worker that its scope holds starts as the app asked and is named with it; one Playwright\'s block answered with none is taken back, but not one a document found; a message keploy did not make changes nothing', async () => { + const bus = channels(); + const doc = (sw) => { + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell', 'm1'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...sw.globals, BroadcastChannel: bus.BroadcastChannel } }); + return { d, told, made: () => w.made.map(([u]) => workerMark(u)) }; + }; + const a = doc(swGlobals()); + const b = doc(swGlobals()); + await turn(); + a.d.run('navigator.serviceWorker.register("/app/sw.js");'); + await turn(); + b.d.run('new SharedWorker("/app/s.js"); new SharedWorker("/s.js");'); + assert.deepEqual(b.made().map((m) => m && m.kind), [null, 's']); + assert.deepEqual(b.told, [['http://h/app/s.js', 'controlled', 'http://h/app/sw.js']]); + assert.deepEqual(bus.all.map((c) => c.name), [`${SW_CHANNEL}m1`, `${SW_CHANNEL}m1`]); + // A registration a document finds as it begins is said to the others. + const c = doc(swGlobals({ regs: [{ scope: 'http://h/f/', active: { scriptURL: 'http://h/f/sw.js' } }] })); + await turn(); await turn(); + a.d.run('new SharedWorker("/f/s.js");'); + assert.deepEqual(a.told, [['http://h/app/sw.js', 'service'], ['http://h/f/s.js', 'controlled', 'http://h/f/sw.js']]); + // Playwright's block: a register said, then taken back once it answers + // none; c keeps the scope it found. + const blocked = swGlobals(); + blocked.globals.ServiceWorkerContainer.prototype.register = async () => undefined; + const p = doc(blocked); + await turn(); + p.d.run('navigator.serviceWorker.register("/q/sw.js"); navigator.serviceWorker.register("/f/sw.js");'); + await turn(); + b.d.run('new SharedWorker("/q/s.js");'); + assert.equal(b.made()[2], null); + await turn(); await turn(); + b.d.run('new SharedWorker("/q/s.js");'); + c.d.run('new SharedWorker("/f/s.js");'); + assert.equal(b.made()[3].kind, 's'); + assert.deepEqual([c.made(), c.told], [[null], [['http://h/f/s.js', 'controlled', 'http://h/f/sw.js']]]); + // Another script's messages on the channel, of any other shape, and one + // at a scope already known: nothing changes, and nothing throws. + const page = new bus.BroadcastChannel(`${SW_CHANNEL}m1`); + for (const m of [null, undefined, 1, 'x', { scope: ['http://h/'], at: 'http://h/x.js' }, { scope: 'http://h/', at: 2 }, { scope: 'http://h/app/', gone: 'yes' }, { scope: 'http://h/app/', at: 'http://h/other.js' }]) page.postMessage(m); + await turn(); + b.d.run('new SharedWorker("/s.js"); new SharedWorker("/app/t.js");'); + assert.deepEqual(b.made().slice(4).map((m) => m && m.kind), ['s', null]); + assert.deepEqual(b.told.at(-1), ['http://h/app/t.js', 'controlled', 'http://h/app/sw.js']); + assert.deepEqual(bus.thrown, []); + // Where Shared Workers are not marked (Firefox), no channel opens. + const before = bus.all.length; + documentAt(initScript(S1, 'tell', 'm1', false), '/p', new Map(), 't0', { report: 'tell', told: [], globals: { ...workerGlobals().globals, ...swGlobals().globals, BroadcastChannel: bus.BroadcastChannel } }); + assert.equal(bus.all.length, before); +}); + test('a worker\'s seeding draws the same numbers for a token and instance every run, others for another; its location shows no mark, its own query and fragment kept', () => { const t = 'cd'.repeat(16); const a = workerAt(markWorker('http://h/w.js', 'w', t), workerScript(t, 0, 'w')).draws(); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index affed4d..0c55e0b 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -678,6 +678,11 @@ async function drawsServer() { 'self.addEventListener(\'fetch\', (e) => { seen.push(e.request.url); }); self.addEventListener(\'message\', (e) => e.source.postMessage({ kind: \'sw-seen\', seen }));', '/swk/w.js': `postMessage({ kind: 'sw-ded', href: location.href, draws: ${DRAWS} });`, '/swk/s.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'sw-shared', href: location.href, draws: ${DRAWS} });`, + // service-known.spec.ts's: /swr/sw.js (scope /swr/) keeps the URL of + // each request it is asked and answers a message with them; /swr/s.js + // is a Shared Worker its scope holds. + '/swr/sw.js': 'const seen = []; self.addEventListener(\'fetch\', (e) => { seen.push(e.request.url); }); self.addEventListener(\'message\', (e) => e.source.postMessage({ kind: \'sw-seen\', seen }));', + '/swr/s.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'swr-shared', href: location.href, draws: ${DRAWS} });`, }[u.pathname]; if (sw !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end(sw); return; } res.writeHead(200, { 'content-type': 'text/html' }); @@ -705,7 +710,7 @@ async function drawsServer() { // /nest holds /frame of its own origin. if (u.pathname === '/nest') { res.end(page('')); return; } // /long/... is a page at an address of any length. - if (u.pathname.startsWith('/long/')) { res.end(page('')); return; } + if (u.pathname.startsWith('/long/') || u.pathname === '/swr/p') { res.end(page('')); return; } // places.spec.ts's: /host puts in a (/fa, its own origin), c (/fc) and // w (/fw, both cross-site) in the order ?order= names; /race holds r0 // and r1 at /fr, and /fr answers late the one ?slow= names; /nested @@ -1645,6 +1650,62 @@ for (const engine of ENGINES) { }); } +// service-known.spec.ts's tests. +const KNOWN_TITLES = ['shared worker early', 'shared worker announced', 'shared worker known']; + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a Shared Worker is marked only once keploy knows no Service Worker's scope holds its URL: one started before the browser answers starts as the app asked and is named; one whose URL the scope of a registration another page made holds starts as the app asked, so that Service Worker sees no mark, and is named with it; one started once the browser said there is none draws from the test's seed, the same at replay (in Firefox, whose route does not reach a Shared Worker's script, each is named for that)`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + let kept = {}; + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', ...(kept[c.body.name] ? { seed: kept[c.body.name] } : {}) } } : undefined)); + const env = (url) => ({ ...keployEnv(url), PW_BROWSER: engine }); + const specs = ['service-known.spec.ts']; + try { + const r = await drawRun(j, env(rec.url), specs, KNOWN_TITLES); + const facts = JSON.stringify(r.drew); + const said = (title) => r.tests[title].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, `)).map((l) => l.replace(/127\.0\.0\.1:\d+\//g, '127.0.0.1:/')).sort(); + const at = (p) => `http://127.0.0.1:/${p}`; + const ff = engine === 'firefox'; + const early = r.drew['shared worker early'].w[0]; + const origin = new URL(early.href).origin; + // Early: as the app asked, named for that. + assert.equal(early.href, `${origin}/swk/s.js`, facts); + assert.deepEqual(said('shared worker early'), [ + unplacedLine('shared worker early', at('swk/s.js'), ff ? 'shared' : 'early'), + sharedWorkerLine('shared worker early', at('swk/s.js')), + ].sort(), facts); + // Announced: named with the Service Worker the other page registered, + // which sees no mark. + const ann = r.drew['shared worker announced']; + assert.equal(ann.w[0].href, `${origin}/swr/s.js`, facts); + assert.deepEqual(said('shared worker announced'), [ + unplacedLine('shared worker announced', at('swr/sw.js'), 'service'), + ff ? unplacedLine('shared worker announced', at('swr/s.js'), 'shared') : unplacedLine('shared worker announced', at('swr/s.js'), 'controlled', 0, at('swr/sw.js')), + sharedWorkerLine('shared worker announced', at('swr/s.js')), + ].sort(), facts); + assert.equal(ann.seen.kind, 'sw-seen', facts); + assert.deepEqual(ann.seen.seen.filter((u) => u.includes('__keploy_worker')), [], facts); + // Known: seeded (Chromium, WebKit), and only the settle line is said. + const known = r.drew['shared worker known'].w[0]; + assert.equal(known.href, `${origin}/swk/s.js`, facts); + assert.deepEqual(said('shared worker known'), [ + ...(ff ? [unplacedLine('shared worker known', at('swk/s.js'), 'shared')] : []), + sharedWorkerLine('shared worker known', at('swk/s.js')), + ].sort(), facts); + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); + kept = r.seeds; + const q = await drawRun(j, env(rep.url), specs, KNOWN_TITLES); + const again = q.drew['shared worker known'].w[0]; + if (ff) assert.notDeepEqual(again.draws, known.draws, facts); + else assert.deepEqual(again.draws, known.draws, facts); + // The early one is not seeded: it draws other numbers at replay. + assert.notDeepEqual(q.drew['shared worker early'].w[0].draws, early.draws, facts); + } finally { await rep.close(); await rec.close(); } + }); +} + test('seeded: keploy answers no seed (an older recording, or an older keploy): the browser draws its own, each test says so, and its end says own', async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok' } } : undefined)); From 8789b88694b2345a654980a2173d17f986accd0f Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 21:56:41 +0530 Subject: [PATCH 38/73] test(playwright): settle rows no longer race a 4 s test timeout The row for a call a new document makes as it loads ended its test by a 4 s test timeout while settle waited. That timeout bounds the test's body too, and on a loaded machine Firefox's goto alone took up to 3.6 s, so the timeout could fire while the document's own request was still in flight: settle then named that request instead of the image. Holding the document's answer 5 s reproduced it in each run. Firefox reported the events in the same order in each of 200 runs, quiet and loaded, so settle itself was right. The row now answers the call once settle's step begins (the rows' reporter asks the page server then), with the config's 30 s timeout: the test ends by settle's wait alone and passes, the step names the image alone, and the call's answer comes before the test's end. The Shared Worker row, which checks that settle does not wait, drops its 4 s timeout too: a wait shows as settle's step. Signed-off-by: slayerjain --- playwright/test/fixtures/steps-reporter.cjs | 6 +++++- playwright/test/rows.test.mjs | 19 ++++++++++++++----- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/playwright/test/fixtures/steps-reporter.cjs b/playwright/test/fixtures/steps-reporter.cjs index a53076f..d455518 100644 --- a/playwright/test/fixtures/steps-reporter.cjs +++ b/playwright/test/fixtures/steps-reporter.cjs @@ -3,13 +3,17 @@ // JSON line {test, step, at} per step begun, appended to STEPS_FILE; at is // the step's file name and line, '' for none. And one line {test, out} for // each write to a test's stdout, in the same file, so that a row can tell -// whether a test's line came before or after one of its steps. +// whether a test's line came before or after one of its steps. With +// RELEASE_AT_STEP set (a URL), it asks that URL once settle's step begins +// (keploy's "wait for the test's ... in flight" step), so a row can answer a +// call only once settle waits on it, however long the test's body took. const fs = require('node:fs'); const path = require('node:path'); class StepsReporter { onStepBegin(test, _result, step) { if (process.env.STEPS_FILE) fs.appendFileSync(process.env.STEPS_FILE, `${JSON.stringify({ test: test.title, step: step.title, at: step.location ? `${path.basename(step.location.file)}:${step.location.line}` : '' })}\n`); + if (process.env.RELEASE_AT_STEP && step.title.startsWith("keploy: wait for the test's ")) void fetch(process.env.RELEASE_AT_STEP).catch(() => {}); } onStdOut(chunk, test) { diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 0c55e0b..3a96eab 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -396,7 +396,11 @@ test('settle: a call cancelled by its page loading another document is not waite // Firefox can report the new document's first call before the frame's // commit; the commit cancels only the old document's calls, so this one is -// still waited for, in every engine. +// still waited for, in every engine. The call is answered only once settle's +// step has begun (the rows' reporter asks /release then), so the test ends +// by settle's wait alone: no deadline bounds its body, which on a loaded box +// can take seconds in Firefox (a 4 s test timeout once fired while the +// document's own request was in flight, and settle named that one). for (const engine of ['chromium', 'firefox', 'webkit']) { test(`settle, in ${engine}: a call the new document makes as it loads is waited for, whichever the browser reports first, the call or its frame's commit`, async () => { const j = journal(); @@ -404,18 +408,23 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { const p = await pageServer(j); try { const title = 'a call its new document makes as it loads'; - const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000', PW_BROWSER: engine }); + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', RELEASE_AT_STEP: `${p.url}/release`, PW_BROWSER: engine }); const t = tests[title]; - assert.equal(t.status, 'timedOut', out); + assert.equal(t.status, 'passed', out); const named = /^keploy: wait for the test's 1 call\(s\) in flight before its browser context closes: GET http:\/\/127\.0\.0\.1:\d+\/slow$/; assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n')); + // The test ended after the call's answer: waited for, not cut. + const order = j.events.map((e) => (e.page ? e.page : e.agent.path)); + assert.deepEqual(order.filter((e) => e.startsWith('slow') || e === '/agent/scope/end'), ['slow sent', 'slow answered', '/agent/scope/end']); } finally { await p.close(); await a.close(); } }); } // A Shared Worker's script: Chromium reports no end for it (Playwright does // not follow Shared Workers), so settle never waits on it, in any engine, -// and the test says so, seeded or not. +// and the test says so, seeded or not. A wait on it shows as settle's step +// (and a timeout at the config's 30 s): the row sets no short timeout, which +// would bound the test's body too, slow in Firefox on a loaded box. for (const engine of ['chromium', 'firefox', 'webkit']) { test(`settle, in ${engine}: a test whose page starts a Shared Worker ends at once, its browser unseeded, and says once that the worker's script and calls are not waited for`, async () => { const j = journal(); @@ -423,7 +432,7 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { const p = await pageServer(j); try { const title = 'a page that starts a Shared Worker'; - const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'off', SLOW_BASE: p.url, PW_TIMEOUT: '4000', PW_BROWSER: engine }); + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), KEPLOY_SEED_RANDOM: 'off', SLOW_BASE: p.url, PW_BROWSER: engine }); const t = tests[title]; assert.equal(t.status, 'passed', out); assert.ok(!t.steps.some((s) => s.startsWith('keploy: wait for')), t.steps.join('\n')); From 6dc2735f997aa776dfef588e80728adf385eef73 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 22:09:01 +0530 Subject: [PATCH 39/73] docs(playwright): say what keploy's route does to the browser cache A seeded test's requests go through keploy's context route, and while a context has a route every engine asks the server for each request rather than answering it from its cache, not only Chromium as the README said. Measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), with no route and with a catch-all context route, each as a record and then a replay: the app's server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same calls at record and at replay in each engine. So a replay asks the app's server for what its recording asked, and the route stays on seeded tests. Signed-off-by: slayerjain --- playwright/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/playwright/README.md b/playwright/README.md index 4f90353..2db9129 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes (in Chromium the browser's cache is off while a context has one, as for any `context.route`). keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), or in a page that cannot read its Service Workers (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), or in a page that cannot read its Service Workers (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); From 5b43fc9edc25111c9210f0219bb68c798359b817 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 22:59:57 +0530 Subject: [PATCH 40/73] fix(playwright): a page begun mid-register leaves its Shared Workers be A page that began while another page of its origin had a Service Worker register in flight could miss that registration: its getRegistrations could answer before the browser held the registration, and the other page had said the register's scope on keploy's channel before this page opened its own. Its next Shared Worker was then marked, and the Service Worker that register installed could see keploy's mark. Each register now holds keploy's Web Lock (keploy-sw-register, shared) from before the browser registers it until it settles, however it ends. A document reads the locks as it begins and asks for its registrations only once the browser has said that no lock of that name is held or asked for. In that order a register is known whenever it began: one that settled before the read is among the registrations, one in flight holds the lock, and one that began after the read is said on the channel the document opened before it. A document that finds the lock held, whose lock read is refused, or that has no Web Locks never knows: its Shared Workers start as the app asked and are named with the line whose next step is serviceWorkers: 'block', which now says this case too. Rows: the lock's order against the channel and its mode, its release on every end of a register, a document begun mid-register, after it, with the lock asked for, refused, malformed or absent, and with another lock; a browser row per engine where a page opened while another page's register waits on its script starts its Shared Worker as the app asked and is named, after its own lock read and getRegistrations answered. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/random.ts | 97 +++++++++--- .../test/fixtures/service-known.spec.ts | 33 +++- playwright/test/random.test.mjs | 146 +++++++++++++++++- playwright/test/rows.test.mjs | 33 +++- 5 files changed, 276 insertions(+), 35 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 2db9129..b84a514 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), or in a page that cannot read its Service Workers (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 8184891..dad00db 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -126,7 +126,8 @@ const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'early' | ' service: 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + `${BLOCK}.`, early: 'it is a Shared Worker started before the browser told keploy which Service Workers the page\'s origin has (keploy asks as each document begins, ' + - 'and a document that cannot read its Service Workers, as a sandboxed one, is never told), and a Service Worker whose scope holds the worker\'s URL ' + + 'and a document that begins while a document of the origin is registering a Service Worker, or that cannot read its Service Workers ' + + 'or its Web Locks, as a sandboxed one, is never told), and a Service Worker whose scope holds the worker\'s URL ' + `would see keploy's __keploy_worker mark in the script's URL, so keploy starts the worker as the app asked; ${BLOCK}.`, nested: 'it is a worker that a worker started, and Chromium and Firefox do not let keploy reach its script to put its seeding ahead of it; ' + 'record and replay the test in WebKit to seed it.', @@ -631,6 +632,11 @@ export const WORKER_PARAM = '__keploy_worker'; // find (initScript). export const SW_CHANNEL = '__keploy_service_workers_'; +// The Web Lock each register of a document holds, shared, from before the +// browser registers it until it settles, so that a document of the origin +// that begins meanwhile knows a registration may be coming (initScript). +export const SW_LOCK = 'keploy-sw-register'; + // A browser script's streams: hash, a string's 128 bits (cyrb128), and // stream, sfc32 over them, 32 random bits a call. A replay draws what its // recording drew only while these stay the same. @@ -1233,11 +1239,20 @@ ${DRAW_JS} // URL; one started before keploy knows, or one a known registration's // scope holds, starts as the app asked and says so (servedBy), as does a // worker of a controlled document. keploy knows the registrations of the - // document's origin once the browser answers the getRegistrations it asks - // as the document begins, and keeps them current: each document of the - // context says on keploy's channel the scope of each register it calls - // (before the browser registers it) and of each registration that answer - // finds, and hears what the others say. A context Playwright blocks + // document's origin once, as the document begins, the browser has said that + // no register of the origin's documents is in flight (none holds or asks for + // keploy's Web Lock, SW_LOCK, which each register takes before the browser + // registers it and lets go once it settles) and then, asked only after that, + // which registrations the origin has. In that order a register is known + // whenever it began: one that settled before the lock was read is among the + // registrations, one in flight holds the lock, and one that began after the + // lock was read is said on keploy's channel, which this document opened + // before it read the lock. A document that finds the lock held, or that has + // no Web Locks, never knows (its next document asks again). keploy keeps + // what it knows current: each document of the context says on keploy's + // channel the scope of each register it calls (before the browser + // registers it) and of each registration that answer finds, and hears + // what the others say. A context Playwright blocks // registers none, and keploy knows that as it begins (the context's first // document finds none: the fixture's context is a new one). A document with // no Service Workers (an insecure context) has none to know. A Service @@ -1258,6 +1273,30 @@ ${DRAW_JS} const say = (m) => { try { apply(post, channel, [m]); } catch (e) {} }; let realRegister = null; try { realRegister = g.ServiceWorkerContainer.prototype.register; } catch (e) { realRegister = null; } + // The document's Web Locks, as the browser gave them (null: none). + let locks = null; + let lockRequest = null; + let lockQuery = null; + try { + locks = g.navigator.locks; + lockRequest = locks.request; + lockQuery = locks.query; + } catch (e) { locks = null; } + const LOCK = ${JSON.stringify(SW_LOCK)}; + // Takes keploy's lock for a register; what it gives lets it go. With no + // Web Locks it takes none, and the documents that begin meanwhile never + // know either way. + const hold = () => { + let free = () => {}; + try { + const settled = new Promise((ok) => { free = ok; }); + // A request the browser refuses says nothing to the app. + apply(lockRequest, locks, [LOCK, { mode: 'shared' }, () => settled]).catch(() => {}); + } catch (e) { + // No lock to take. + } + return () => { free(); }; + }; try { if (/Service Worker registration blocked by Playwright/.test(Function.prototype.toString.call(realRegister))) known = true; } catch (e) { @@ -1298,12 +1337,16 @@ ${DRAW_JS} scope = sc.href; } catch (e) { scope = ''; } // Known before the browser registers it: here, and in each other - // document of the context once it hears it. + // document of the context once it hears it; the lock first, so that + // a document that begins after it is said finds the lock held. + const free = hold(); const had = regs.has(scope); if (!had) { regs.set(scope, at); say({ scope, at }); } - const p = realRegister.apply(this, arguments); + let p; + try { p = realRegister.apply(this, arguments); } catch (e) { free(); throw e; } try { p.then((r) => { + free(); if (!r) { // Playwright's block registered none: what this register held // goes, unless the browser found that registration. @@ -1311,9 +1354,11 @@ ${DRAW_JS} return; } if (tell !== null) { const t = tell(at, 'service', 0); if (t && typeof t.catch === 'function') t.catch(() => {}); } - }, () => {}); + }, () => { free(); }); } catch (e) { - // A register that returned no promise: nothing to hear. + // A register that returned no promise: nothing to hear, nothing + // in flight. + free(); } return p; } }.register); @@ -1326,22 +1371,28 @@ ${DRAW_JS} const c = g.navigator.serviceWorker; if (c === undefined) known = true; else { - c.getRegistrations().then((rs) => { - for (const r of rs) { - const s = String(r.scope); - found.add(s); - if (!regs.has(s)) { - const a = swAddress(r.active || r.waiting || r.installing); - regs.set(s, a); - say({ scope: s, at: a }); + // keploy's lock, held or asked for: a register in flight. One of the + // app's own at that name counts too. + const busy = (l) => l.name === LOCK; + apply(lockQuery, locks, []).then((q) => { + if (q.held.some(busy) || q.pending.some(busy)) return undefined; + return c.getRegistrations().then((rs) => { + for (const r of rs) { + const s = String(r.scope); + found.add(s); + if (!regs.has(s)) { + const a = swAddress(r.active || r.waiting || r.installing); + regs.set(s, a); + say({ scope: s, at: a }); + } } - } - known = true; - }, () => {}); + known = true; + }); + }).catch(() => {}); } } catch (e) { - // No Service Worker can be read here (an opaque origin): keploy does not - // know. + // No Service Worker can be read here (an opaque origin), or no Web Locks: + // keploy does not know. } // The address of the Service Worker that may answer the script of a worker // at href ('' when keploy cannot read it); null when none may; false for diff --git a/playwright/test/fixtures/service-known.spec.ts b/playwright/test/fixtures/service-known.spec.ts index 3af7cbd..3e77771 100644 --- a/playwright/test/fixtures/service-known.spec.ts +++ b/playwright/test/fixtures/service-known.spec.ts @@ -11,8 +11,15 @@ import { test } from '../../dist'; // the Service Worker /swr/sw.js (scope /swr/, which b's own address is // not in), starts /swr/s.js once that Service Worker is active; the Service // Worker then sends the URL of each request it was asked. +// - 'shared worker registering': the page a registers /swh/sw.js (scope +// /swh/), whose script the server answers only once the page b, opened +// while the register is in flight, has started /swh/s.js, after the +// browser answered b's own lock read and getRegistrations (which come +// after keploy's). // - 'shared worker known': the page starts it once the browser has answered -// that its origin has no Service Worker. +// its own lock read and that its origin has no Service Worker (keploy asks +// for the registrations once its lock read answers, so a page that waits +// for both hears after keploy). const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); @@ -59,9 +66,33 @@ test('shared worker announced', async ({ context }) => { keep(test.info().title, { w: [got], seen }); }); +type Reg = { __reg: Promise }; + +test('shared worker registering', async ({ context }) => { + const a = await context.newPage(); + await a.goto(`${base}/swh/p`); + await a.evaluate(() => { (window as unknown as Reg).__reg = navigator.serviceWorker.register('/swh/sw.js'); }); + await fetch(`${base}/swh/asked`); + const b = await context.newPage(); + await b.goto(`${base}/long/registering`); + const got = await b.evaluate(async () => { + await navigator.locks.query(); + await navigator.serviceWorker.getRegistrations(); + return new Promise((ok) => { + const s = new SharedWorker('/swh/s.js'); + s.port.onmessage = (e) => ok(e.data); + s.port.start(); + }); + }); + await fetch(`${base}/swh/release`); + await a.evaluate(() => (window as unknown as Reg).__reg.then(() => true)); + keep(test.info().title, { w: [got] }); +}); + test('shared worker known', async ({ page }) => { await page.goto(`${base}/long/known`); const got = await page.evaluate(async () => { + await navigator.locks.query(); await navigator.serviceWorker.getRegistrations(); return new Promise((ok) => { const s = new SharedWorker('/swk/s.js'); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 263063f..2749428 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, markWorker, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -1465,24 +1465,50 @@ test('the lines for a worker keploy cannot reach say what it is, that each worke 'it is a worker that a worker started, and Chromium and Firefox do not let keploy reach its script to put its seeding ahead of it; record and replay the test in WebKit to seed it.'); assert.equal(unplacedLine('t', 'http://h/s.js', 'early'), 'keploy: in t, the Shared Worker at http://h/s.js draws the browser\'s own random numbers, ' + KIDS + 'it is a Shared Worker started before the browser told keploy which Service Workers the page\'s origin has (keploy asks as each document begins, ' + - 'and a document that cannot read its Service Workers, as a sandboxed one, is never told), and a Service Worker whose scope holds the worker\'s URL ' + + 'and a document that begins while a document of the origin is registering a Service Worker, or that cannot read its Service Workers ' + + 'or its Web Locks, as a sandboxed one, is never told), and a Service Worker whose scope holds the worker\'s URL ' + 'would see keploy\'s __keploy_worker mark in the script\'s URL, so keploy starts the worker as the app asked; if the tests do not need it, ' + 'set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered).'); assert.match(unplacedLine('t', 'http://h/w.js', 'routed'),/^keploy: in t, the worker at http:\/\/h\/w\.js draws the browser's own random numbers, and so does each worker it starts, so their draws may differ at replay: a route of the test's own /); }); +// An origin's Web Locks, as the browser's LockManager: request takes the +// lock at once (none here waits on another), holds it until the callback's +// promise settles and answers that; query answers what is held, and pending +// is what the row puts there. asked keeps each request, and log each +// request, query and release in order. +const lockManager = () => { + const held = []; + const pending = []; + const asked = []; + const log = []; + const locks = { + request(name, options, cb) { + if (typeof options === 'function') { cb = options; options = {}; } + const l = { name, mode: options.mode ?? 'exclusive' }; + held.push(l); asked.push(l); log.push(['request', l.name, l.mode]); + return Promise.resolve().then(() => cb(l)).finally(() => { held.splice(held.indexOf(l), 1); log.push(['release', l.name]); }); + }, + query() { log.push(['query', held.length]); return Promise.resolve({ held: held.map((l) => ({ ...l })), pending: pending.map((l) => ({ ...l })) }); }, + }; + return { locks, held, pending, asked, log }; +}; + // A document's ServiceWorkerContainer: register answers what answer gives // (a registration of its script, by default), getRegistrations the -// registrations regs, and the controller is state.controller. -const swGlobals = ({ answer = (u) => Promise.resolve({ active: { scriptURL: `http://h${u}` } }), regs = [], late = false, state = { controller: null } } = {}) => { +// registrations regs, and the controller is state.controller; its navigator's +// Web Locks are locks (a manager of its own by default; null: none). +const swGlobals = ({ answer = (u) => Promise.resolve({ active: { scriptURL: `http://h${u}` } }), regs = [], late = false, state = { controller: null }, locks = lockManager(), log = [] } = {}) => { class ServiceWorkerContainer { register(u) { return answer(u); } // late: the registrations come 20 ms after. - getRegistrations() { return late ? new Promise((r) => setTimeout(() => r(regs), 20)) : Promise.resolve(regs); } + getRegistrations() { log.push('getRegistrations'); return late ? new Promise((r) => setTimeout(() => r(regs), 20)) : Promise.resolve(regs); } get controller() { return state.controller; } } - return { state, globals: { ServiceWorkerContainer, navigator: { serviceWorker: new ServiceWorkerContainer() } } }; + const navigator = { serviceWorker: new ServiceWorkerContainer() }; + if (locks !== null) navigator.locks = locks.locks; + return { state, locks, globals: { ServiceWorkerContainer, navigator } }; }; const turn = () => new Promise((r) => setImmediate(r)); @@ -1726,6 +1752,114 @@ test('the documents of a context say on keploy\'s channel each registration they assert.equal(bus.all.length, before); }); +// Rejections no handler took while fn ran, and a few turns after. +const unhandled = async (fn) => { + const got = []; + const take = (e) => { got.push(e); }; + process.on('unhandledRejection', take); + try { await fn(); for (let i = 0; i < 5; i++) await turn(); } finally { process.off('unhandledRejection', take); } + return got; +}; + +test('each register holds keploy\'s Web Lock, shared, taken before the channel hears it and let go once the register settles, however it ends; a document asks for its registrations only after the browser said that no lock of that name is held or asked for: one that begins while another document\'s register is in flight never knows (its Shared Workers start as the app asked and are named), and neither does one whose lock read is refused or makes no sense, or one with no Web Locks; another lock changes nothing', async () => { + const bus = channels(); + const m = lockManager(); + // The channel's messages, in order with the locks'. + class BC extends bus.BroadcastChannel { postMessage(x) { m.log.push(['say', x.scope]); super.postMessage(x); } } + const doc = (sw) => { + const told = []; + const w = workerGlobals(); + const d = documentAt(initScript(S1, 'tell', 'm1'), '/p', new Map(), 't0', { report: 'tell', told, globals: { ...w.globals, ...sw.globals, BroadcastChannel: BC } }); + return { d, told, made: () => w.made.map(([u]) => workerMark(u)) }; + }; + // a's register is in flight until release. + let release = null; + const a = doc(swGlobals({ locks: m, log: m.log, answer: (u) => new Promise((ok) => { release = () => ok({ active: { scriptURL: `http://h${u}` } }); }) })); + await turn(); await turn(); + assert.deepEqual(m.log, [['query', 0], 'getRegistrations']); + m.log.length = 0; + a.d.run('navigator.serviceWorker.register("/app/sw.js");'); + assert.deepEqual(m.log, [['request', SW_LOCK, 'shared'], ['say', 'http://h/app/']]); + // b begins meanwhile: it finds the lock held, never asks for its + // registrations, and starts its Shared Workers as the app asked, even once + // the register has settled. + m.log.length = 0; + const b = doc(swGlobals({ locks: m, log: m.log })); + await turn(); await turn(); + assert.deepEqual(m.log, [['query', 1]]); + b.d.run('new SharedWorker("/s.js");'); + release(); + await turn(); await turn(); + assert.deepEqual(m.held, []); + b.d.run('new SharedWorker("/t.js");'); + assert.deepEqual([b.made(), b.told], [[null, null], [['http://h/s.js', 'early'], ['http://h/t.js', 'early']]]); + // c begins after: no lock, so it asks, and finds the registration. + m.log.length = 0; + const c = doc(swGlobals({ locks: m, log: m.log, regs: [{ scope: 'http://h/app/', active: { scriptURL: 'http://h/app/sw.js' } }] })); + await turn(); await turn(); + assert.deepEqual(m.log.slice(0, 2), [['query', 0], 'getRegistrations']); + c.d.run('new SharedWorker("/s.js"); new SharedWorker("/app/s.js");'); + assert.deepEqual([c.made().map((x) => x && x.kind), c.told], [['s', null], [['http://h/app/s.js', 'controlled', 'http://h/app/sw.js']]]); + // Two registers in flight at once both hold it. + const two = lockManager(); + const t = doc(swGlobals({ locks: two, answer: () => new Promise(() => {}) })); + t.d.run('navigator.serviceWorker.register("/x/sw.js"); navigator.serviceWorker.register("/y/sw.js");'); + await turn(); + assert.deepEqual(two.held, [{ name: SW_LOCK, mode: 'shared' }, { name: SW_LOCK, mode: 'shared' }]); + // However the register ends, the lock goes: refused, thrown, Playwright's + // block, no promise at all. + for (const [how, answer, script] of [ + ['refused', () => Promise.reject(new TypeError('no')), 'navigator.serviceWorker.register("/sw.js").catch(() => {});'], + ['thrown', () => { throw new TypeError('no'); }, 'try { navigator.serviceWorker.register("/sw.js"); } catch (e) {}'], + ['blocked', async () => undefined, 'navigator.serviceWorker.register("/sw.js");'], + ['no promise', () => 1, 'navigator.serviceWorker.register("/sw.js");'], + ]) { + const l = lockManager(); + const d = doc(swGlobals({ locks: l, answer })); + d.d.run(script); + await turn(); await turn(); + assert.deepEqual([l.asked.length, l.held], [1, []], how); + } + // A lock held or asked for, a lock read refused or one whose answer makes no + // sense, or no Web Locks: never known. A lock of another name: known. + const never = async (l) => { + const d = doc(swGlobals({ locks: l })); + await turn(); await turn(); + d.d.run('new SharedWorker("/s.js");'); + return [d.made(), d.told]; + }; + const pending = lockManager(); + pending.pending.push({ name: SW_LOCK, mode: 'shared' }); + const refused = lockManager(); + refused.locks.query = () => Promise.reject(new Error('SecurityError')); + const odd = lockManager(); + odd.locks.query = () => Promise.resolve({}); + for (const [how, l] of [['pending', pending], ['refused', refused], ['odd', odd], ['none', null]]) { + assert.deepEqual(await never(l), [[null], [['http://h/s.js', 'early']]], how); + } + const other = lockManager(); + other.held.push({ name: 'app-lock', mode: 'exclusive' }); + other.pending.push({ name: 'app-lock', mode: 'shared' }); + assert.deepEqual((await never(other))[1], []); + // With no Web Locks, a register still answers as the browser's does, and a + // context Playwright blocks still knows at once. + const bare = doc(swGlobals({ locks: null })); + assert.deepEqual(await bare.d.run('navigator.serviceWorker.register("/sw.js")'), { active: { scriptURL: 'http://h/sw.js' } }); + const blocked = swGlobals({ locks: null }); + blocked.globals.ServiceWorkerContainer.prototype.register = async () => { void 'Service Worker registration blocked by Playwright'; }; + const k = doc(blocked); + k.d.run('new SharedWorker("/s.js");'); + assert.equal(k.made()[0].kind, 's'); + // A lock request or read the browser refuses says nothing to the app. + assert.deepEqual(await unhandled(async () => { + const r = lockManager(); + r.locks.request = () => Promise.reject(new Error('AbortError')); + r.locks.query = () => Promise.reject(new Error('SecurityError')); + doc(swGlobals({ locks: r })).d.run('navigator.serviceWorker.register("/sw.js");'); + }), []); + assert.deepEqual(bus.thrown, []); +}); + test('a worker\'s seeding draws the same numbers for a token and instance every run, others for another; its location shows no mark, its own query and fragment kept', () => { const t = 'cd'.repeat(16); const a = workerAt(markWorker('http://h/w.js', 'w', t), workerScript(t, 0, 'w')).draws(); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 3a96eab..a011f58 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -633,6 +633,10 @@ async function drawsServer() { }; })(); const rheld = []; + // service-known.spec.ts's registering: /swh/sw.js is answered only at + // /swh/release; /swh/asked answers once it has been asked for. + const swheld = []; + const swasked = []; // Every URL the server was asked for, as the request named it. const seen = []; const srv = http.createServer((req, res) => { @@ -644,6 +648,14 @@ async function drawsServer() { // parse.spec.ts's removed: /rhold.js is answered at /rrelease. if (u.pathname === '/rhold.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); rheld.push(res); return; } if (u.pathname === '/rrelease') { for (const r of rheld.splice(0)) r.end(''); res.end(''); return; } + if (u.pathname === '/swh/sw.js') { + res.writeHead(200, { 'content-type': 'text/javascript' }); + swheld.push(res); + for (const r of swasked.splice(0)) r.end(''); + return; + } + if (u.pathname === '/swh/asked') { if (swheld.length > 0) res.end(''); else swasked.push(res); return; } + if (u.pathname === '/swh/release') { for (const r of swheld.splice(0)) r.end('self.addEventListener(\'fetch\', () => {});'); res.end(''); return; } // workers.spec.ts's workers, and the scripts they load; /wk/ref answers // the Referer it was sent. if (u.pathname === '/wk/ref') { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(req.headers.referer ?? 'none'); return; } @@ -692,6 +704,8 @@ async function drawsServer() { // is a Shared Worker its scope holds. '/swr/sw.js': 'const seen = []; self.addEventListener(\'fetch\', (e) => { seen.push(e.request.url); }); self.addEventListener(\'message\', (e) => e.source.postMessage({ kind: \'sw-seen\', seen }));', '/swr/s.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'swr-shared', href: location.href, draws: ${DRAWS} });`, + // A Shared Worker the scope of /swh/sw.js holds. + '/swh/s.js': `onconnect = (e) => e.ports[0].postMessage({ kind: 'swh-shared', href: location.href, draws: ${DRAWS} });`, }[u.pathname]; if (sw !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end(sw); return; } res.writeHead(200, { 'content-type': 'text/html' }); @@ -719,7 +733,7 @@ async function drawsServer() { // /nest holds /frame of its own origin. if (u.pathname === '/nest') { res.end(page('')); return; } // /long/... is a page at an address of any length. - if (u.pathname.startsWith('/long/') || u.pathname === '/swr/p') { res.end(page('')); return; } + if (u.pathname.startsWith('/long/') || u.pathname === '/swr/p' || u.pathname === '/swh/p') { res.end(page('')); return; } // places.spec.ts's: /host puts in a (/fa, its own origin), c (/fc) and // w (/fw, both cross-site) in the order ?order= names; /race holds r0 // and r1 at /fr, and /fr answers late the one ?slow= names; /nested @@ -1660,10 +1674,10 @@ for (const engine of ENGINES) { } // service-known.spec.ts's tests. -const KNOWN_TITLES = ['shared worker early', 'shared worker announced', 'shared worker known']; +const KNOWN_TITLES = ['shared worker early', 'shared worker announced', 'shared worker registering', 'shared worker known']; for (const engine of ENGINES) { - test(`seeded, in ${engine}: a Shared Worker is marked only once keploy knows no Service Worker's scope holds its URL: one started before the browser answers starts as the app asked and is named; one whose URL the scope of a registration another page made holds starts as the app asked, so that Service Worker sees no mark, and is named with it; one started once the browser said there is none draws from the test's seed, the same at replay (in Firefox, whose route does not reach a Shared Worker's script, each is named for that)`, async () => { + test(`seeded, in ${engine}: a Shared Worker is marked only once keploy knows no Service Worker's scope holds its URL: one started before the browser answers starts as the app asked and is named; one whose URL the scope of a registration another page made holds starts as the app asked, so that Service Worker sees no mark, and is named with it; one of a page that began while another page's register was in flight starts as the app asked and is named, even once keploy's lock read would have answered; one started once the browser said there is none draws from the test's seed, the same at replay (in Firefox, whose route does not reach a Shared Worker's script, each is named for that)`, async () => { const j = journal(); const rec = await fakeAgent(j); let kept = {}; @@ -1696,6 +1710,15 @@ for (const engine of ENGINES) { ].sort(), facts); assert.equal(ann.seen.kind, 'sw-seen', facts); assert.deepEqual(ann.seen.seen.filter((u) => u.includes('__keploy_worker')), [], facts); + // Registering: the page began while the other page's register held + // keploy's lock, so it never knows: as the app asked, named. + const reg = r.drew['shared worker registering']; + assert.equal(reg.w[0].href, `${origin}/swh/s.js`, facts); + assert.deepEqual(said('shared worker registering'), [ + unplacedLine('shared worker registering', at('swh/sw.js'), 'service'), + unplacedLine('shared worker registering', at('swh/s.js'), ff ? 'shared' : 'early'), + sharedWorkerLine('shared worker registering', at('swh/s.js')), + ].sort(), facts); // Known: seeded (Chromium, WebKit), and only the settle line is said. const known = r.drew['shared worker known'].w[0]; assert.equal(known.href, `${origin}/swk/s.js`, facts); @@ -1709,8 +1732,10 @@ for (const engine of ENGINES) { const again = q.drew['shared worker known'].w[0]; if (ff) assert.notDeepEqual(again.draws, known.draws, facts); else assert.deepEqual(again.draws, known.draws, facts); - // The early one is not seeded: it draws other numbers at replay. + // The early one is not seeded: it draws other numbers at replay, as + // does the registering one. assert.notDeepEqual(q.drew['shared worker early'].w[0].draws, early.draws, facts); + assert.notDeepEqual(q.drew['shared worker registering'].w[0].draws, reg.w[0].draws, facts); } finally { await rep.close(); await rec.close(); } }); } From d66e2f5eb38c69a267ce3636a834da664281e556 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 23:00:07 +0530 Subject: [PATCH 41/73] test(playwright): settle rows that wait on a call end by settle's wait Eleven settle rows showed that a call is still waited for by letting the test run into a 4 s timeout: a deadline that bounds the test's body too, which on a loaded machine can take seconds. Each now has the page server answer the call only once settle's step has begun, and asserts that the test passes, that the step names the call, and that the test's end comes after the call's answer; a settle that did not wait ends the test with the call unanswered. /slowdoc now says when its last byte went. The row whose subject is the timeout itself, a call that never ends, keeps it. Signed-off-by: slayerjain --- playwright/test/rows.test.mjs | 62 ++++++++++++++--------------------- 1 file changed, 25 insertions(+), 37 deletions(-) diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index a011f58..3d3d795 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -78,6 +78,7 @@ async function pageServer(j) { j.add({ page: 'slowdoc sent' }); res.writeHead(200, { 'content-type': 'text/html' }); res.write('loading'); + res.on('finish', () => j.add({ page: 'slowdoc answered' })); void released.then(() => res.end('')); return; } @@ -442,28 +443,34 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { }); } -test('settle: a route change inside the page cancels nothing, so its call in flight is waited for', async () => { +// Settle rows where a call is still waited for: the page server answers it +// only once settle's step has begun (the rows' reporter asks /release then), +// so the test ends by settle's own wait, with no deadline on its body. The +// test passes, settle's step names the call (method, URL), and the test's +// end comes after the call's answer. A settle that did not wait for it ends +// the test with the call unanswered. +async function waitedFor(title, method, call) { const j = journal(); const a = await fakeAgent(j); const p = await pageServer(j); try { - const { tests, out } = await playwright(['settle.spec.ts', '-g', 'route change inside the page'], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000' }); - const t = tests['a call in flight across a route change inside the page']; - assert.equal(t.status, 'timedOut', out); - assert.ok(t.steps.some((s) => /^keploy: wait for the test's 1 call\(s\) in flight .*\/slow$/.test(s)), t.steps.join('\n')); + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', RELEASE_AT_STEP: `${p.url}/release` }); + const t = tests[title]; + assert.equal(t.status, 'passed', out); + const named = new RegExp(`^keploy: wait for the test's 1 call\\(s\\) in flight before its browser context closes: ${method} http://127\\.0\\.0\\.1:\\d+${call}$`); + assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n')); + const name = call.slice(1); + const order = j.events.map((e) => (e.page ? e.page : e.agent.path)); + assert.deepEqual(order.filter((e) => e === `${name} sent` || e === `${name} answered` || e === '/agent/scope/end'), [`${name} sent`, `${name} answered`, '/agent/scope/end'], order.join(', ')); } finally { await p.close(); await a.close(); } +} + +test('settle: a route change inside the page cancels nothing, so its call in flight is waited for', async () => { + await waitedFor('a call in flight across a route change inside the page', 'GET', '/slow'); }); test('settle: a navigation answered 204 replaces no document, so a call in flight is still waited for', async () => { - const j = journal(); - const a = await fakeAgent(j); - const p = await pageServer(j); - try { - const { tests, out } = await playwright(['settle.spec.ts', '-g', 'navigation answered 204'], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000' }); - const t = tests['a call in flight across a navigation answered 204']; - assert.equal(t.status, 'timedOut', out); - assert.ok(t.steps.some((s) => /^keploy: wait for the test's 1 call\(s\) in flight .*\/slow$/.test(s)), t.steps.join('\n')); - } finally { await p.close(); await a.close(); } + await waitedFor('a call in flight across a navigation answered 204', 'GET', '/slow'); }); test('an unnamed Playwright project claims with the key ".", and its tests name no project', async () => { @@ -572,21 +579,11 @@ for (const [title, why, call] of [ } test('settle: a beacon from a page that stays is still waited for', async () => { - const j = journal(); - const a = await fakeAgent(j); - const p = await pageServer(j); - try { - const title = 'a beacon in flight from the page'; - const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000' }); - const t = tests[title]; - assert.equal(t.status, 'timedOut', out); - const named = /^keploy: wait for the test's 1 call\(s\) in flight before its browser context closes: POST http:\/\/127\.0\.0\.1:\d+\/beacon$/; - assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n')); - } finally { await p.close(); await a.close(); } + await waitedFor('a beacon in flight from the page', 'POST', '/beacon'); }); -// Settle rows where nothing was cancelled: the call is still waited for, and -// the test times out naming it. +// Settle rows where nothing was cancelled: the call is still waited for +// (waitedFor). for (const [title, why, call] of [ ['a call in flight across a navigation answered 205', 'a navigation answered 205 replaces no document', '/slow'], ['a call in flight across a download', 'a navigation answered with an attachment (a download, "Attachment") replaces no document', '/slow'], @@ -598,16 +595,7 @@ for (const [title, why, call] of [ ['a service worker call in flight while another page of the test closes', 'another page closing cancels nothing of a service worker', '/slow'], ]) { test(`settle: ${why}, so the call is still waited for`, async () => { - const j = journal(); - const a = await fakeAgent(j); - const p = await pageServer(j); - try { - const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000' }); - const t = tests[title]; - assert.equal(t.status, 'timedOut', out); - const named = new RegExp(`^keploy: wait for the test's 1 call\\(s\\) in flight before its browser context closes: GET http://127\\.0\\.0\\.1:\\d+${call}$`); - assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n')); - } finally { await p.close(); await a.close(); } + await waitedFor(title, 'GET', call); }); } From 6791c839fe219e0b1fedd379459434d6773e00d9 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sat, 14 Nov 2026 23:55:45 +0530 Subject: [PATCH 42/73] feat(playwright): name each place a replay draws otherwise than recorded A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. Frames at one path put in in another order or number, a worker fixture or a library that draws in whichever test first needs it, and a draw made a varying number of times move the draws after them, and nothing said so. A seeded test's end now sends keploy a note: how many times the test process drew each stream (Math.random, each seeded faker instance), how many times each placed document drew Math.random, getRandomValues and randomUUID, and how many frames each document put in at each path. The page's seeding reports a document once per task in which it drew or numbered a frame, and only once its place is known. The note is JSON of at most 16 KiB: past it, documents are cut, then streams, and counted. keploy keeps the note with a record and answers it to the replay's begin. The replay's end prints one line for each difference, naming the test, what differs and the next step, at most 8 a test. A seeded replay answered no note says so once per worker. A begin answered a note that is no string of at most 16 KiB fails the test. Signed-off-by: slayerjain --- playwright/README.md | 10 +- playwright/src/agent.ts | 21 ++- playwright/src/index.ts | 51 +++++- playwright/src/note.ts | 198 +++++++++++++++++++++ playwright/src/random.ts | 65 ++++++- playwright/test/fixtures/note.spec.ts | 26 +++ playwright/test/note.test.mjs | 244 ++++++++++++++++++++++++++ playwright/test/rows.test.mjs | 123 +++++++++++-- 8 files changed, 706 insertions(+), 32 deletions(-) create mode 100644 playwright/src/note.ts create mode 100644 playwright/test/fixtures/note.spec.ts create mode 100644 playwright/test/note.test.mjs diff --git a/playwright/README.md b/playwright/README.md index b84a514..825d7ad 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -76,6 +76,12 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco A document the seeding cannot place draws the browser's own numbers, never another page's. The test then prints `keploy: in , the document at
drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. The address is the document's origin and path; for an opaque origin (`about:blank`, `about:srcdoc`, a `data:` URL, a sandboxed frame or page) it is the document's URL with no query or fragment, and `about:blank` for a window's first document that has no URL yet (Firefox runs init scripts there before it has one). The limits below list those documents. - **What keploy is told.** The begin says what the test means to do (`random: "seeded"` or `"own"`). The end says what its pages drew from: `"seeded"` when a page opened in its seeded `context`, `"own"` when a page opened there unseeded, or when keploy answered no seed. When a test's calls still vary between runs, keploy can then say whether its pages were seeded, so the variation comes from somewhere the seeding does not reach, or whether seeding them is left to try. + - **When a replay draws otherwise.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. So a seeded test's end also sends keploy a note (`note`, JSON of at most 16 KiB) of how many times the test process drew each of its streams (`Math.random` and each seeded faker instance), how many times each placed document drew each function, and how many frames each document put in at each path. A record keeps the note with the test's recording, and keploy never reads it. A replay's begin is answered the recorded note, and the replay's end prints one line for each difference, with its next step: + - `keploy: drew Math.random 2 times in the test process at this replay and once in its recording, so ...`: register worker fixtures with this package's `test.extend`, use a library that draws on its first use once outside the tests, and draw a fixed number of times; + - `keploy: : the document at
drew crypto.randomUUID 3 times at this replay and 2 times in its recording, so ...`: draw a fixed number of times, and give each frame its own path; + - `keploy: : the document at
put in 2 frame(s) at /chat at this replay and 1 in its recording, so ...`: give each frame at that path its own `src` path, or put them in in a fixed order and number. + + A test prints at most 8 such lines, then one that says how many more differences it found. A note that would be over 16 KiB keeps the documents that reported first and counts the rest as cut; a document one of the two notes cut is compared with nothing. A seeded replay that keploy answers with no note says so once per worker: its recording keeps none (recorded with keploy v3.7.5 or older, or with @keploy/playwright 0.1.1 or older), or this keploy keeps none (v3.7.5 or older). Record the tests again with a keploy that keeps notes. - **On by default.** To turn it off, for record and replay alike: - for the whole run: `KEPLOY_SEED_RANDOM=off` (or `0`, `false`; unset, `on`, `1` or `true` seed; any other value fails every test, naming the variable); - for a project, a file or a test: the option `keploySeedRandom: false`, in the config's or a project's `use`, or with `test.use({ keploySeedRandom: false })`. Its type is `KeployOptions`: `defineConfig({ use: { keploySeedRandom: false } })`. @@ -101,8 +107,8 @@ Any answer from the agent that this package cannot trust fails the test. That co A page whose own draws vary in number before another script draws (a draw per timer tick) still shifts that function's stream within the document. - Two frames at one path can still trade numbers, unnamed, where the order they are counted in is not the same each run: - - a frame's number at its path is how many frames its parent document had put in at that path before it, so frames at one path that the page or a third-party script puts in, in an order or a number that varies run to run (as their scripts load, in network order; a re-render that makes a varying number of frames), trade numbers, unnamed. Chat and support widgets often put in `about:blank` frames as their scripts load; + Two frames at one path can still trade numbers where the order they are counted in is not the same each run. The replay names it (above) where the number of frames at that path, or what one of the two documents drew, differs from the recording; two that trade places and draw alike are not named: + - a frame's number at its path is how many frames its parent document had put in at that path before it, so frames at one path that the page or a third-party script puts in, in an order or a number that varies run to run (as their scripts load, in network order; a re-render that makes a varying number of frames), trade numbers. Chat and support widgets often put in `about:blank` frames as their scripts load; - frames under a parent of another origin are counted by their origin in the tab, so two of one origin at one path take their numbers in the order their documents begin, which may change at replay; - a frame put into a shadow root already in the page is not seen as it is put in, since the parent document watches only the nodes added to or removed from it outside its shadow roots: the frame is numbered at the document's next numbering (when the app next puts a frame in outside its shadow roots, or when a document of a frame not yet numbered, its own included, begins), so one put there before another frame at its path, at a moment that varies run to run, may take that frame's number. diff --git a/playwright/src/agent.ts b/playwright/src/agent.ts index 17b6d2f..1c5eb0f 100644 --- a/playwright/src/agent.ts +++ b/playwright/src/agent.ts @@ -33,8 +33,12 @@ export interface Answer { timeFile?: string; variable?: string; seed?: string; + note?: string; } +/** The largest note keploy keeps (its models.MaxScopeNote), in UTF-8 bytes. */ +export const MAX_NOTE = 16 << 10; + /** An answer this package cannot trust, or the agent's own refusal. */ export class KeployError extends Error { /** The HTTP status of a non-2xx answer; 0 when there was none. */ @@ -150,7 +154,10 @@ export class AgentClient { * when it means to seed them, is one drawn afresh for this begin; the * answer's seed is the one to draw from: in a record the begin's own, in a * replay the one the test's recording kept, and none when it kept none. An - * agent from before either field ignores it and answers no seed. + * agent from before either field ignores it and answers no seed. The + * answer's note, in a replay, is the one the test's recording's end sent + * (end); a record's begin, a recording that kept none and an agent from + * before notes answer none. */ async begin(test: TestRef, random?: Random, seed?: string): Promise { const a = await this.onWire(PATH_BEGIN, { ...test, canSkip: true, ...(random ? { random } : {}), ...(seed ? { seed } : {}) }, `${test.name} cannot begin`); @@ -164,16 +171,22 @@ export class AgentClient { throw new KeployError(`keploy: the keploy agent answered the begin of ${test.name} with the seed ${quoted(typeof a.seed === 'string' ? a.seed : String(JSON.stringify(a.seed)))}, ` + 'which is no seed keploy keeps (8 to 64 lower-case hex digits)'); } + if (a.note !== undefined && (typeof a.note !== 'string' || Buffer.byteLength(a.note, 'utf8') > MAX_NOTE)) { + throw new KeployError(`keploy: the keploy agent answered the begin of ${test.name} with the note ${quoted(typeof a.note === 'string' ? a.note : String(JSON.stringify(a.note)))}, ` + + `which is no note keploy keeps (a string of at most ${MAX_NOTE} bytes)`); + } return a; } /** * Tells the agent the test ended, with its outcome, and, when it is known, * how its browser drew its random numbers (random), which replaces what its - * begin meant to do. + * begin meant to do, and the note (note.ts) keploy keeps with a record and + * answers to the test's replay's begin, which it never reads. An agent + * from before notes ignores it. */ - async end(test: TestRef, outcome: Outcome, random?: Random): Promise { - const a = await this.onWire(PATH_END, { ...test, outcome, ...(random ? { random } : {}) }, `${test.name} cannot end`); + async end(test: TestRef, outcome: Outcome, random?: Random, note?: string): Promise { + const a = await this.onWire(PATH_END, { ...test, outcome, ...(random ? { random } : {}), ...(note ? { note } : {}) }, `${test.name} cannot end`); if (a.status !== 'ok') { throw new KeployError(`keploy: the keploy agent answered the end of ${test.name} with status ${JSON.stringify(a.status ?? '')}`); } diff --git a/playwright/src/index.ts b/playwright/src/index.ts index d6d5548..178d8e3 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -48,7 +48,13 @@ // instances and those the suite gives seedFaker, so data the test makes // there (a name stamped with Math.random()) is the recording's; KEPLOY_SEED_RANDOM=off // or the option keploySeedRandom: false turns it off, and the worker -// says once what that costs. +// says once what that costs; +// - a seeded test's end sends keploy a note of how many times the test +// drew each stream, in its test process and in each of its documents, +// and how many frames each document put in at each path (note.ts), +// which a record keeps; its replay's begin is answered that note, and +// its end prints one line for each difference, naming what differs and +// the next step, since a draw that moves shifts the draws after it. // Any answer of the agent this package cannot trust fails the test. import { test as base, expect } from '@playwright/test'; @@ -59,7 +65,8 @@ import { ACTION_FAIL, ACTION_SKIP, AgentClient, KeployError, RANDOM_OWN, RANDOM_ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; -import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; +import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; +import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; import { Inflight, sharedWorkerLine, sharedWorkerScript } from './settle'; export { expect, KeployError }; @@ -84,6 +91,8 @@ let seedingOn = false; let saidSeeded = false; // Whether this worker has said what a test with seeding off costs. let saidOff = false; +// Whether this worker has said that a seeded replay was answered no note. +let saidNoNote = false; // The reasons this worker has said it cannot seed faker, each said once. const saidFaker = new Set(); // How many contexts this worker seeded: each one's report function has its @@ -103,6 +112,8 @@ type PageSeeding = { // (seededPage) or not (ownPage). seededPage: boolean; ownPage: boolean; + // What its context's documents drew, for its note. + drawn: DocLog; }; // The open test's, while one is open. let seeding: PageSeeding | null = null; @@ -178,6 +189,9 @@ const keployTestType = base.extend({ // A document the seeding cannot place draws the browser's own numbers // and says so here, once for each address and reason in the test. const report = `__keploy_random_unplaced_${++reports}`; + // How each placed document drew, for the test's note. + const drawnReport = `__keploy_random_drawn_${reports}`; + await context.exposeBinding(drawnReport, (_source, key: unknown, at: unknown, n: unknown, f: unknown) => { keepDocDraws(s.drawn, key, at, n, f); }); const told = new Set(); let opened: (page: Page | null | undefined) => Promise = async () => false; await context.exposeBinding(report, async (source, at: unknown, why: unknown, above: unknown) => { @@ -243,7 +257,7 @@ const keployTestType = base.extend({ process.stdout.write(`${unplacedLine(name, a, why)}\n`); }, }, browserName === 'webkit'); - await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox') }); + await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox', drawnReport) }); const placing = placePages(context, mark); restore = placing.restore; opened = placing.opened; @@ -302,7 +316,8 @@ const keployTestType = base.extend({ throw new KeployError(`keploy: ${again.message} (${again.remedy})${told}`); } const asked = seedingOn && keploySeedRandom; - const b = await c.begin(ref, asked ? RANDOM_SEEDED : RANDOM_OWN, asked ? freshSeed() : undefined); + const sent = asked ? freshSeed() : undefined; + const b = await c.begin(ref, asked ? RANDOM_SEEDED : RANDOM_OWN, sent); if (b.action === ACTION_SKIP) { // The framework's own skip, with keploy's reason; no end follows. // Playwright's list, line and dot reporters print a test's output but @@ -312,7 +327,10 @@ const keployTestType = base.extend({ testInfo.skip(true, why); return; } - const s: PageSeeding = { asked, seed: asked ? b.seed ?? '' : '', seededPage: false, ownPage: false }; + const s: PageSeeding = { asked, seed: asked ? b.seed ?? '' : '', seededPage: false, ownPage: false, drawn: docLog() }; + // A replay: keploy answered the seed its recording kept, not the one + // this begin sent (a record answers the begin's own). + const replaying = s.seed !== '' && s.seed !== sent; if (s.seed !== '') { if (!saidSeeded) { saidSeeded = true; @@ -340,8 +358,10 @@ const keployTestType = base.extend({ // The test process draws Math.random, and faker's instances draw, from // the test's seed from here, its beforeEach hooks included, until its // afterEach hooks and fixtures end. - const unseat = s.seed !== '' ? seatRandom(processRandom(s.seed)) : null; - const unseatFaker = s.seed !== '' ? seatFaker(s.seed, fakers.found) : null; + // How many times the test process drew each stream, for the note. + const processDrawn = new Map(); + const unseat = s.seed !== '' ? seatRandom(tally(processDrawn, 'Math.random', processRandom(s.seed))) : null; + const unseatFaker = s.seed !== '' ? seatFaker(s.seed, fakers.found, processDrawn) : null; try { await use(); } finally { @@ -350,7 +370,22 @@ const keployTestType = base.extend({ } open = null; seeding = null; - const e = await c.end(ref, outcomeOf(testInfo), randomDone(s)); + const note = s.seed !== '' ? noteText(processDrawn, s.drawn) : undefined; + if (replaying && note !== undefined) { + // What differs from the recording's draws: one line each. + if (b.note === undefined || b.note === '') { + if (!saidNoNote) { + saidNoNote = true; + process.stdout.write(`${NO_NOTE_LINE}\n`); + } + } else { + const recorded = readNote(b.note); + const now = readNote(note); + if (typeof recorded === 'string') process.stdout.write(`${unreadNoteLine(ref.name, recorded)}\n`); + else if (typeof now !== 'string') for (const line of differences(ref.name, recorded, now)) process.stdout.write(`${line}\n`); + } + } + const e = await c.end(ref, outcomeOf(testInfo), randomDone(s), note); if (e.action === ACTION_FAIL) throw new KeployError(keployText(e.reason, 'keploy: the keploy agent failed this test and gave no reason')); }, { auto: true }], }); diff --git a/playwright/src/note.ts b/playwright/src/note.ts new file mode 100644 index 0000000..7e89c97 --- /dev/null +++ b/playwright/src/note.ts @@ -0,0 +1,198 @@ +// The note a seeded test leaves with its recording, and what its replay says +// when it draws otherwise. +// +// A recording keeps a test's seed, not its numbers, so a replay draws what +// its recording drew only while each stream is drawn in the same order and +// the same number of times. Some things move a draw between runs and no +// seeding can place them: frames at one path put in in another order or +// number, a worker fixture or a library that draws in whichever test first +// needs it, a draw made a varying number of times. The record's end sends +// keploy a note of how many times each stream was drawn and how many frames +// each document put in at each path; keploy keeps it with the test's +// recording, never reads it, and answers it to the replay's begin. The +// replay's end compares and prints one line for each difference, naming the +// test, what differs and the next step. + +import { MAX_NOTE } from './agent'; +import { pageAddress } from './random'; + +export { MAX_NOTE }; + +/** The version of the note this package writes and reads. */ +export const NOTE_VERSION = 1; + +/** The functions a document's draws are counted for, in a count's order. */ +export const DOC_FUNCTIONS = ['Math.random', 'crypto.getRandomValues', 'crypto.randomUUID'] as const; + +/** + * What one document drew and put in: at, its address (for the lines); n, + * how many times it drew each of DOC_FUNCTIONS; f, how many frames it put in + * at each path. + */ +export type DocDraws = { at: string; n: number[]; f: Record }; + +/** + * A test's note: v, NOTE_VERSION; p, how many times the test process drew + * each of its streams; d, each document's draws by its place (the key its + * streams are made from, without the seed); cut, how many documents and + * streams were left out to keep the note within MAX_NOTE. + */ +export type TestNote = { v: number; p: Record; d: Record; cut: number }; + +/** At most this many documents are kept for one test, the rest are cut. */ +export const MAX_DOCS = 1000; + +/** At most this many difference lines are printed for one test. */ +export const MAX_LINES = 8; + +/** + * What a test's documents drew: docs, each one's by its place; dropped, the + * places of those left out once MAX_DOCS were kept. + */ +export type DocLog = { docs: Map; dropped: Set }; + +export function docLog(): DocLog { + return { docs: new Map(), dropped: new Set() }; +} + +/** + * Keeps one report of a document's draws (the seeding's in the page sends + * one after each task in which the document drew or numbered a frame): the + * counts only grow, so each is the larger of the two. A report that is not + * a document's draws is not kept, nor is a new document once MAX_DOCS are. + */ +export function keepDocDraws(log: DocLog, key: unknown, at: unknown, n: unknown, f: unknown): void { + if (typeof key !== 'string' || key === '' || !Array.isArray(n) || n.length !== DOC_FUNCTIONS.length || !n.every(count)) return; + if (f === null || typeof f !== 'object' || Array.isArray(f) || !Object.values(f).every(count)) return; + const docs = log.docs; + const had = docs.get(key); + if (had === undefined && docs.size >= MAX_DOCS) { + log.dropped.add(key); + return; + } + const frames: Record = { ...had?.f }; + for (const [p, c] of Object.entries(f as Record)) frames[p] = Math.max(frames[p] ?? 0, c); + docs.set(key, { + at: had?.at ?? pageAddress(at), + n: (n as number[]).map((c, i) => Math.max(had?.n[i] ?? 0, c)), + f: frames, + }); +} + +function count(c: unknown): c is number { + return typeof c === 'number' && Number.isSafeInteger(c) && c >= 0; +} + +/** + * The note of a test that drew process from its test process's streams and + * log in its documents, as JSON of at most MAX_NOTE UTF-8 bytes: the + * documents that would take it past the bound are cut, the last kept first, + * then the streams, the last drawn first (a stream's name is the suite's + * own when it seeds a faker instance under a long name); they are counted, + * and so are the documents keepDocDraws did not keep. A note with neither + * is always within the bound. + */ +export function noteText(process: ReadonlyMap, log: DocLog): string { + const entries = [...log.docs]; + const streams = [...process]; + for (;;) { + const cut = log.dropped.size + log.docs.size - entries.length + process.size - streams.length; + const text = JSON.stringify({ v: NOTE_VERSION, p: Object.fromEntries(streams), d: Object.fromEntries(entries), cut } satisfies TestNote); + if (Buffer.byteLength(text, 'utf8') <= MAX_NOTE) return text; + if (entries.length > 0) entries.pop(); + else streams.pop(); + } +} + +/** + * The note a recording kept, read; a string that says why it cannot be read + * instead (another version's, or no note of this package's). + */ +export function readNote(text: string): TestNote | string { + let v: unknown; + try { + v = JSON.parse(text); + } catch { + return 'it is not JSON'; + } + if (v === null || typeof v !== 'object' || Array.isArray(v)) return 'it is not an object'; + const o = v as Record; + if (o.v !== NOTE_VERSION) return `it is version ${JSON.stringify(o.v ?? null)}, and this @keploy/playwright reads version ${NOTE_VERSION}`; + const p = o.p, d = o.d, cut = o.cut; + if (p === null || typeof p !== 'object' || Array.isArray(p) || !Object.values(p).every(count)) return 'its test process counts are not counts'; + if (d === null || typeof d !== 'object' || Array.isArray(d)) return 'its documents are not an object'; + for (const doc of Object.values(d as Record)) { + const x = doc as Partial | null; + if (x === null || typeof x !== 'object' || typeof x.at !== 'string' || !Array.isArray(x.n) || x.n.length !== DOC_FUNCTIONS.length || !x.n.every(count) || + x.f === null || typeof x.f !== 'object' || Array.isArray(x.f) || !Object.values(x.f).every(count)) return 'a document\'s counts are not counts'; + } + if (!count(cut)) return 'its cut is not a count'; + return { v: NOTE_VERSION, p: p as Record, d: d as Record, cut }; +} + +const TIMES = (n: number): string => (n === 1 ? 'once' : `${n} times`); + +const PROCESS_NEXT = 'so its draws from that stream after the first one that differs are not the recording\'s, nor is what the test makes from them. ' + + 'In the test process: register a worker fixture with this package\'s test.extend (one registered through @playwright/test\'s own test, ' + + 'or through a test type mergeTests returned, draws in whichever test first asks for it); use a library that draws on its first use once ' + + 'outside the tests (in a worker fixture of this package\'s test); and draw a fixed number of times (a retry loop\'s jitter draws a varying number)'; + +const DOC_NEXT = 'so its draws of that function after the first one that differs are not the recording\'s: ' + + 'a draw made a varying number of times (one for each timer tick, a retry\'s jitter) shifts the ones after it, ' + + 'and a frame at a path where frames are put in in a varying order or number may take another frame\'s place; ' + + 'draw a fixed number of times, and give each frame its own path'; + +const FRAME_NEXT = 'so the frames at that path may take other places than in the recording and draw other numbers: ' + + 'give each frame at that path its own src path, or put them in in a fixed order and number'; + +/** + * The lines a replay of test name prints where it drew otherwise than its + * recording (recorded: the recording's note; now: this replay's): one for + * each stream of the test process drawn another number of times, each + * function a document drew another number of times, and each path at which + * a document put in another number of frames; at most MAX_LINES, then one + * that says how many more. A stream or a document one note cut is compared + * with nothing. + */ +export function differences(name: string, recorded: TestNote, now: TestNote): string[] { + const lines: string[] = []; + const streams = [...new Set([...Object.keys(recorded.p), ...Object.keys(now.p)])]; + for (const s of streams) { + // A stream a note cut is not known to differ. + if ((recorded.p[s] === undefined && recorded.cut > 0) || (now.p[s] === undefined && now.cut > 0)) continue; + const r = recorded.p[s] ?? 0, n = now.p[s] ?? 0; + if (r !== n) lines.push(`keploy: ${name} drew ${s} ${TIMES(n)} in the test process at this replay and ${TIMES(r)} in its recording, ${PROCESS_NEXT}.`); + } + const keys = [...new Set([...Object.keys(recorded.d), ...Object.keys(now.d)])]; + for (const k of keys) { + const r = recorded.d[k], n = now.d[k]; + // A document a note cut is not known to differ. + if ((r === undefined && recorded.cut > 0) || (n === undefined && now.cut > 0)) continue; + const at = n?.at ?? r?.at ?? ''; + DOC_FUNCTIONS.forEach((fn, i) => { + const rc = r?.n[i] ?? 0, nc = n?.n[i] ?? 0; + if (rc !== nc) lines.push(`keploy: ${name}: the document at ${at} drew ${fn} ${TIMES(nc)} at this replay and ${TIMES(rc)} in its recording, ${DOC_NEXT}.`); + }); + const paths = [...new Set([...Object.keys(r?.f ?? {}), ...Object.keys(n?.f ?? {})])]; + for (const p of paths) { + const rc = r?.f[p] ?? 0, nc = n?.f[p] ?? 0; + if (rc !== nc) lines.push(`keploy: ${name}: the document at ${at} put in ${nc} frame(s) at ${pageAddress(p)} at this replay and ${rc} in its recording, ${FRAME_NEXT}.`); + } + } + if (lines.length <= MAX_LINES) return lines; + return [...lines.slice(0, MAX_LINES), `keploy: ${name} drew otherwise than its recording in ${lines.length - MAX_LINES} more way(s) than the lines above name.`]; +} + +/** + * The line a seeded replay prints, once per worker, when keploy answered + * its begin with no note, so a difference in its draws cannot be named. + */ +export const NO_NOTE_LINE = 'keploy: keploy answered a seeded replay\'s begin with no note of how its recording drew, so where a test draws otherwise than its recording, ' + + 'no line names it: the recording keeps none (it was recorded with keploy v3.7.5 or older, or with @keploy/playwright 0.1.1 or older), ' + + 'or this keploy keeps none (keploy v3.7.5 or older). Record the tests again with a keploy that keeps notes to have each difference named.'; + +/** The line a replay prints when its recording's note cannot be read (why). */ +export function unreadNoteLine(name: string, why: string): string { + return `keploy: the note keploy kept of how ${name}'s recording drew cannot be read (${why}), so where its replay draws otherwise, no line names it: ` + + 'record the test again with this @keploy/playwright.'; +} diff --git a/playwright/src/random.ts b/playwright/src/random.ts index dad00db..e226437 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -231,6 +231,17 @@ export function processRandom(seed: string, name = 'Math.random'): () => number return () => ((next() >>> 5) * 67108864 + (next() >>> 6)) / 9007199254740992; } +/** + * Gives draw back, counting each of its draws under name in counts (the + * test process's part of a test's note, note.ts). + */ +export function tally(counts: Map, name: string, draw: () => number): () => number { + return () => { + counts.set(name, (counts.get(name) ?? 0) + 1); + return draw(); + }; +} + /** * Puts draw in Math.random's place in this process until the returned * function puts the process's own back. That one puts it back only while @@ -538,9 +549,10 @@ function ownStream(name: string): string { * set up), an instance draws its own. A test that seeds an instance itself * (faker.seed(n)) draws that instance's own from then on, seeded as it * asked. An instance's own generator does not move while seated, so outside - * every test it goes on as if no test had drawn. + * every test it goes on as if no test had drawn. counts, when given, counts + * each seated draw under its stream's name (tally). */ -export function seatFaker(seed: string, found: Fakers['found']): () => void { +export function seatFaker(seed: string, found: Fakers['found'], counts?: Map): () => void { const aside = asideNow; const puts: Array<() => void> = []; const done = new Set(); @@ -549,7 +561,7 @@ export function seatFaker(seed: string, found: Fakers['found']): () => void { done.add(r); const own = r.next; const ownSeed = r.seed; - const draw = processRandom(seed, stream); + const draw = counts !== undefined ? tally(counts, stream, processRandom(seed, stream)) : processRandom(seed, stream); const seated: Randomizer = { next(this: unknown): number { return aside.size > 0 ? Reflect.apply(own, this, []) as number : draw(); }, seed(this: unknown, ...args: unknown[]): unknown { @@ -867,8 +879,15 @@ export function placePages(context: BrowserContext, mark: string): { restore: () * * mark: the value by which firstScript marks a page's first document ('' for * none, so no document is marked). + * + * drawn: the name of a function the fixture exposed ('' for none) that + * hears how a placed document drew: after each task in which it drew or + * numbered a frame, once its place is known, with its place (the key its + * streams are made from, without the seed), its address, how many times it + * drew Math.random, crypto.getRandomValues and crypto.randomUUID, and how + * many frames it numbered at each path (the note, note.ts). */ -export function initScript(seed: string, report = '', mark = '', shared = true): string { +export function initScript(seed: string, report = '', mark = '', shared = true, drawn = ''): string { return `(() => { const seed = ${JSON.stringify(seed)}; const mark = ${JSON.stringify(mark)}; @@ -879,6 +898,35 @@ ${STREAM_JS} // off the window so the page does not see it. let tell = null; ${report === '' ? '' : `try { const f = g[${JSON.stringify(report)}]; if (typeof f === 'function') tell = f; delete g[${JSON.stringify(report)}]; } catch (e) { tell = null; }`} + // The fixture's function that hears how a placed document drew, taken + // off the window the same way. + let tellDrawn = null; + ${drawn === '' ? '' : `try { const f = g[${JSON.stringify(drawn)}]; if (typeof f === 'function') tellDrawn = f; delete g[${JSON.stringify(drawn)}]; } catch (e) { tellDrawn = null; }`} + const later = typeof g.queueMicrotask === 'function' ? g.queueMicrotask.bind(g) : null; + // Tells the fixture how document state s drew so far, once its place is + // known: a document whose place no one asked for yet is told when one + // does (pageOf), so this never asks for it before the page script ran. + const sendDrawn = (s) => { + s.due = false; + if (s.page === undefined) { s.wants = true; return; } + if (s.page === null || tellDrawn === null || stale()) return; + const k = (seed + '\\n' + s.page + '\\n' + s.place + '\\n' + s.loads + '\\n').split(seed + '\\n').join(''); + let at = ''; + try { + const o = location.origin; + at = o && o !== 'null' ? o + location.pathname : String(location.href).split(/[?#]/)[0]; + } catch (e) { at = ''; } + const f = {}; + s.numbers.forEach((c, p) => { f[p] = c; }); + try { const r = tellDrawn(k, at, [s.n[0], s.n[1], s.n[2]], f); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} + }; + // Marks s as having drawn or numbered a frame: it is told at the end of + // the task, once however many times it drew. + const drew = (s) => { + if (tellDrawn === null || s.due) return; + s.due = true; + if (later !== null) later(() => sendDrawn(s)); else sendDrawn(s); + }; // Taken as the script begins: the page's own scripts run after it. const WM = WeakMap; const Mp = Map; @@ -917,6 +965,7 @@ ${STREAM_JS} const at = srcOf(el, s.doc); const n = s.numbers.get(at) || 0; s.numbers.set(at, n + 1); + drew(s); return 'f' + at + '\\n' + n; }; const isFrame = (n) => { const t = n.localName; return t === 'iframe' || t === 'frame'; }; @@ -1068,7 +1117,7 @@ ${STREAM_JS} // An opener of another origin. given = null; } - const s = { doc, place, loads, counted, kind, up, frame, given, page: undefined, first: false, uncounted: false, opened: [], workers: new Mp(), els: new WM(), held: 0, bound: new WM(), mo: null, numbers: new Mp(), counts: new Mp(), streams: null }; + const s = { doc, place, loads, counted, kind, up, frame, given, page: undefined, first: false, uncounted: false, opened: [], workers: new Mp(), els: new WM(), held: 0, bound: new WM(), mo: null, numbers: new Mp(), counts: new Mp(), streams: null, n: [0, 0, 0], due: false, wants: false }; watch(s); return s; }; @@ -1133,6 +1182,7 @@ ${STREAM_JS} p = k === null ? null : k + s.frame; } s.page = p; + if (s.wants) { s.wants = false; drew(s); } } return s.page; }; @@ -1198,7 +1248,10 @@ ${STREAM_JS} try { const r = tell(at, why, aboveFor()); if (r && typeof r.catch === 'function') r.catch(() => {}); } catch (e) {} } } - return s.streams === false ? null : s.streams[name]; + if (s.streams === false) return null; + s.n[name === 'random' ? 0 : name === 'grv' ? 1 : 2]++; + drew(s); + return s.streams[name]; }; // A page's stacks show no URL keploy marked: an error goes on as it is. const thrown = (e) => e; diff --git a/playwright/test/fixtures/note.spec.ts b/playwright/test/fixtures/note.spec.ts new file mode 100644 index 0000000..b7d904e --- /dev/null +++ b/playwright/test/fixtures/note.spec.ts @@ -0,0 +1,26 @@ +import { faker } from '@faker-js/faker'; +import { test } from '../../dist'; + +// A test that draws in its test process (Math.random and faker) and in its +// page, whose page puts in frames at one path. NOTE_MORE=1 draws Math.random +// once more in each and getRandomValues twice more in the page, and puts in +// one more frame, as a replay that draws otherwise than its recording does. +const more = process.env.NOTE_MORE === '1' ? 1 : 0; + +test('noted', async ({ page }) => { + for (let i = 0; i <= more; i++) Math.random(); + faker.person.firstName(); + // goto waits for the frames' loads, and so for their draws. + await page.goto(`${process.env.DRAWS_BASE}/note?frames=${1 + more}&draws=${more}`); +}); + +// A page that puts in a frame before it draws, and never draws itself. +test('plain', async ({ page }) => { + await page.goto(`${process.env.DRAWS_BASE}/notef`); +}); + +// A document seeding cannot place (a CSP sandbox: no session storage), +// which puts in a frame. +test('unplaced', async ({ page }) => { + await page.goto(`${process.env.DRAWS_BASE}/cspf`); +}); diff --git a/playwright/test/note.test.mjs b/playwright/test/note.test.mjs new file mode 100644 index 0000000..0b8282d --- /dev/null +++ b/playwright/test/note.test.mjs @@ -0,0 +1,244 @@ +// Unit rows for a test's note (note.ts) and the agent client's note fields, +// against a fake agent. No browser: the browser rows are in rows.test.mjs. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import http from 'node:http'; +import { AgentClient, KeployError, MAX_NOTE } from '../dist/agent.js'; +import { MAX_DOCS, MAX_LINES, NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine } from '../dist/note.js'; +import { seatFaker, setAside, seatRandom, tally } from '../dist/random.js'; + +async function agent(answer) { + const calls = []; + const srv = http.createServer((req, res) => { + let body = ''; + req.on('data', (d) => { body += d; }); + req.on('end', () => { + const call = { path: req.url, body: body ? JSON.parse(body) : null }; + calls.push(call); + res.writeHead(200, { 'content-type': 'application/json' }); + res.end(JSON.stringify(answer(call))); + }); + }); + await new Promise((r) => srv.listen(0, '127.0.0.1', r)); + return { url: `http://127.0.0.1:${srv.address().port}`, calls, close: () => new Promise((r) => srv.close(r)) }; +} + +const note = (p = {}, d = {}, cut = 0) => ({ v: 1, p, d, cut }); +const doc = (at, n = [0, 0, 0], f = {}) => ({ at, n, f }); + +test('note: tally counts each draw under its name and gives the draw\'s number', () => { + const counts = new Map(); + let i = 0; + const a = tally(counts, 'a', () => ++i / 10); + const b = tally(counts, 'b', () => 0.5); + assert.deepEqual([a(), a(), b()], [0.1, 0.2, 0.5]); + assert.deepEqual([...counts], [['a', 2], ['b', 1]]); +}); + +test('note: a document\'s reports keep the larger of each count and its first address; a report that is no document\'s draws is not kept', () => { + const log = docLog(); + keepDocDraws(log, 'k', 'http://h/p', [1, 0, 2], { '/f': 1 }); + keepDocDraws(log, 'k', 'http://h/other', [3, 0, 1], { '/f': 0, '/g': 2 }); + assert.deepEqual([...log.docs], [['k', { at: 'http://h/p', n: [3, 0, 2], f: { '/f': 1, '/g': 2 } }]]); + // Each of these is dropped whole. + for (const [key, n, f] of [ + [1, [0, 0, 0], {}], ['', [0, 0, 0], {}], ['x', [0, 0], {}], ['x', [0, 0, 0, 0], {}], ['x', 'abc', {}], + ['x', [0, -1, 0], {}], ['x', [0, 1.5, 0], {}], ['x', [0, '1', 0], {}], ['x', [0, 2 ** 53, 0], {}], + ['x', [0, 0, 0], null], ['x', [0, 0, 0], []], ['x', [0, 0, 0], 'f'], ['x', [0, 0, 0], { '/f': -1 }], ['x', [0, 0, 0], { '/f': 'one' }], + ]) keepDocDraws(log, key, 'http://h/x', n, f); + assert.deepEqual([...log.docs.keys()], ['k']); + assert.equal(log.dropped.size, 0); + // An address is kept as the run's output shows one: one line. + keepDocDraws(log, 'nl', 'http://h/a\nb', [0, 0, 0], {}); + assert.equal(log.docs.get('nl').at, 'http://h/a?b'); +}); + +test('note: past MAX_DOCS documents a new one is not kept but counted as cut; a kept one still takes its reports', () => { + const log = docLog(); + for (let i = 0; i < MAX_DOCS; i++) keepDocDraws(log, `k${i}`, 'http://h/p', [1, 0, 0], {}); + keepDocDraws(log, 'new', 'http://h/p', [1, 0, 0], {}); + keepDocDraws(log, 'new', 'http://h/p', [2, 0, 0], {}); + keepDocDraws(log, 'newer', 'http://h/p', [1, 0, 0], {}); + keepDocDraws(log, 'k0', 'http://h/p', [5, 0, 0], {}); + assert.equal(log.docs.size, MAX_DOCS); + assert.equal(log.docs.has('new'), false); + assert.deepEqual(log.docs.get('k0').n, [5, 0, 0]); + assert.deepEqual([...log.dropped], ['new', 'newer']); + const n = readNote(noteText(new Map(), log)); + assert.equal(typeof n, 'object'); + assert.ok(n.cut >= 2, String(n.cut)); +}); + +test('note: the note is JSON of at most MAX_NOTE UTF-8 bytes, cutting the last documents and counting them', () => { + const small = docLog(); + keepDocDraws(small, 'a', 'http://h/a', [1, 2, 3], { '/f': 1 }); + const p = new Map([['Math.random', 4]]); + assert.deepEqual(JSON.parse(noteText(p, small)), { v: 1, p: { 'Math.random': 4 }, d: { a: { at: 'http://h/a', n: [1, 2, 3], f: { '/f': 1 } } }, cut: 0 }); + // Documents of multi-byte addresses: the bound is in bytes, not characters. + const big = docLog(); + for (let i = 0; i < 400; i++) keepDocDraws(big, `doc ${i}`, `http://h/${'é'.repeat(40)}${i}`, [i, 0, 0], {}); + const text = noteText(p, big); + assert.ok(Buffer.byteLength(text, 'utf8') <= MAX_NOTE, String(Buffer.byteLength(text, 'utf8'))); + const n = JSON.parse(text); + const kept = Object.keys(n.d); + assert.ok(kept.length > 0 && kept.length < 400, String(kept.length)); + // The first ones are kept, in order; the rest are counted as cut. + assert.deepEqual(kept, Array.from({ length: kept.length }, (_, i) => `doc ${i}`)); + assert.equal(n.cut, 400 - kept.length); + // One document more would take it past the bound. + const one = docLog(); + for (let i = 0; i <= kept.length; i++) keepDocDraws(one, `doc ${i}`, `http://h/${'é'.repeat(40)}${i}`, [i, 0, 0], {}); + one.docs.delete(`doc ${kept.length}`); + assert.equal(noteText(p, one), JSON.stringify({ ...n, cut: 0 })); +}); + +test('note: a note of exactly MAX_NOTE bytes is kept whole; past it, once no document is left, the streams are cut, the last first, and counted', () => { + // The bound is met through the key's length (an address is cut at 512). + const one = (a) => { const log = docLog(); keepDocDraws(log, `k${a}`, 'http://h/a', [0, 0, 0], {}); return log; }; + const base = Buffer.byteLength(noteText(new Map(), one('')), 'utf8'); + const exact = noteText(new Map(), one('a'.repeat(MAX_NOTE - base))); + assert.equal(Buffer.byteLength(exact, 'utf8'), MAX_NOTE); + assert.equal(JSON.parse(exact).cut, 0); + assert.equal(Object.keys(JSON.parse(exact).d).length, 1); + const over = JSON.parse(noteText(new Map(), one('a'.repeat(MAX_NOTE - base + 1)))); + assert.deepEqual([over.d, over.cut], [{}, 1]); + // A suite's own faker instance under a long name: streams go once no + // document is left, the last drawn first. + const long = '@faker-js/faker own ' + 'x'.repeat(MAX_NOTE); + const p = new Map([['Math.random', 2], [long, 1], ['late', 3]]); + const n = JSON.parse(noteText(p, one('http://h/a'))); + assert.deepEqual(n, { v: 1, p: { 'Math.random': 2 }, d: {}, cut: 3 }); + const huge = JSON.parse(noteText(new Map([[long, 1]]), docLog())); + assert.deepEqual(huge, { v: 1, p: {}, d: {}, cut: 1 }); +}); + +test('note: the test process counts the draws of each faker instance seated with counts under its stream, not those set aside; without counts it counts none', () => { + const r = { next: () => 0.25, seed: () => undefined }; + const counts = new Map(); + const unseatR = seatRandom(() => 0.5); + const unseat = seatFaker('0123abcd', [['faker', r]], counts); + r.next(); r.next(); + const put = setAside(); + assert.equal(r.next(), 0.25); + put(); + unseat(); unseatR(); + assert.deepEqual([...counts], [['@faker-js/faker faker', 2]]); + const r2 = { next: () => 0.25, seed: () => undefined }; + const unseat2 = seatFaker('0123abcd', [['faker', r2]]); + assert.notEqual(r2.next(), 0.25); + unseat2(); +}); + +test('note: readNote reads this package\'s note and says why it cannot read another', () => { + const good = note({ 'Math.random': 1 }, { k: doc('http://h/p', [1, 0, 0], { '/f': 2 }) }, 3); + assert.deepEqual(readNote(JSON.stringify(good)), good); + for (const [text, why] of [ + ['{', 'it is not JSON'], + ['[1]', 'it is not an object'], + ['null', 'it is not an object'], + ['"x"', 'it is not an object'], + [JSON.stringify({ ...good, v: 2 }), 'it is version 2, and this @keploy/playwright reads version 1'], + [JSON.stringify({ p: {}, d: {}, cut: 0 }), 'it is version null, and this @keploy/playwright reads version 1'], + [JSON.stringify({ ...good, p: null }), 'its test process counts are not counts'], + [JSON.stringify({ ...good, p: [1] }), 'its test process counts are not counts'], + [JSON.stringify({ ...good, p: { a: -1 } }), 'its test process counts are not counts'], + [JSON.stringify({ ...good, d: null }), 'its documents are not an object'], + [JSON.stringify({ ...good, d: [] }), 'its documents are not an object'], + [JSON.stringify({ ...good, d: { k: null } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: { ...doc('x'), at: 1 } } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: doc('x', [0, 0]) } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: doc('x', [0, 0, -1]) } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: doc('x', 'abc') } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: doc('x', [0, 0, 0], null) } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: doc('x', [0, 0, 0], []) } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, d: { k: doc('x', [0, 0, 0], { '/f': 0.5 }) } }), 'a document\'s counts are not counts'], + [JSON.stringify({ ...good, cut: -1 }), 'its cut is not a count'], + [JSON.stringify({ v: 1, p: {}, d: {} }), 'its cut is not a count'], + ]) assert.equal(readNote(text), why, text); + assert.equal(unreadNoteLine('t', 'it is not JSON'), 'keploy: the note keploy kept of how t\'s recording drew cannot be read (it is not JSON), ' + + 'so where its replay draws otherwise, no line names it: record the test again with this @keploy/playwright.'); +}); + +test('note: a replay that draws as its recording did has no difference; each stream, function and path that differs has one line', () => { + const rec = note({ 'Math.random': 3, '@faker-js/faker faker': 2 }, { a: doc('http://h/a', [2, 1, 1], { '/f': 1, '/g': 2 }), b: doc('http://h/b', [1, 0, 0]) }); + assert.deepEqual(differences('t', rec, rec), []); + const now = note({ 'Math.random': 1, 'other': 1 }, { a: doc('http://h/a2', [2, 0, 3], { '/f': 2 }), c: doc('http://h/c', [0, 0, 1]) }); + const lines = differences('t', rec, now); + assert.deepEqual(lines.map((l) => l.split(', so ')[0]), [ + 'keploy: t drew Math.random once in the test process at this replay and 3 times in its recording', + 'keploy: t drew @faker-js/faker faker 0 times in the test process at this replay and 2 times in its recording', + 'keploy: t drew other once in the test process at this replay and 0 times in its recording', + 'keploy: t: the document at http://h/a2 drew crypto.getRandomValues 0 times at this replay and once in its recording', + 'keploy: t: the document at http://h/a2 drew crypto.randomUUID 3 times at this replay and once in its recording', + 'keploy: t: the document at http://h/a2 put in 2 frame(s) at /f at this replay and 1 in its recording', + 'keploy: t: the document at http://h/a2 put in 0 frame(s) at /g at this replay and 2 in its recording', + 'keploy: t: the document at http://h/b drew Math.random 0 times at this replay and once in its recording', + 'keploy: t drew otherwise than its recording in 1 more way(s) than the lines above name.', + ]); + assert.equal(lines.length, MAX_LINES + 1); + // Each line names its next step. + assert.match(lines[0], /, so its draws from that stream after the first one that differs are not the recording's, .*register a worker fixture with this package's test\.extend.*\.$/); + assert.match(lines[3], /, so its draws of that function after the first one that differs are not the recording's: .*give each frame its own path\.$/); + assert.match(lines[5], /, so the frames at that path may take other places than in the recording and draw other numbers: give each frame at that path its own src path, or put them in in a fixed order and number\.$/); + // At most MAX_LINES: exactly that many are all said. + const eight = differences('t', note({ a: 1, b: 1, c: 1, d: 1, e: 1, f: 1, g: 1, h: 1 }), note()); + assert.equal(eight.length, MAX_LINES); + assert.ok(!eight.some((l) => l.includes('more way(s)'))); + // A path that is no one line is shown as one. + assert.match(differences('t', note({}, { a: doc('h', [0, 0, 0], { 'x\ny': 1 }) }), note({}, { a: doc('h') }))[0], / at x\?y at this replay/); +}); + +test('note: a document one note cut is compared with nothing', () => { + const rec = note({}, { a: doc('http://h/a', [1, 0, 0]) }, 1); + const now = note({}, { a: doc('http://h/a', [1, 0, 0]), b: doc('http://h/b', [4, 0, 0]) }); + assert.deepEqual(differences('t', rec, now), []); + assert.equal(differences('t', { ...rec, cut: 0 }, now).length, 1); + // A stream one note cut is compared with nothing too. + assert.deepEqual(differences('t', note({ 'Math.random': 1 }, {}, 1), note({ 'Math.random': 1, s: 2 })), []); + assert.deepEqual(differences('t', note({ 'Math.random': 1, s: 2 }), note({ 'Math.random': 1 }, {}, 1)), []); + assert.equal(differences('t', note({ 'Math.random': 1, s: 2 }), note({ 'Math.random': 1 })).length, 1); + const cutNow = note({}, {}, 1); + assert.deepEqual(differences('t', note({}, { a: doc('http://h/a', [1, 0, 0]) }), cutNow), []); + assert.equal(differences('t', note({}, { a: doc('http://h/a', [1, 0, 0]) }), note()).length, 1); +}); + +test('note: the no-note line names what keeps none and the next step', () => { + assert.match(NO_NOTE_LINE, /^keploy: keploy answered a seeded replay's begin with no note .*keploy v3\.7\.5 or older.*@keploy\/playwright 0\.1\.1 or older.*Record the tests again with a keploy that keeps notes/); +}); + +test('the client: a begin\'s answer gives the note keploy answered; one that is no note keploy keeps fails the test; an end sends a note it is given', async () => { + const atBound = 'n'.repeat(MAX_NOTE - 2) + 'é'; + for (const [body, want] of [[{ v: 1, status: 'ok', note: '{"v":1}' }, '{"v":1}'], [{ v: 1, status: 'ok', note: '' }, ''], [{ v: 1, status: 'ok' }, undefined], [{ v: 1, status: 'ok', note: atBound }, atBound]]) { + const a = await agent(() => body); + try { + assert.equal((await new AgentClient(a.url, '').begin({ name: 'a', file: 'a.spec.ts' }, 'seeded', 'ffffffff')).note, want); + } finally { await a.close(); } + } + for (const [note, re] of [ + [12, /^keploy: the keploy agent answered the begin of a with the note "12", which is no note keploy keeps \(a string of at most 16384 bytes\)$/], + [null, /with the note "null", which is no note/], + [{ v: 1 }, /with the note "\{\\"v\\":1\}", which is no note/], + ['n'.repeat(MAX_NOTE - 1) + 'é', /^keploy: the keploy agent answered the begin of a with the note "n{512}"\.\.\., which is no note keploy keeps/], + ]) { + const a = await agent(() => ({ v: 1, status: 'ok', note })); + try { + await assert.rejects(new AgentClient(a.url, '').begin({ name: 'a', file: 'a.spec.ts' }, 'seeded', 'ffffffff'), (err) => { + assert.ok(err instanceof KeployError, String(err)); + assert.match(err.message, re); + return true; + }); + } finally { await a.close(); } + } + const a = await agent(() => ({ v: 1, status: 'ok' })); + try { + const c = new AgentClient(a.url, ''); + await c.end({ name: 'a', file: 'a.spec.ts' }, 'passed', 'seeded', '{"v":1}'); + await c.end({ name: 'b', file: 'a.spec.ts' }, 'passed', 'seeded', ''); + await c.end({ name: 'c', file: 'a.spec.ts' }, 'passed'); + assert.deepEqual(a.calls.map((x) => x.body), [ + { name: 'a', file: 'a.spec.ts', outcome: 'passed', random: 'seeded', note: '{"v":1}' }, + { name: 'b', file: 'a.spec.ts', outcome: 'passed', random: 'seeded' }, + { name: 'c', file: 'a.spec.ts', outcome: 'passed' }, + ]); + } finally { await a.close(); } +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 3d3d795..67413da 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -12,6 +12,7 @@ import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, markWorker, workerMark, workerScript } from '../dist/random.js'; import { sharedWorkerLine } from '../dist/settle.js'; +import { NO_NOTE_LINE, readNote, unreadNoteLine } from '../dist/note.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; const PKG = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); @@ -156,13 +157,17 @@ async function playwright(specs, env, cwd = PKG, config = CONFIG) { const keployEnv = (url) => ({ KEPLOY_MOCK_AGENT: url, KEPLOY_MOCK_AGENT_TOKEN: 'tok', KEPLOY_MOCK_PROJECT: PKG, KEPLOY_TEST_PID: undefined }); const calls = (j) => j.events.filter((e) => e.agent).map((e) => e.agent); // Every call's path and body, each begin's seed shown as FRESH once it is -// checked to be a fresh one: 128 bits of hex, no two begins the same. +// checked to be a fresh one: 128 bits of hex, no two begins the same; and +// each end's note as NOTE once it is checked to be one this package reads. const FRESH = ''; +const NOTE = ''; function wire(j) { const seeds = calls(j).filter((c) => c.path === '/agent/scope/begin' && c.body.seed !== undefined).map((c) => c.body.seed); for (const sd of seeds) assert.match(sd, /^[0-9a-f]{32}$/); assert.equal(new Set(seeds).size, seeds.length, seeds.join(' ')); - return calls(j).map((c) => [c.path, c.body && c.body.seed !== undefined ? { ...c.body, seed: FRESH } : c.body]); + for (const c of calls(j)) if (c.path === '/agent/scope/end' && c.body.note !== undefined) assert.equal(typeof readNote(c.body.note), 'object', c.body.note); + return calls(j).map((c) => [c.path, c.body && c.body.seed !== undefined ? { ...c.body, seed: FRESH } + : c.body && c.body.note !== undefined ? { ...c.body, note: NOTE } : c.body]); } const ref = (file, name) => ({ name, file: `test/fixtures/${file}`, project: 'p1' }); @@ -193,11 +198,11 @@ test('a worker claims the run, each test begins before its beforeEach and ends w // b fails, so Playwright runs c and d in a new worker, which claims again. assert.deepEqual(wire(j), [ ['/agent/instance', claim], - ['/agent/scope/begin', { ...r('a'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('a'), outcome: 'passed' }], - ['/agent/scope/begin', { ...r('b'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('b'), outcome: 'failed' }], + ['/agent/scope/begin', { ...r('a'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('a'), outcome: 'passed', note: NOTE }], + ['/agent/scope/begin', { ...r('b'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('b'), outcome: 'failed', note: NOTE }], ['/agent/instance', claim], - ['/agent/scope/begin', { ...r('c'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('c'), outcome: 'skipped' }], - ['/agent/scope/begin', { ...r('d'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('d'), outcome: 'passed' }], + ['/agent/scope/begin', { ...r('c'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('c'), outcome: 'skipped', note: NOTE }], + ['/agent/scope/begin', { ...r('d'), canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/given', given], ['/agent/scope/end', { ...r('d'), outcome: 'passed', note: NOTE }], ]); assert.ok(calls(j).every((c) => c.auth === 'Bearer tok')); // KEPLOY_TEST_PID named the claiming worker (test a checks it). @@ -270,7 +275,7 @@ test('a test Playwright retries is not begun again: the retry fails, saying why, const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; assert.deepEqual(wire(j), [ ['/agent/instance', claim], - ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'failed' }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'failed', note: NOTE }], // The retry runs in a new worker, which claims the run again. ['/agent/instance', claim], ['/agent/refusal', retryRefusal], @@ -313,7 +318,7 @@ test('a test --repeat-each runs again is not begun again: the repeat fails, sayi const claim = { key: 'p1', dir: fs.realpathSync(PKG) }; assert.deepEqual(wire(j), [ ['/agent/instance', claim], - ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'passed' }], + ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], ['/agent/scope/end', { ...r, outcome: 'passed', note: NOTE }], // Playwright runs each repeat in a worker of its own. ['/agent/instance', claim], ['/agent/refusal', { code: 'test_run_again', message, remedy }], @@ -483,7 +488,7 @@ test('an unnamed Playwright project claims with the key ".", and its tests name assert.deepEqual(wire(j), [ ['/agent/instance', { key: '.', dir: fs.realpathSync(PKG) }], ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], - ['/agent/scope/end', { ...r, outcome: 'passed' }], + ['/agent/scope/end', { ...r, outcome: 'passed', note: NOTE }], ]); } finally { await a.close(); } }); @@ -503,7 +508,7 @@ test('a worker started above keploy\'s project claims with the project\'s folder assert.deepEqual(wire(j), [ ['/agent/instance', { key: 'p1', dir: fs.realpathSync(PKG) }], ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], - ['/agent/scope/end', { ...r, outcome: 'passed' }], + ['/agent/scope/end', { ...r, outcome: 'passed', note: NOTE }], ]); } finally { await a.close(); } }); @@ -519,7 +524,7 @@ test('when keploy names no project, a worker claims with the current folder, whi assert.deepEqual(wire(j), [ ['/agent/instance', { key: 'p1', dir: fs.realpathSync(cwd) }], ['/agent/scope/begin', { ...r, canSkip: true, random: 'seeded', seed: FRESH }], - ['/agent/scope/end', { ...r, outcome: 'passed' }], + ['/agent/scope/end', { ...r, outcome: 'passed', note: NOTE }], ]); } finally { await a.close(); } }); @@ -632,6 +637,8 @@ async function drawsServer() { const u = new URL(req.url, 'http://x'); // /csp is served with a CSP sandbox: an opaque origin, no session storage. if (u.pathname === '/csp') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('csp')); return; } + // /cspf: the same, with a frame of its own origin in it. + if (u.pathname === '/cspf') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('')); return; } if (u.pathname === '/held.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); held.script(() => res.end('')); return; } // parse.spec.ts's removed: /rhold.js is answered at /rrelease. if (u.pathname === '/rhold.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); rheld.push(res); return; } @@ -718,6 +725,15 @@ async function drawsServer() { } if (u.pathname === '/draws') { res.end(page(``)); return; } if (u.pathname === '/frame') { res.end(page('frame')); return; } + // note.spec.ts's page: draws Math.random once and getRandomValues twice + // more times after DRAWS, then puts in frames frames at /frame. + if (u.pathname === '/note') { + const extra = Number(u.searchParams.get('draws')), frames = Number(u.searchParams.get('frames')); + const draw = 'Math.random(); crypto.getRandomValues(new Uint8Array(1)); crypto.getRandomValues(new Uint8Array(1));'; + res.end(page(`${''.repeat(frames)}`)); + return; + } + if (u.pathname === '/notef') { res.end(''); return; } // /nest holds /frame of its own origin. if (u.pathname === '/nest') { res.end(page('')); return; } // /long/... is a page at an address of any length. @@ -904,6 +920,88 @@ test('seeded: a record seeds each test\'s pages from the fresh seed its begin se } finally { await a.close(); } }); +for (const engine of ['chromium', 'firefox', 'webkit']) { + test(`seeded (${engine}): a record's end sends keploy a note of how the test drew, in its test process and in each document, and the frames each document put in at each path; its replay prints one line for each difference, none when it draws as recorded, and says once per worker when keploy answered no note`, async () => { + const j = journal(); + const d = await drawsServer(); + const rec = await fakeAgent(j); + // What the recording kept: each test's seed and note. + let kept = {}; + // The note each begin is answered: the recording's by default. + let noteOf = (name) => ({ note: kept[name].note }); + const rep = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' && c.body.seed + ? { body: { v: 1, status: 'ok', seed: kept[c.body.name].seed, ...noteOf(c.body.name) } } : undefined)); + const run = async (url, more) => { + const from = j.events.length; + const { tests, out } = await playwright(['note.spec.ts'], { ...keployEnv(url), DRAWS_BASE: d.url, PW_BROWSER: engine, NOTE_MORE: more ? '1' : undefined }); + for (const [k, t] of Object.entries(tests)) assert.equal(t.status, 'passed', `${k}: ${t.error}\n${out}`); + const sent = calls({ events: j.events.slice(from) }); + const seeds = Object.fromEntries(sent.filter((c) => c.path === '/agent/scope/begin').map((c) => [c.body.name, c.body.seed])); + const notes = Object.fromEntries(sent.filter((c) => c.path === '/agent/scope/end').map((c) => [c.body.name, c.body.note])); + // What each test said, but the seeding's line and the unplaced document's. + const said = Object.fromEntries(Object.entries(tests).map(([k, t]) => [k, t.stdout.split('\n').filter((l) => l.startsWith('keploy:') && l !== SEEDED_LINE && !l.startsWith('keploy: in '))])); + return { seeds, notes, said, out }; + }; + const at = (n) => Object.values(n.d).map((x) => [x.at.replace(/:\d+\//, ':/'), x.n, x.f]).sort((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0)); + try { + const r = await run(rec.url, false); + kept = Object.fromEntries(Object.keys(r.seeds).map((k) => [k, { seed: r.seeds[k], note: r.notes[k] }])); + const noted0 = readNote(r.notes.noted); + assert.equal(typeof noted0, 'object', r.notes.noted); + // The test process drew Math.random once, and faker; the page drew + // DRAWS (Math.random twice, crypto.getRandomValues and + // crypto.randomUUID once) and put in one frame at /frame, whose + // document drew DRAWS. No place keeps the seed. + assert.deepEqual(Object.keys(noted0.p).sort(), ['@faker-js/faker faker', 'Math.random']); + assert.equal(noted0.p['Math.random'], 1); + assert.ok(noted0.p['@faker-js/faker faker'] > 0, String(noted0.p['@faker-js/faker faker'])); + assert.equal(noted0.cut, 0); + assert.deepEqual(at(noted0), [ + ['http://127.0.0.1:/frame', [2, 1, 1], {}], + ['http://127.0.0.1:/note', [2, 1, 1], { '/frame': 1 }], + ]); + for (const k of Object.keys(noted0.d)) assert.ok(!k.includes(r.seeds.noted), k); + // A document that puts in a frame before its place is known (it never + // draws) is told once its frame's document asks for it. + const plain = readNote(r.notes.plain); + assert.deepEqual(at(plain), [['http://127.0.0.1:/frame', [2, 1, 1], {}], ['http://127.0.0.1:/notef', [0, 0, 0], { '/frame': 1 }]]); + assert.ok(Object.keys(plain.d).includes('t0\n/notef\n0\n'), Object.keys(plain.d).join(' | ')); + // A document with no place is not in the note, nor are the frames it + // put in. + assert.deepEqual(readNote(r.notes.unplaced), { v: 1, p: {}, d: {}, cut: 0 }); + // A record says nothing of notes. + assert.deepEqual(r.said, { noted: [], plain: [], unplaced: [] }, r.out); + // A replay that draws as its recording did says nothing. + const same = await run(rep.url, false); + assert.deepEqual(same.said, { noted: [], plain: [], unplaced: [] }, same.out); + // One that draws once more in each place (getRandomValues twice more + // in the page), with one more frame at /frame, names each difference; + // faker drew as recorded. + const more = await run(rep.url, true); + const lines = more.said.noted.map((l) => l.replace(/:\d+\//, ':/')); + assert.deepEqual(lines.map((l) => l.split(', so ')[0]), [ + 'keploy: noted drew Math.random 2 times in the test process at this replay and once in its recording', + 'keploy: noted: the document at http://127.0.0.1:/note drew Math.random 3 times at this replay and 2 times in its recording', + 'keploy: noted: the document at http://127.0.0.1:/note drew crypto.getRandomValues 3 times at this replay and once in its recording', + 'keploy: noted: the document at http://127.0.0.1:/note put in 2 frame(s) at /frame at this replay and 1 in its recording', + 'keploy: noted: the document at http://127.0.0.1:/frame drew Math.random 2 times at this replay and 0 times in its recording', + 'keploy: noted: the document at http://127.0.0.1:/frame drew crypto.getRandomValues once at this replay and 0 times in its recording', + 'keploy: noted: the document at http://127.0.0.1:/frame drew crypto.randomUUID once at this replay and 0 times in its recording', + ], more.out); + assert.deepEqual([more.said.plain, more.said.unplaced], [[], []]); + // A replay keploy answered no note (none, or an empty one) says so once + // in the worker. + noteOf = (name) => (name === 'noted' ? {} : { note: '' }); + const none = await run(rep.url, false); + assert.deepEqual([...none.said.noted, ...none.said.plain, ...none.said.unplaced], [NO_NOTE_LINE], none.out); + // A note this package cannot read is named, and nothing is compared. + noteOf = (name) => ({ note: name === 'noted' ? '{"v":2}' : kept[name].note }); + const other = await run(rep.url, true); + assert.deepEqual(other.said, { noted: [unreadNoteLine('noted', 'it is version 2, and this @keploy/playwright reads version 1')], plain: [], unplaced: [] }, other.out); + } finally { await rep.close(); await rec.close(); await d.close(); } + }); +} + test('seeded: a replay draws the numbers its recording drew, from the seed keploy answers, not the one its begin sent', async () => { const j = journal(); const rec = await fakeAgent(j); @@ -1797,7 +1895,8 @@ test('KEPLOY_SEED_RANDOM=off, or no keploy: the test browser draws its own rando // Under keploy with it off, each begin says the browser draws its own // and sends no seed, and each end says it did. assert.deepEqual(calls(j).filter((c) => c.path === '/agent/scope/begin').map((c) => [c.body.random, c.body.seed]), Array(8).fill(['own', undefined])); - assert.deepEqual(calls(j).filter((c) => c.path === '/agent/scope/end').map((c) => c.body.random), Array(8).fill('own')); + // Unseeded, a test's end sends no note. + assert.deepEqual(calls(j).filter((c) => c.path === '/agent/scope/end').map((c) => [c.body.random, c.body.note]), Array(8).fill(['own', undefined])); } finally { await a.close(); } }); From 08b591926f22832605717e57df9a65be26e1ba85 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sun, 15 Nov 2026 00:42:21 +0530 Subject: [PATCH 43/73] docs(playwright): say what the run names where seeding cannot reach The README's "Limits of this version" listed the cases where a test draws numbers other than its recording's or where settle cannot wait. Those cases now print a line as the test runs, or at its replay, so the section becomes "What the run says when ...": the line each case prints and its next step. Two cases no line names yet are said plainly there: in WebKit, a worker that a Shared Worker starts, and two frames at one path that trade places and draw alike. - A keploy too old for given, seeds or notes; a call that does not end; a page that starts a Shared Worker. - Documents that cannot be placed, and workers that cannot be seeded, among them a Shared Worker started before keploy knows that no Service Worker's scope holds its URL (one a page's own script starts as it loads, or one in a page begun while another page registers a Service Worker), whose line names serviceWorkers: 'block'. - What moves a stream at replay (worker fixtures registered through Playwright's own test or mergeTests, libraries that draw on first use, a varying number of draws, frames at one path in a varying order or number), which the replay's note lines now name. What keploy leaves as it is (the app's server, crypto.subtle's keys, the run clock, a context the test makes itself) and what seeding adds to a page get their own sections; the calls outside every test move to the claim, and a context the test makes itself to settle. Signed-off-by: slayerjain --- playwright/README.md | 98 +++++++++++++++++++++++--------------------- 1 file changed, 51 insertions(+), 47 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 825d7ad..b9f94ee 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -40,12 +40,12 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco ## Under keploy -- **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. +- **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim: each test is skipped at its begin, after the group's hooks have started. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. - **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped )`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. -- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. +- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. -- **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see the limits below). +- **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see "What the run says when ..." below). - **Seeded random numbers.** Each test's browser draws its random numbers (`Math.random`, `crypto.getRandomValues`, `crypto.randomUUID`) from a seeded stream, so a replay draws the numbers its recording drew. A page that picks what to load at random (an A/B coin, such as stripe.js's at each page load) then makes the same calls in replay as in its recording, which keploy can answer. - **Where the seed comes from.** Each test's begin sends a seed drawn afresh (`seed`, 128 bits as hex). keploy answers the seed to draw from. A record answers the begin's own, and keeps it in the test's mapping entry (`seed:`, beside its `at:`). A replay (and keploy's auto-replay) answers the one the recording kept. So two tests draw different numbers, and recording a test again draws new ones: an ID the browser makes is not sent twice to a real server. The seed is not derived from the test's name, so a test whose entry is renamed keeps its seed with the rest of the entry. @@ -54,7 +54,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - A draw made while a stack trace is prepared is the worker's own, not the test's. Playwright maps stack frames through source maps, and the first time a test's stack reaches a file (a helper or page-object file's `test.step`, an `expect`, a read of an Error's stack), it parses that file's map with a sort that draws `Math.random`. A worker parses each file once, in whichever test reaches it first. So those draws do not depend on which tests its worker ran before it. - To tell those draws apart, every stack trace is prepared through a wrapper while a test is seeded: `Error.prepareStackTrace` reads back a wrapper of the function last set there (the same one on each read), and setting that wrapper sets its function. At the test's end `Error.prepareStackTrace` is again a plain property holding the function last set, unless the test defined that property itself, which it keeps. - A worker fixture registered with this package's `test.extend` or `test.use` draws the worker's own `Math.random` while it is set up, until it calls `use()` (or fails before). Playwright sets a worker fixture up in the first test that asks for it, after that test's seeding began, unless it is `auto`. Its draws (a test account made once for the worker's tests) would otherwise come from that test's stream and shift the test's own. A test fixture is the test's, and draws from its stream. Anything else that draws while a worker fixture is set up (a timer or an event handler of the test's) draws the worker's own too. - - With both, a test draws the same numbers whichever tests its worker ran before it: replayed alone, after keploy set an earlier test aside, or in a new worker after a test failed. The limits below name what a worker does once, in whichever test first needs it, that neither covers. + - With both, a test draws the same numbers whichever tests its worker ran before it: replayed alone, after keploy set an earlier test aside, or in a new worker after a test failed. "What the run says when ..." (below) names what a worker does once, in whichever test first needs it, that neither covers. - **Your own faker instances.** An instance the suite makes itself as a test file loads (`new Faker(...)`, `new SimpleFaker(...)`) seeds its own generator from the worker's `Math.random` then, so it draws other numbers each run. Give it to `seedFaker` and it is seeded as the prebuilt ones are, from its own stream of each test's seed: ```ts @@ -74,7 +74,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - a frame of its parent document's origin, by that document and by its place among the document's frames: the path its element asks for (`src`, or `about:srcdoc`, or `about:blank` for none) and how many frames the document had put in at that path before it, counted in the order the document holds the frame elements when it numbers them (as the task that put them in ends, or as a frame's document begins), never by their windows (the document finds a frame by its `frameElement`). For frames one task puts in out of document order, that is document order in Chromium and WebKit and the order they were put in in Firefox: the same each run, for a given app and engine. Frames at other paths, and the order the frames' documents load in, do not move it; it keeps its place whatever it loads, and one put in after a frame's removal takes the next number at its path. A frame in an open shadow root is placed so too. An `about:blank` frame does not draw its parent document's numbers; - a frame under a parent of another origin, which it cannot read (a cross-site frame), by the page it is in and its path. One with no session storage to count its documents in (a sandboxed frame without `allow-same-origin`) is never seeded (below). - A document the seeding cannot place draws the browser's own numbers, never another page's. The test then prints `keploy: in , the document at
drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. The address is the document's origin and path; for an opaque origin (`about:blank`, `about:srcdoc`, a `data:` URL, a sandboxed frame or page) it is the document's URL with no query or fragment, and `about:blank` for a window's first document that has no URL yet (Firefox runs init scripts there before it has one). The limits below list those documents. + A document the seeding cannot place draws the browser's own numbers, never another page's. The test then prints `keploy: in , the document at
drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. The address is the document's origin and path; for an opaque origin (`about:blank`, `about:srcdoc`, a `data:` URL, a sandboxed frame or page) it is the document's URL with no query or fragment, and `about:blank` for a window's first document that has no URL yet (Firefox runs init scripts there before it has one). "What the run says when ..." (below) lists those documents. - **What keploy is told.** The begin says what the test means to do (`random: "seeded"` or `"own"`). The end says what its pages drew from: `"seeded"` when a page opened in its seeded `context`, `"own"` when a page opened there unseeded, or when keploy answered no seed. When a test's calls still vary between runs, keploy can then say whether its pages were seeded, so the variation comes from somewhere the seeding does not reach, or whether seeding them is left to try. - **When a replay draws otherwise.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. So a seeded test's end also sends keploy a note (`note`, JSON of at most 16 KiB) of how many times the test process drew each of its streams (`Math.random` and each seeded faker instance), how many times each placed document drew each function, and how many frames each document put in at each path. A record keeps the note with the test's recording, and keploy never reads it. A replay's begin is answered the recorded note, and the replay's end prints one line for each difference, with its next step: - `keploy: drew Math.random 2 times in the test process at this replay and once in its recording, so ...`: register worker fixtures with this package's `test.extend`, use a library that draws on its first use once outside the tests, and draw a fixed number of times; @@ -90,50 +90,54 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco Any answer from the agent that this package cannot trust fails the test. That covers an agent it cannot reach, an error answer, an unreadable answer, and an answer that does not speak scope protocol v1. Nothing is retried. -## Limits of this version - -- Settle covers the `context` and `page` fixtures. A context the test makes with `browser.newContext()` is the test's own to close. -- Settle's rows run in Chromium, and its row for a call a new document makes as it loads runs in Firefox and WebKit too. Which calls the browser itself ends (a removed frame's) is proven in Chromium only. -- A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too: settle cannot tell it from a call of the new document. If the browser never reports its end, the test waits until its timeout fails it, naming the call. Settle fails toward waiting, never toward a cut call. -- Seeding's rows run in Chromium. Its rows for frames, for a popup that a page's first document opens, for top documents with no session storage, and for the test's own context init script run in Firefox and WebKit too. A popup that a page's first document opens with `window.open('about:blank')` draws the browser's own numbers and is named in every engine: in Chromium it has no session storage (its opener's origin is opaque), and in Firefox and WebKit it has no place, since its document begins before `window.open` returns. -- Seeding covers the pages of the `context` and `page` fixtures. These draws keep the browser's own random numbers, or the app's: - - the app's server; - - in the test process: `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`), and `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there differs each run: - - a faker instance the suite makes itself with `new Faker(...)` as a test file loads and does not give to `seedFaker` (see above): it seeds its own generator from the worker's `Math.random` then; - - data a suite makes once in `beforeAll` or a worker fixture (a test account made once for the worker's tests); - - a Service Worker, a worker that a Service Worker may answer, a worker that a worker starts in Chromium and Firefox, and the workers that one of these starts (each says so as the test runs, above); in WebKit and Firefox, a worker that a Shared Worker starts (in Firefox the Shared Worker's line names it; in WebKit, whose route reaches nothing a Shared Worker asks for after its script, nothing is said of it yet); - - a context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one; - - `crypto.subtle`'s keys. - - A page whose own draws vary in number before another script draws (a draw per timer tick) still shifts that function's stream within the document. - - Two frames at one path can still trade numbers where the order they are counted in is not the same each run. The replay names it (above) where the number of frames at that path, or what one of the two documents drew, differs from the recording; two that trade places and draw alike are not named: - - a frame's number at its path is how many frames its parent document had put in at that path before it, so frames at one path that the page or a third-party script puts in, in an order or a number that varies run to run (as their scripts load, in network order; a re-render that makes a varying number of frames), trade numbers. Chat and support widgets often put in `about:blank` frames as their scripts load; - - frames under a parent of another origin are counted by their origin in the tab, so two of one origin at one path take their numbers in the order their documents begin, which may change at replay; - - a frame put into a shadow root already in the page is not seen as it is put in, since the parent document watches only the nodes added to or removed from it outside its shadow roots: the frame is numbered at the document's next numbering (when the app next puts a frame in outside its shadow roots, or when a document of a frame not yet numbered, its own included, begins), so one put there before another frame at its path, at a moment that varies run to run, may take that frame's number. - - In the test process, what a worker does once, in whichever test first needs it, draws from that test's stream and shifts the test's later draws, so a replay where another test is the first to need it (one test alone, or after keploy set an earlier test aside) draws other numbers than the recording. This package keeps two such cases out of the test's stream: source maps parsed while a stack trace is prepared, and the setup of worker fixtures registered with its own `test.extend` or `test.use`. These it does not: - - a worker fixture registered through `@playwright/test`'s own `test`, or through a test type that `mergeTests` returned, whose `extend` and `use` are Playwright's own. The usual case is fixtures built on `@playwright/test` and joined to this package's `test` with `mergeTests`. Register worker fixtures with this package's `test.extend`, or with `extend` on a test type made from it that way; - - a library that draws `Math.random` the first time it is used, inside a test (a pool or a cache it builds on first use). - - In the test process one stream serves all of a test's `Math.random` draws, its own and those of the code it calls (Playwright's multipart form boundary among them), so a draw made a varying number of times (a retry loop's jitter) shifts the draws after it. -- The test process's clock is not seeded: it is keploy's run clock. A replayed test begins at its recorded begin, or later where the replay is behind, and its clock runs from there. So a value the test makes from the time during the test (`Date.now()`, `new Date()`) is off its recording's by as much as the replay ran faster or slower to that read, while a mocked dependency answers with the recording's: a name stamped with the time in milliseconds differs, and a test that checks it against the dependency's answer fails or is set aside. -- Seeding cannot place these documents, so they draw the browser's own numbers and are named (above): +## What the run says when ... + +keploy seeds, places and waits for what it can reach in the test browser and the test process. What it cannot reach, the run names as it happens, with the next step, as each case below says. Each line starts with `keploy:`. + +- **... keploy's agent is missing, too old or answers wrongly.** Any answer from the agent that this package cannot trust fails the test (above). A keploy too old for one feature fails or names only that feature: + - `given` needs the agent route `POST /agent/scope/given` (keploy/keploy#5016): keploy v3.6.192 and later serve it; with the enterprise keploy, that is an enterprise release on keploy v3.6.192 or later. An older keploy answers that route with a plain 404, and `given` fails the test with `keploy: this keploy agent answered 404 to POST /agent/scope/given, so it cannot take the given value : given needs keploy v3.6.192 or later (keploy/keploy#5016), or an enterprise release on keploy v3.6.192 or later; an older keploy answers 404 there`. Claim, begin, settle and end do not need that route. + - Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser and test process draw their own random numbers, and each test says so (`keploy: keploy answered the begin of with no seed, ...`, above). Claim, begin, settle and end work with it as before. + - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). +- **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. +- **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in , a page started the Shared Worker at
: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. +- **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in , the document at
drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: - a popup opened with noopener, as a link with `target="_blank"` opens one, or opened other than by `window.open` (a form, or a link with `rel="opener"`, that targets a new window); - - a popup at another origin than its opener's page, or whose opener's page has loaded another document by the time the popup's begins, or whose opener the seeding could not place (below); - - a frame of another origin inside one of those popups; + - a popup at another origin than its opener's page, or whose opener's page has loaded another document by the time the popup's begins, or whose opener the seeding could not place, and a frame of another origin inside one of those popups; + - a popup that a page's first document opens with `window.open('about:blank')`, in every engine: in Chromium it has no session storage (its opener's origin is opaque), and in Firefox and WebKit its document begins before `window.open` returns, before it has a place; - a frame its parent document cannot list (one in a closed shadow root, or in an `object` or `embed` element), one whose document began after the frame was removed, and a frame inside one; - - a frame under a parent of another origin that has no session storage to count its documents in (a sandboxed frame without `allow-same-origin`), alone under its parent or not, and a frame inside one: nothing tells its documents apart from the next ones at its path there (a reload, or a frame the app puts in again at its place) or from those of a frame beside it, so seeded, they could draw the same numbers. Its line says how many other frames its parent holds, counted once the parent's parser has ended (its `DOMContentLoaded`), so frames its markup puts in after a script that holds the parser count even when this frame drew, and loaded, before they came; when the parser has not ended by the test's end, counted then; when the frame is gone by then (its parent moved on to another document), the line names no other frame; - - a top document with no session storage that seeding can keep its count in, and a frame of its origin inside one. A document of an opaque origin has none: an `about:blank` loaded after its page's first document, a `data:` URL, a page served with a CSP `sandbox` without `allow-same-origin`, or a popup that a page's first document opens; nor has a document whose app has filled its session storage. Nothing tells such a document apart from the next one at its path in the tab (a reload, or another visit to that path), so seeded, they could draw the same numbers. So a test that loads `about:blank` (`page.goto('about:blank')`) and writes its page into it with `page.setContent` draws the browser's own numbers there: write into the page's first document (before its first `goto`), or serve the page from the app's origin, to seed it. A page's first document has no session storage in any engine either, but it is seeded: it is the one document of its kind in its tab (no later document there is marked, so the rule above holds for each of them), and before `newPage` returns the fixture marks it as the page's first with a value drawn afresh for each context and given only to that context's init script, so a page's own script cannot mark another document so. A frame of another origin inside such a document is placed as any frame under a parent of another origin (above). - - The end still says `"seeded"` for such a test: its `context` was seeded. A context init script of the test's own that draws random numbers as a document begins runs before the page is named, so that document has no place either; its line says it is in a page the test opened. -- Seeding's cost in a page: each document keeps a `MutationObserver` on its nodes (`childList`, its whole subtree) and looks at each node put in (and, while the document holds a numbered frame, each node taken out) for a frame; it walks the document (and its open shadow roots) to number its frames only when the app puts a frame in, or when a document of a frame not yet numbered begins. Most of the cost is the browser's own for such an observer, for each node put in, the parser's included. Measured as the median of five runs: in Chromium, seeding adds about 3 µs a node put in, about 170 ms to the load of a page of 40,000 elements, of which an observer that does nothing takes about 160 ms; in Firefox it adds about 45 ms to that load (27 for that observer), and in WebKit about 40 ms (29). A page that changes 300 times beside 20,000 elements or 2,000 open shadow roots takes within 2 ms of an unseeded one in Chromium and Firefox. -- The end reports only what the `context` fixture's pages drew from. A test none of whose pages came from it (an API-only test, a context it made itself) says nothing at its end, so keploy keeps what its begin said. -- Seeding keeps a count of each document path of a top document, and of a frame under a parent of another origin, in its origin's session storage in the tab, under the key `__keploy_random_loads`; a frame of its parent's origin is counted by its parent document. A frame under a parent of another origin with no session storage (a sandboxed frame) is never seeded (above). A top document with no session storage (an opaque origin) is never seeded either, but for a page's first document (above). The count is kept per origin, so the first documents of one page, or of frames under parents of other origins, at the same path in two origins draw the same numbers. Seeding also keeps two properties on each window, not enumerable: `__keploy_random_page` (the page's name) and `__keploy_random_doc` (what a popup or a frame reads to take its place). In a seeded page, `Math.random`, `crypto.getRandomValues`, `crypto.randomUUID` and `window.open` (which records the popups a document opens) are seeding's own functions, not the browser's native ones: `String(window.open)` shows no `[native code]`. -- Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. -- The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim. Each test is skipped at its begin, after the group's hooks have started; skipping the whole group before its hooks run needs a way to ask keploy for the set-aside tests first, which this version does not have. -- `given` needs the agent route `POST /agent/scope/given` (keploy/keploy#5016). keploy v3.6.192 and later serve it; with the enterprise keploy, that is an enterprise release on keploy v3.6.192 or later. An older keploy answers that route with a plain 404, and `given` fails the test with `keploy: this keploy agent answered 404 to POST /agent/scope/given, so it cannot take the given value : given needs keploy v3.6.192 or later (keploy/keploy#5016), or an enterprise release on keploy v3.6.192 or later; an older keploy answers 404 there`. Claim, begin, settle and end do not need that route. -- Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser and test process draw their own random numbers and print the line that says so (above). Claim, begin, settle and end work with it as before. + - a frame under a parent of another origin that has no session storage to count its documents in (a sandboxed frame without `allow-same-origin`), alone under its parent or not, and a frame inside one: nothing tells its documents apart from the next ones at its path there (a reload, or a frame the app puts in again at its place) or from those of a frame beside it. Its line says how many other frames its parent holds, counted once the parent's parser has ended (its `DOMContentLoaded`), so frames its markup puts in after a script that holds the parser count even when this frame drew, and loaded, before they came; when the parser has not ended by the test's end, counted then; when the frame is gone by then (its parent moved on to another document), the line names no other frame; + - a top document with no session storage that seeding can keep its count in, and a frame of its origin inside one. A document of an opaque origin has none: an `about:blank` loaded after its page's first document, a `data:` URL, or a page served with a CSP `sandbox` without `allow-same-origin`; nor has a document whose app has filled its session storage. So a test that loads `about:blank` (`page.goto('about:blank')`) and writes its page into it with `page.setContent` draws the browser's own numbers there: write into the page's first document (before its first `goto`), or serve the page from the app's origin, to seed it. A page's first document has no session storage in any engine either, but it is seeded: it is the one document of its kind in its tab (no later document there is marked, so the rule above holds for each of them), and before `newPage` returns the fixture marks it as the page's first with a value drawn afresh for each context and given only to that context's init script, so a page's own script cannot mark another document so. A frame of another origin inside such a document is placed as any frame under a parent of another origin (above); + - a document in which a context init script of the test's own draws random numbers as the document begins: it runs before the page is named, and its line says it is in a page the test opened. + + The end still says `"seeded"` for such a test: its `context` was seeded. +- **... a worker cannot be seeded.** It starts as the app asked, with no mark, and draws the browser's own numbers, and the test prints `keploy: in , the at
draws the browser's own random numbers, ...` with the reason and the next step, once for each address. A worker that one of these starts draws the browser's own numbers too, and the line says so. + - A Service Worker: keploy cannot put its seeding ahead of its script. Next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. + - A worker whose script a Service Worker may answer, since that Service Worker would see keploy's mark: a Web Worker of a page a Service Worker controls, and a Shared Worker of such a page or whose URL the scope of a registration keploy knows of holds. The line names the Service Worker and the same next step. + - A Shared Worker started before keploy knows that no Service Worker's scope holds its URL: one a script starts as its page loads, before the browser has answered keploy's question for the page's registrations; one in a page that began while another page of its origin was registering a Service Worker (keploy's Web Lock, above); one in a page that cannot read its Service Workers or its Web Locks (a sandboxed one). keploy marks a Shared Worker only once it knows, as a Service Worker that saw the mark would serve the app a URL other than the one it asked for. The line names the one remedy: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds these. + - In Firefox, a Shared Worker: Firefox's route does not reach its script. Next step: record and replay the test in Chromium or WebKit to seed it. + - In Chromium and Firefox, a worker that a worker starts: their routes reach no script of such a worker. Next step: record and replay the test in WebKit to seed it. + - A worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with keploy's mark (a Trusted Types policy). Next step: start it from its script's http(s) URL. + - A worker whose script a route of the test's own answered itself (`route.fulfill`). Next step: have that route call `route.continue()` or `route.fallback()` for the worker's script. + - In WebKit, a worker that a Shared Worker starts draws the browser's own numbers too, as WebKit's route reaches nothing a Shared Worker asks for after its script, but no line names it yet (in Firefox the Shared Worker's own line names it). +- **... a replay draws otherwise than its recording.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. The replay's end prints one line for each stream, document and frame path whose count differs from the recording's (above), naming the test, the two counts and the next step. These move a stream: + - in the test process, what a worker does once, in whichever test first needs it, when another test needs it first at replay (one test replayed alone, or after keploy set an earlier test aside): a worker fixture registered through `@playwright/test`'s own `test`, or through a test type that `mergeTests` returned, whose `extend` and `use` are Playwright's own (register worker fixtures with this package's `test.extend`, or with `extend` on a test type made from it), and a library that draws `Math.random` the first time it is used inside a test (a pool or a cache it builds on first use: use it once outside the tests). This package keeps two such cases out of the test's stream itself: source maps parsed while a stack trace is prepared, and the setup of worker fixtures registered with its own `test.extend` or `test.use` (above); + - in the test process, a draw made a varying number of times (a retry loop's jitter): one stream serves all of a test's `Math.random` draws, its own and those of the code it calls, Playwright's multipart form boundary among them; + - in a page, a document whose own draws vary in number before another script draws (a draw per timer tick); + - frames at one path that the page or a third-party script puts in, in an order or a number that varies run to run (as their scripts load, in network order; a re-render that makes a varying number of frames; chat and support widgets often put in `about:blank` frames as their scripts load): a frame's number at its path is how many frames its parent document had put in at that path before it. Frames under a parent of another origin are counted by their origin in the tab, so two of one origin at one path take their numbers in the order their documents begin. A frame put into a shadow root already in the page is numbered at the document's next numbering (when the app next puts a frame in outside its shadow roots, or when a document of a frame not yet numbered begins), since the parent document watches only the nodes added to or removed from it outside its shadow roots. The line names the frame path whose number of frames differs, and each document whose draws differ; two frames that trade places and draw alike are not named. Next step: give each frame at that path its own `src` path, or put them in in a fixed order and number. +- **... the test process draws outside a test's seed.** The worker's seeded line, once at its first seeded test, names what keeps its own numbers there: the test process's `crypto` (`randomUUID`, `getRandomValues`, `randomBytes`); `Math.random` and faker outside every test (`beforeAll`, `afterAll`, a test file as it loads) and while a worker fixture of this package's `test` is set up, so data made there (a test account made once for the worker's tests) differs each run; and a faker instance the suite makes itself as a test file loads (`new Faker(...)`), for which it names `seedFaker` (above). Where faker is installed but keploy cannot load it, the worker says so once with the reason. +- **... seeding is off.** The first such test of a worker prints what that costs (above). + +## What keploy leaves as it is + +- The app's server: keploy changes only the test browser and the test process, never the app or its server. +- `crypto.subtle`'s keys in the browser: they stay the browser's own. +- The test process's clock is keploy's run clock, not seeded. A replayed test begins at its recorded begin, or later where the replay is behind, and its clock runs from there. So a value the test makes from the time during the test (`Date.now()`, `new Date()`) is off its recording's by as much as the replay ran faster or slower to that read, while a mocked dependency answers with the recording's: a name stamped with the time in milliseconds differs, and a test that checks it against the dependency's answer fails or is set aside. Give such a value to keploy with `given` (above). +- A context the test makes itself (`browser.newContext()`, `browser.newPage()`), and a `context` fixture the suite overrides without building on this one: keploy does not seed it or wait for its calls, and the test closes it. The end reports only what the `context` fixture's pages drew from, so a test none of whose pages came from it (an API-only test, a context it made itself) says nothing at its end, and keploy keeps what its begin said. + +## What seeding adds to a page + +- Each document keeps a `MutationObserver` on its nodes (`childList`, its whole subtree) and looks at each node put in (and, while the document holds a numbered frame, each node taken out) for a frame; it walks the document (and its open shadow roots) to number its frames only when the app puts a frame in, or when a document of a frame not yet numbered begins. Most of the cost is the browser's own for such an observer, for each node put in, the parser's included. Measured as the median of five runs: in Chromium, seeding adds about 3 µs a node put in, about 170 ms to the load of a page of 40,000 elements, of which an observer that does nothing takes about 160 ms; in Firefox it adds about 45 ms to that load (27 for that observer), and in WebKit about 40 ms (29). A page that changes 300 times beside 20,000 elements or 2,000 open shadow roots takes within 2 ms of an unseeded one in Chromium and Firefox. +- Seeding keeps a count of each document path of a top document, and of a frame under a parent of another origin, in its origin's session storage in the tab, under the key `__keploy_random_loads`; a frame of its parent's origin is counted by its parent document. The count is kept per origin, so the first documents of one page, or of frames under parents of other origins, at the same path in two origins draw the same numbers. Seeding also keeps two properties on each window, not enumerable: `__keploy_random_page` (the page's name) and `__keploy_random_doc` (what a popup or a frame reads to take its place). In a seeded page, `Math.random`, `crypto.getRandomValues`, `crypto.randomUUID` and `window.open` (which records the popups a document opens) are seeding's own functions, not the browser's native ones: `String(window.open)` shows no `[native code]`. ## Releasing From b3a2216fa9ddfb72503f72f3b9190433086ebbff Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sun, 15 Nov 2026 01:52:29 +0530 Subject: [PATCH 44/73] fix(playwright): keep a test route's options on a worker's script A seeded test whose own route continued a worker's script with options (a catch-all that adds the header the app's server needs, or a stand-in at another URL) had them dropped: keploy's patched continue took no arguments and fetched the script at the app's URL as the browser asked, so the server answered 401 or 404 and the worker failed to start, with no line naming keploy. With seeding off the same test passed. keploy's fetch of the script now sends what the test's routes asked: continue's URL, method, headers and post data, and those a fallback gave (keploy's last route answers a script at a URL a fallback gave too). A Web Worker's seeding loads the app's script through the test's routes again, so they ask the same of that load. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script the test's routes sent on otherwise than the browser asked gets the answer to what they asked, whole, unseeded, and the test names it once with the next step (continue it with no options); with the browser's own headers it is seeded. Rows: a unit row for continue and fallback with headers (another, one taken off, or one the browser sent with another value) and with a URL, for Web and Shared Workers (classic and module); a browser row in Chromium, Firefox and WebKit for each on the page and on the context, with a server that needs the header or has only the stand-in. Signed-off-by: slayerjain --- playwright/README.md | 3 +- playwright/src/index.ts | 11 ++- playwright/src/random.ts | 108 +++++++++++++++++---- playwright/test/fixtures/overrides.spec.ts | 78 +++++++++++++++ playwright/test/random.test.mjs | 84 +++++++++++++++- playwright/test/rows.test.mjs | 69 +++++++++++++ 6 files changed, 323 insertions(+), 30 deletions(-) create mode 100644 playwright/test/fixtures/overrides.spec.ts diff --git a/playwright/README.md b/playwright/README.md index b9f94ee..db47d95 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above; one that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker's seeding then loads the app's script through the test's routes again, so they ask the same of that load. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (other headers, or a URL, method or post data of its own) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -118,6 +118,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - In Chromium and Firefox, a worker that a worker starts: their routes reach no script of such a worker. Next step: record and replay the test in WebKit to seed it. - A worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with keploy's mark (a Trusted Types policy). Next step: start it from its script's http(s) URL. - A worker whose script a route of the test's own answered itself (`route.fulfill`). Next step: have that route call `route.continue()` or `route.fallback()` for the worker's script. + - A Shared Worker whose script a route of the test's own sent on with a URL, method, headers or post data of its own (`route.continue` or `route.fallback` with options), in Chromium and WebKit: the seeding's load of the app's script, which these engines do not route, could not carry them. Next step: have that route continue or fall back with no options for the Shared Worker's script. - In WebKit, a worker that a Shared Worker starts draws the browser's own numbers too, as WebKit's route reaches nothing a Shared Worker asks for after its script, but no line names it yet (in Firefox the Shared Worker's own line names it). - **... a replay draws otherwise than its recording.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. The replay's end prints one line for each stream, document and frame path whose count differs from the recording's (above), naming the test, the two counts and the next step. These move a stream: - in the test process, what a worker does once, in whichever test first needs it, when another test needs it first at replay (one test replayed alone, or after keploy set an earlier test aside): a worker fixture registered through `@playwright/test`'s own `test`, or through a test type that `mergeTests` returned, whose `extend` and `use` are Playwright's own (register worker fixtures with this package's `test.extend`, or with `extend` on a test type made from it), and a library that draws `Math.random` the first time it is used inside a test (a pool or a cache it builds on first use: use it once outside the tests). This package keeps two such cases out of the test's stream itself: source maps parsed while a stack trace is prepared, and the setup of worker fixtures registered with its own `test.extend` or `test.use` (above); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 178d8e3..ab6ced8 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -235,16 +235,17 @@ const keployTestType = base.extend({ const mark = freshSeed(); // The workers' scripts: Firefox's route does not reach a Shared // Worker's, so there it is not marked, and says so. A worker whose - // script a route of the test's own answered says so too, once for each - // address. + // script a route of the test's own answered, or a Shared Worker whose + // script one sent on with options of its own, says so too, once for + // each address. await workerRoutes(context, { answered: (r) => inflight.answered(r), - routed: (at) => { + routed: (at, why) => { const a = pageAddress(at); - const k = `routed\n${a}`; + const k = `${why}\n${a}`; if (told.has(k)) return; told.add(k); - process.stdout.write(`${unplacedLine(name, a, 'routed')}\n`); + process.stdout.write(`${unplacedLine(name, a, why)}\n`); }, // A worker a seeded Web Worker started that keploy cannot mark, or // whose script the route does not reach: said as a page's is, once diff --git a/playwright/src/random.ts b/playwright/src/random.ts index e226437..49531c1 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -97,7 +97,9 @@ export function pageAddress(at: unknown): string { * a frame inside one, 'nostorage' for a top document with no session * storage to count its documents in (other than a page's first document), * or a frame of its origin inside one, 'routed' for a worker whose script - * a route of the test's own answered itself (workerRoutes), 'nested' for a + * a route of the test's own answered itself (workerRoutes), 'continued' for a + * Shared Worker whose script a route of the test's own sent on with options + * of its own (workerRoutes), 'nested' for a * Web Worker a seeded Web Worker started where the route does not reach its * script (NESTED_JS), 'service' for a * Service Worker a document registered, 'controlled' for a worker whose @@ -105,7 +107,7 @@ export function pageAddress(at: unknown): string { * before keploy knew whether a Service Worker may answer its script * (initScript). */ -export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'service' | 'controlled' | 'early' | 'nested'; +export type Unplaced = 'own' | 'frame' | 'popup' | 'uncounted' | 'nostorage' | 'worker' | 'shared' | 'routed' | 'continued' | 'service' | 'controlled' | 'early' | 'nested'; // The next step for a Service Worker the tests do not need. const BLOCK = 'if the tests do not need it, set serviceWorkers: \'block\' in the project\'s use (Playwright then registers none, and keploy seeds the workers it would have answered)'; @@ -115,7 +117,7 @@ const BLOCK = 'if the tests do not need it, set serviceWorkers: \'block\' in the const KIDS = 'and so does each worker it starts, so their draws may differ at replay'; // A worker keploy cannot reach: its line's reason (unplacedLine). -const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'early' | 'nested', string> = { +const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'continued' | 'service' | 'early' | 'nested', string> = { worker: 'it is a worker started from a script keploy cannot mark (a blob: or data: URL, or a URL the browser refused with keploy\'s mark, as a Trusted Types policy may), ' + 'so keploy cannot put its seeding ahead of the script; start it from the script\'s http(s) URL to seed it.', shared: 'it is a Shared Worker, and Firefox does not let keploy reach a Shared Worker\'s script to put its seeding ahead of it; ' + @@ -123,6 +125,10 @@ const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'early' | ' routed: 'a route of the test\'s own answered the worker\'s script itself (route.fulfill), so keploy cannot put its seeding ahead of the script, ' + 'and the worker\'s URL, which its location shows, carries keploy\'s __keploy_worker mark; have that route call route.continue() or route.fallback() ' + 'for the worker\'s script to seed it.', + continued: 'a route of the test\'s own sent the Shared Worker\'s script on with a URL, method, headers or post data of its own (route.continue or route.fallback with options), ' + + 'and Chromium and WebKit route nothing a Shared Worker loads after its script, as keploy\'s seeding loads the app\'s script, so that load could not carry them; ' + + 'keploy answers the worker with the script that route asked for, unseeded, and the worker\'s URL, which its location shows, carries keploy\'s __keploy_worker mark; ' + + 'have that route continue or fall back with no options for the Shared Worker\'s script to seed it.', service: 'keploy cannot put its seeding ahead of a Service Worker\'s script, and a worker whose script the Service Worker may answer starts unseeded (each one is named); ' + `${BLOCK}.`, early: 'it is a Shared Worker started before the browser told keploy which Service Workers the page\'s origin has (keploy asks as each document begins, ' + @@ -133,7 +139,7 @@ const UNREACHED: Record<'worker' | 'shared' | 'routed' | 'service' | 'early' | ' 'record and replay the test in WebKit to seed it.', }; -const UNPLACED: Record, string> = { +const UNPLACED: Record, string> = { own: 'it is in a page the test opened, and it drew before keploy named that page: a script the test added with context.addInitScript ' + 'runs in each of the page\'s documents before keploy\'s, and so does a draw the test makes there before newPage returns.', frame: 'it is a frame keploy cannot place among its parent\'s frames (one in a closed shadow root or in an object or embed element, ' + @@ -161,8 +167,8 @@ export function unplacedLine(name: string, at: string, why: Unplaced = 'popup', `a Service Worker${by === '' ? '' : ` (at ${by})`} may answer the worker's script, and it would see keploy's __keploy_worker mark in the script's URL, so keploy starts the worker as the app asked; ` + `${BLOCK}.`; } - if (why === 'worker' || why === 'shared' || why === 'routed' || why === 'service' || why === 'early' || why === 'nested') { - const what = why === 'shared' || why === 'early' ? 'Shared Worker' : why === 'service' ? 'Service Worker' : 'worker'; + if (why === 'worker' || why === 'shared' || why === 'routed' || why === 'continued' || why === 'service' || why === 'early' || why === 'nested') { + const what = why === 'shared' || why === 'early' || why === 'continued' ? 'Shared Worker' : why === 'service' ? 'Service Worker' : 'worker'; return `keploy: in ${name}, the ${what} at ${at} draws the browser's own random numbers, ${why === 'service' ? 'so its draws may differ at replay' : KIDS}: ${UNREACHED[why]}`; } const n = Math.floor(Number(others)); @@ -1831,9 +1837,10 @@ export type WorkerHooks = { answered?: (r: Request) => void; /** * A route of the test's own answered the script of the worker at at (its - * origin and path) itself: it runs unseeded, at its marked URL. + * origin and path) itself ('routed'), or sent a Shared Worker's on with + * options of its own ('continued'): it runs unseeded, at its marked URL. */ - routed?: (at: string) => void; + routed?: (at: string, why: 'routed' | 'continued') => void; /** * A seeded Web Worker started a worker at at (its origin and path, or its * scheme: blob:, data:) that keploy cannot mark ('worker') or whose script @@ -1881,6 +1888,21 @@ function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Ro return add(); } +/** What a test's route.continue or route.fallback asks a request be sent with. */ +type Asked = NonNullable[0]>; + +/** + * Whether headers a and b are the same: each name, in any case, with the + * same value (Playwright sends no header whose value is undefined). + */ +function sameHeaders(a: Record, b: Record): boolean { + const low = (h: Record): Map => + new Map(Object.entries(h).filter(([, v]) => v !== undefined).map(([k, v]) => [k.toLowerCase(), String(v)])); + const x = low(a); + const y = low(b); + return x.size === y.size && [...x].every(([k, v]) => y.get(k) === v); +} + /** * Answers each marked worker script the test's context asks for with * workerAnswer, under the status and headers the app's server answered for @@ -1912,7 +1934,15 @@ function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Ro * it). A route of the test's that answers it itself (fulfill) answers it so, * unseeded, and the test hears of it (routed); one that aborts it aborts * it. One that falls back, or none, leaves it to keploy's last route, the - * context's first, which answers it as above. + * context's first, which answers it as above, at the URL a fallback gave it + * too. What the test's continue and fallbacks ask (a URL, method, headers or + * post data of their own) is what keploy's fetch of the script sends, and a + * Web Worker's seeding loads the app's script through the test's routes + * again, so they ask the same of it. A Shared Worker's seeding cannot + * (Chromium and WebKit route none of its loads): one whose script the + * test's routes asked for otherwise than the browser sent it gets the answer + * to what they asked for, whole, unseeded, and the test hears of it + * (routed, 'continued'). */ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}, reach = false): Promise { const counts = new Map(); @@ -1924,10 +1954,25 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = return out; }; // The worker script loads handed on to the test's routes: each one's mark, - // and the route's own fulfill; and their URLs, for the last route. - const handed = new WeakMap(); + // the route's own fulfill, the headers the browser sent it with, and what + // the test's routes asked of it (their continue's and fallbacks' options); + // and the URLs it may reach the last route at (the app's, or one a + // fallback of the test's gave it). + type Handed = { m: WorkerMark; fulfill: Route['fulfill']; sent: Record; asked: Asked }; + const handed = new WeakMap(); const urls = new Set(); - const answer = async (route: Route, m: WorkerMark, own: Route['fulfill']): Promise => { + const ask = (h: Handed, o: Asked | undefined): void => { + for (const [k, v] of Object.entries(o ?? {})) if (v !== undefined) (h.asked as Record)[k] = v; + }; + // Whether the test's routes asked for a worker's script otherwise than the + // browser sent it: a URL, method or post data of their own, or headers + // other than the browser's. + const otherwise = (h: Handed): boolean => Object.entries(h.asked) + .some(([k, v]) => k !== 'headers' || !sameHeaders(v as Record, h.sent)); + // A route's fetch with o is the request the test's routes asked for: o's + // URL, method, headers and post data, or the request's (the URL the app + // asked for, or one a fallback gave, and what fallbacks gave it). + const answer = async (route: Route, m: WorkerMark, own: Route['fulfill'], h?: Handed, o: Asked = {}): Promise => { const fulfill = async (o: Parameters[0]): Promise => { await own(o); hooks.answered?.(route.request()); @@ -1938,12 +1983,21 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } const n = counts.get(m.token) ?? 0; counts.set(m.token, n + 1); + // Chromium and WebKit route nothing a Shared Worker loads after its + // script, so its seeding's load of the app's script goes out as the + // browser asks: one the test's routes asked for otherwise gets what they + // asked for, whole, unseeded, and is named. + const whole = (m.kind === 's' || m.kind === 'x') && h !== undefined && otherwise(h); let status = 200; let headers: Record = { 'content-type': 'text/javascript' }; try { - const got = await route.fetch({ url: m.url }); - if (!got.ok()) { + const got = await route.fetch(o); + if (whole || !got.ok()) { await fulfill({ status: got.status(), headers: headersOf(got.headers()), body: await got.body() }); + if (whole) { + const u = new URL(m.url); + hooks.routed?.(u.origin + u.pathname, 'continued'); + } return; } status = got.status(); @@ -1979,15 +2033,31 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await answer(route, m, own); return; } - handed.set(r, { m, fulfill: own }); + const h: Handed = { m, fulfill: own, sent: r.headers(), asked: {} }; + handed.set(r, h); urls.add(new URL(m.url).href); - route.continue = async () => answer(route, m, own); + const fallback = route.fallback.bind(route); + route.continue = async (o) => { + ask(h, o); + await answer(route, m, own, h, o); + }; + route.fallback = async (o) => { + ask(h, o); + if (o?.url !== undefined) { + try { + urls.add(new URL(o.url).href); + } catch { + // Not a URL: Playwright refuses it. + } + } + await fallback(o); + }; route.fulfill = async (o) => { const u = new URL(m.url); - hooks.routed?.(u.origin + u.pathname); + hooks.routed?.(u.origin + u.pathname, 'routed'); await own(o); }; - await route.fallback({ url: m.url }); + await fallback({ url: m.url }); }; const last = async (route: Route): Promise => { const h = handed.get(route.request()); @@ -1995,7 +2065,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await route.fallback(); return; } - await answer(route, h.m, h.fulfill); + await answer(route, h.m, h.fulfill, h); }; const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM); const lastUrl = (u: URL): boolean => urls.has(u.href); diff --git a/playwright/test/fixtures/overrides.spec.ts b/playwright/test/fixtures/overrides.spec.ts new file mode 100644 index 0000000..7c0813e --- /dev/null +++ b/playwright/test/fixtures/overrides.spec.ts @@ -0,0 +1,78 @@ +import fs from 'node:fs'; +import type { Page, Route } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER). Each test adds a route of +// its own, on its page or on its context, that continues or falls back with +// what the app's server needs for a worker's script: a header it answers +// 401 without (/ov/a*.js), or a stand-in at another URL for a script it does +// not have (/ov/o*.js, answered by /ov/m*.js). The last continues with the +// browser's own headers (/ov/p*.js). Then the test's page starts a Web +// Worker and a Shared Worker at those scripts and keeps what each sent (its +// kind, location and draws), or 'error', under the test's title. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +// With Playwright blocking Service Workers, keploy knows at once that none +// can answer a Shared Worker's script, and marks it. +test.use({ serviceWorkers: 'block' }); + +const start = async (page: Page, k: string): Promise => { + await page.goto(`${base}/ov`); + keep(test.info().title, await page.evaluate((x) => Promise.all([ + new Promise((resolve) => { + const w = new Worker(`/ov/${x}w.js`); + w.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + }), + new Promise((resolve) => { + const s = new SharedWorker(`/ov/${x}s.js`); + s.port.onmessage = (e) => resolve(e.data); + s.onerror = () => resolve('error'); + // A Shared Worker whose script failed may report nothing. + setTimeout(() => resolve('none'), 10000); + }), + ]), k)); +}; +const auth = (r: Route): { headers: Record } => ({ headers: { ...r.request().headers(), 'x-auth': 'k' } }); +const standIn = (r: Route): { url: string } => ({ url: r.request().url().replace('/ov/o', '/ov/m') }); + +test('continue headers on page', async ({ page }) => { + await page.route('**/*', (r) => r.continue(auth(r))); + await start(page, 'a'); +}); + +test('continue headers on context', async ({ context, page }) => { + await context.route('**/*', (r) => r.continue(auth(r))); + await start(page, 'a'); +}); + +test('continue url on page', async ({ page }) => { + await page.route('**/ov/o*.js', (r) => r.continue(standIn(r))); + await start(page, 'o'); +}); + +test('continue url on context', async ({ context, page }) => { + await context.route('**/ov/o*.js', (r) => r.continue(standIn(r))); + await start(page, 'o'); +}); + +test('fallback headers on page', async ({ page }) => { + await page.route('**/*', (r) => r.fallback(auth(r))); + await start(page, 'a'); +}); + +test('fallback headers on context', async ({ context, page }) => { + await context.route('**/*', (r) => r.fallback(auth(r))); + await start(page, 'a'); +}); + +test('fallback url on page', async ({ page }) => { + await page.route('**/ov/o*.js', (r) => r.fallback(standIn(r))); + await start(page, 'o'); +}); + +test('browser headers on page', async ({ page }) => { + await page.route('**/*', (r) => r.continue({ headers: r.request().headers() })); + await start(page, 'p'); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 2749428..f5beb76 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -2037,9 +2037,10 @@ test('a worker\'s seeding: an error the browser\'s own call throws in a seeded g // A fake browser context for workerRoutes, in Playwright's order: a page's // routes before its context's, the last added first; a route that falls // back hands the request on, at the URL its fallback gave, to the next -// route that matches it there. What answered it is what ask returns: the -// fetches and the fulfill, continue or abort that ended it, or, when no -// route did, the fallback it went on to the server with. +// route that matches it there, with the headers it gave. What answered it +// is what ask returns: the fetches (each one's URL, and the headers it sent +// when a route gave some) and the fulfill, continue or abort that ended it, +// or, when no route did, the fallback it went on to the server with. function routed() { const r = {}; const target = () => { @@ -2067,10 +2068,15 @@ function routed() { let at = url; let over; let ended = false; - const request = { url: () => at, resourceType: () => as.type, headers: () => as.headers, postData: () => as.body ?? null }; + const request = { url: () => at, resourceType: () => as.type, headers: () => over?.headers ?? as.headers, postData: () => as.body ?? null }; const route = { request: () => request, - fetch: async (o) => { did.push(['fetch', o.url]); if (answer instanceof Error) throw answer; return answer; }, + fetch: async (o = {}) => { + const h = o.headers ?? over?.headers; + did.push(h === undefined ? ['fetch', o.url ?? at] : ['fetch', o.url ?? at, h]); + if (answer instanceof Error) throw answer; + return answer; + }, fulfill: async (o) => { ended = true; did.push(['fulfill', { ...o, body: String(o.body) }]); }, continue: async (o) => { ended = true; did.push(['continue', o?.url ?? at]); }, abort: async () => { ended = true; did.push(['abort']); }, @@ -2321,6 +2327,74 @@ test('workerRoutes keeps keploy\'s route ahead of every route the test adds to i assert.equal(context.routes[2].match(new URL('http://h/v.js')), false); }); +test('workerRoutes sends a worker\'s script on with what the test\'s continue and fallbacks asked (a URL or headers of their own), at a URL a fallback gave too; a Shared Worker\'s, classic or module, that they asked for otherwise than the browser sent it gets the answer to what they asked, whole, unseeded, and is heard; the browser\'s own headers in another case or with one given as undefined, an option left undefined, or none, seed it', async () => { + const { r, context } = routed(); + const heard = []; + await workerRoutes(context, { routed: (at, why) => heard.push([at, why]) }); + const page = r.page(); + let mine = (route) => route.continue(); + await page.route(() => true, (route) => mine(route)); + const t = 'cd'.repeat(16); + const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; + const ask = (m, body, dest) => r.ask(m, answerOf(200, body), { type: 'script', headers: { 'sec-fetch-dest': dest, accept: '*/*' } }, page); + const sent = (dest) => ({ 'sec-fetch-dest': dest, accept: '*/*' }); + const auth = (dest) => ({ ...sent(dest), 'x-auth': 'k' }); + const seeded = (url, h, n, kind, own, tok = t) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body: workerAnswer(tok, n, kind, own) }]]; + const whole = (url, h, body) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body }]]; + // A Web Worker's: keploy's fetch sends what the test's route asked, and + // its seeding then loads the app's script through the test's routes again. + const w = markWorker('http://h/w.js', 'w', t); + mine = (route) => route.continue({ headers: auth('worker') }); + assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 0, 'w', 'http://h/w.js')); + mine = (route) => route.continue({ url: 'http://h/stand.js' }); + assert.deepEqual(await ask(w, 'stand()', 'worker'), seeded('http://h/stand.js', undefined, 1, 'w', 'http://h/w.js')); + // Fallbacks: keploy's last route answers it with what they gave, at the + // URL one gave too. + mine = (route) => route.fallback({ headers: auth('worker') }); + assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 2, 'w', 'http://h/w.js')); + mine = (route) => route.fallback({ url: 'http://h/stand.js' }); + assert.deepEqual(await ask(w, 'stand()', 'worker'), seeded('http://h/stand.js', undefined, 3, 'w', 'http://h/w.js')); + assert.deepEqual(heard, []); + // A Shared Worker's: the browser's own headers (in another case), an + // option left undefined, or none, seed it. + const ts = 'ce'.repeat(16); + const s = markWorker('http://h/s.js', 's', ts); + const other = { 'Sec-Fetch-Dest': 'sharedworker', ACCEPT: '*/*' }; + mine = (route) => route.continue({ headers: other, url: undefined }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), seeded('http://h/s.js', other, 0, 's', 'http://h/s.js', ts)); + mine = (route) => route.continue(); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), seeded('http://h/s.js', undefined, 1, 's', 'http://h/s.js', ts)); + // A header given as undefined, which Playwright does not send. + const none = { ...sent('sharedworker'), 'x-none': undefined }; + mine = (route) => route.continue({ headers: none }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), seeded('http://h/s.js', none, 2, 's', 'http://h/s.js', ts)); + assert.deepEqual(heard, []); + // Another header, one taken off, one the browser sent with another value, + // another URL, or a fallback's header: the answer to what they asked, + // whole, and heard. + mine = (route) => route.continue({ headers: auth('sharedworker') }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', auth('sharedworker'), 'app()')); + const fewer = { ...sent('sharedworker'), accept: undefined }; + mine = (route) => route.continue({ headers: fewer }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', fewer, 'app()')); + const changed = { ...sent('sharedworker'), accept: 'text/javascript' }; + mine = (route) => route.continue({ headers: changed }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', changed, 'app()')); + mine = (route) => route.continue({ url: 'http://h/stand.js' }); + assert.deepEqual(await ask(s, 'stand()', 'sharedworker'), whole('http://h/stand.js', undefined, 'stand()')); + mine = (route) => route.fallback({ headers: auth('sharedworker') }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', auth('sharedworker'), 'app()')); + // A module Shared Worker's too. + const tx = 'cf'.repeat(16); + const x = markWorker('http://h/x.js', 'x', tx); + mine = (route) => route.continue({ headers: auth('sharedworker') }); + assert.deepEqual(await ask(x, 'app()', 'sharedworker'), whole('http://h/x.js', auth('sharedworker'), 'app()')); + mine = (route) => route.continue(); + assert.deepEqual(await ask(x, 'app()', 'sharedworker'), seeded('http://h/x.js', undefined, 1, 'x', 'http://h/x.js', tx)); + assert.deepEqual(heard, [...Array(5).fill(['http://h/s.js', 'continued']), ['http://h/x.js', 'continued']]); + assert.match(unplacedLine('t', 'http://h/s.js', 'continued'), /^keploy: in t, the Shared Worker at http:\/\/h\/s\.js draws the browser's own random numbers, and so does each worker it starts, so their draws may differ at replay: a route of the test's own sent the Shared Worker's script on with a URL, method, headers or post data of its own .* have that route continue or fall back with no options for the Shared Worker's script to seed it\.$/); +}); + test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module one whose policy refuses data: with its seeding in the answer; each request it fulfills is answered', async () => { const { r, context } = routed(); const answered = []; diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 67413da..8fadc6a 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -651,6 +651,21 @@ async function drawsServer() { } if (u.pathname === '/swh/asked') { if (swheld.length > 0) res.end(''); else swasked.push(res); return; } if (u.pathname === '/swh/release') { for (const r of swheld.splice(0)) r.end('self.addEventListener(\'fetch\', () => {});'); res.end(''); return; } + // overrides.spec.ts's: /ov/aw.js and /ov/as.js answer 401 to a request + // with no x-auth: k; /ov/ow.js and /ov/os.js are not there (404), and + // their stand-ins /ov/mw.js and /ov/ms.js are; /ov/pw.js and /ov/ps.js + // ask nothing. Each Web Worker (w) and Shared Worker (s) sends its kind, + // its location and what it drew. + if (u.pathname === '/ov') { res.writeHead(200, { 'content-type': 'text/html' }); res.end(page('ov')); return; } + if (u.pathname === '/ov/ow.js' || u.pathname === '/ov/os.js') { res.writeHead(404, { 'content-type': 'text/plain' }); res.end('none'); return; } + const ov = /^\/ov\/([amp])([ws])\.js$/.exec(u.pathname); + if (ov !== null) { + if (ov[1] === 'a' && req.headers['x-auth'] !== 'k') { res.writeHead(401, { 'content-type': 'text/plain' }); res.end('no auth'); return; } + const sent = `{ kind: '${ov[1]}${ov[2]}', href: location.href, draws: ${DRAWS} }`; + res.writeHead(200, { 'content-type': 'text/javascript' }); + res.end(ov[2] === 's' ? `onconnect = (e) => e.ports[0].postMessage(${sent});` : `postMessage(${sent});`); + return; + } // workers.spec.ts's workers, and the scripts they load; /wk/ref answers // the Referer it was sent. if (u.pathname === '/wk/ref') { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(req.headers.referer ?? 'none'); return; } @@ -1630,6 +1645,60 @@ for (const engine of ENGINES) { }); } +// overrides.spec.ts's tests, and the scripts each one's page starts its +// workers at (/ov/w.js and /ov/s.js). +const OV = [['continue headers on page', 'a'], ['continue headers on context', 'a'], ['continue url on page', 'o'], ['continue url on context', 'o'], + ['fallback headers on page', 'a'], ['fallback headers on context', 'a'], ['fallback url on page', 'o'], ['browser headers on page', 'p']]; + +// The marks a document at /ov of page t0 gives the workers overrides.spec.ts +// starts there. +function ovMarks(seed, k) { + const made = []; + class Worker { constructor(url) { made.push(workerMark(String(url))); } } + documentAt(initScript(seed), '/ov', new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }) + .run(`new Worker('/ov/${k}w.js'); new SharedWorker('/ov/${k}s.js');`); + return made; +} + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a worker whose script the test's own route, on its page or its context, continues or falls back with a header the app's server needs or with a stand-in's URL runs what that route asked for; a Web Worker's is seeded (its seeding loads the app's script through the test's routes again) at the app's URL; a Shared Worker's runs unseeded and is named in Chromium and WebKit (nothing a Shared Worker loads after its script is routed there), and is seeded when the route continues with the browser's own headers; Firefox's route reaches no Shared Worker's script, the test's neither, so there it fails as with no keploy`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + try { + const r = await drawRun(j, { ...keployEnv(rec.url), PW_BROWSER: engine }, ['overrides.spec.ts'], OV.map(([t]) => t)); + for (const [title, k] of OV) { + const [web, shared] = r.drew[title]; + const [wm, sm] = ovMarks(r.seeds[title], k); + const ran = k === 'o' ? 'm' : k; + assert.equal(web.kind, `${ran}w`, `${title}: ${JSON.stringify(web)}`); + assert.match(web.href, new RegExp(`^http://127\\.0\\.0\\.1:\\d+/ov/${k}w\\.js$`), title); + assert.deepEqual(web.draws, workerAt(`http://h/ov/${k}w.js`, workerScript(wm.token, 0, 'w')).draws(), title); + // Firefox's route reaches no Shared Worker's script, keploy's or the + // test's: what the test's route asks of it never reaches the server, + // and the worker fails as it does with no keploy. + if (engine === 'firefox' && k !== 'p') assert.ok(['error', 'none'].includes(shared), `${title}: ${JSON.stringify(shared)}`); + else assert.equal(shared.kind, `${ran}s`, `${title}: ${JSON.stringify(shared)}`); + const at = `http://127.0.0.1:/ov/${k}s.js`; + const said = r.tests[title].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, `)).map((l) => l.replace(/:\d+\//, ':/')); + const lines = [sharedWorkerLine(title, at)]; + const model = workerAt(`http://h/ov/${k}s.js`, workerScript(sm.token, 0, 's')).draws(); + if (engine === 'firefox') { + lines.push(unplacedLine(title, at, 'shared')); + } else if (k === 'p') { + assert.deepEqual(shared.draws, model, title); + assert.match(shared.href, new RegExp(`^http://127\\.0\\.0\\.1:\\d+/ov/ps\\.js$`), title); + } else { + lines.push(unplacedLine(title, at, 'continued')); + assert.notDeepEqual(shared.draws, model, title); + assert.match(shared.href, new RegExp(`^http://127\\.0\\.0\\.1:\\d+/ov/${k}s\\.js\\?__keploy_worker=s\\.[0-9a-f]{32}$`), title); + } + assert.deepEqual(said.sort(), lines.sort(), title); + } + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); + } finally { await rec.close(); } + }); +} + // The marks a document at /nw of page t0 gives the workers nested.spec.ts's // page starts, and the marks those Web Workers' seedings give theirs. function nestedMarks(seed) { From cd612ebcd3b0101c610238d4d1cebe8c3df8e476 Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sun, 15 Nov 2026 05:04:48 +0530 Subject: [PATCH 45/73] fix(playwright): answer a Web Worker's second script load from its first A seeded test whose own route treated a Web Worker's script loads apart (one that answers once with times: 1, one that picks the worker's script by its destination, one that counts its loads) had the script loaded twice: keploy's answer to the worker's own load runs the seeding, which then loads the app's script from the app's URL, and that second load went through the test's routes again. A route spent on the first load sent the second to the server without the header it needed, and the worker failed to start, with no line naming keploy; with seeding off it ran. keploy now keeps the script its fetch got for a Web Worker's own load and answers the seeding's load of the app's script with it, ahead of the test's routes, so they and the app's server see the script loaded once, as with seeding off. It tells that load by its URL, its Referer (the worker's marked URL, or what the script's Referrer-Policy sends to its own origin: Chromium sends that for both kinds of worker, Firefox and WebKit for a module one) and its kind (importScripts for a classic worker, a static import for a module one, in each engine's terms); a worker's script load keploy handed on to the test's routes is never one. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker's script is still asked for twice. A Shared Worker whose script a route continues with the request's own URL or method is now seeded, as one continued with the browser's own headers is: only headers, a URL or a method other than the request's, or post data, send it on otherwise. The README also says where a worker that runs at its marked URL with no seeding shows the mark to the server. Rows, in Chromium, Firefox and WebKit, seeded and with seeding off: a route that adds a header once (on the page and on the context, classic and module), one that adds it by destination, the same once under a Referrer-Policy of no-referrer and origin, a counting route (2 loads in both modes, and the same scripts asked of the server), a Web Worker answered by the test's route and a Shared Worker continued with a header at one address (each named), and two routes whose last one adds the header. Unit rows: how the seeding's load is told and taken once, a load keploy handed on, the policy read, and the request's own URL and method. Signed-off-by: slayerjain --- playwright/README.md | 2 +- playwright/src/random.ts | 132 +++++++++++++++++++-- playwright/test/fixtures/once.spec.ts | 104 ++++++++++++++++ playwright/test/fixtures/overrides.spec.ts | 7 +- playwright/test/random.test.mjs | 130 +++++++++++++++++++- playwright/test/rows.test.mjs | 85 ++++++++++++- 6 files changed, 435 insertions(+), 25 deletions(-) create mode 100644 playwright/test/fixtures/once.spec.ts diff --git a/playwright/README.md b/playwright/README.md index db47d95..df202dc 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts` (keploy's answer loads a classic worker's own script so), of the load of a module worker's own script, and of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer; and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), so the server is asked for a worker's script twice: by keploy, for those, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker's seeding then loads the app's script through the test's routes again, so they ask the same of that load. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (other headers, or a URL, method or post data of its own) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), and keploy answers a Web Worker's seeding's load of the app's script with the script it got so, ahead of the test's routes: the test's routes and the app's server see a Web Worker's script loaded once, as with seeding off. keploy tells that load by its URL, its `Referer` (the worker's marked URL, or what the script's `Referrer-Policy` sends to its own origin: none under `no-referrer`, the origin under `origin` and `strict-origin`) and its kind (`importScripts` for a classic worker, a static import for a module one), and a worker's script load it handed on the test's routes is never one. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker's seeding's load of the app's script gets the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 49531c1..d182d67 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1903,13 +1903,65 @@ function sameHeaders(a: Record, b: Record): bool return x.size === y.size && [...x].every(([k, v]) => y.get(k) === v); } +/** Whether URLs a and b are the same, fragment aside (false for one that is no URL). */ +function sameUrl(a: string, b: string): boolean { + try { + return noFragment(new URL(a).href) === noFragment(new URL(b).href); + } catch { + return false; + } +} + +function noFragment(url: string): string { + const i = url.indexOf('#'); + return i < 0 ? url : url.slice(0, i); +} + +const POLICIES = new Set(['no-referrer', 'no-referrer-when-downgrade', 'same-origin', 'origin', 'strict-origin', 'origin-when-cross-origin', 'strict-origin-when-cross-origin', 'unsafe-url']); + +/** + * The Referer a worker whose script is at at sends with a request to its + * own origin, under policy, the Referrer-Policy its script was answered + * with (the last token it names that is a policy, in any case): none ('') + * under no-referrer, its origin under origin and strict-origin, else its + * URL with no fragment or user info. + */ +export function ownReferer(at: string, policy = ''): string { + let p = ''; + for (const t of policy.split(',')) { + const v = t.trim().toLowerCase(); + if (POLICIES.has(v)) p = v; + } + const u = new URL(noFragment(at)); + if (p === 'no-referrer') return ''; + if (p === 'origin' || p === 'strict-origin') return `${u.origin}/`; + u.username = ''; + u.password = ''; + return u.href; +} + +/** + * Whether r is a load of the kind a Web Worker's seeding makes of the app's + * script (workerAnswer): importScripts for a classic one (sec-fetch-dest + * script where the route sees it; Chromium types it other), a static import + * for a module one (sec-fetch-dest worker; Chromium types it script). + */ +function seedingLoad(r: Request, kind: 'w' | 'm'): boolean { + const dest = r.headers()['sec-fetch-dest']; + if (dest !== undefined) return dest === (kind === 'w' ? 'script' : 'worker'); + return r.resourceType() === (kind === 'w' ? 'other' : 'script'); +} + /** * Answers each marked worker script the test's context asks for with * workerAnswer, under the status and headers the app's server answered for * the script with no mark (a worker's policy comes from its script's * answer), and a module Web Worker's seeding (marked 'p') with - * workerScript. So the app's server is asked for a worker's script twice: - * here, for that answer, and by the worker, for the app's script. An answer + * workerScript. A Web Worker's seeding's load of the app's script is + * answered with what keploy's fetch for that answer got (kept), so the app's + * server is asked for a Web Worker's script once, here; Chromium and WebKit + * route nothing a Shared Worker loads after its script, so it is asked for a + * Shared Worker's twice: here, and by the worker, for the app's script. An answer * that is not ok goes to the worker as the server gave it; when keploy * cannot fetch the script, the worker gets its seeding with no header of the * app's and loads the app's script itself, as the browser would. Each @@ -1937,9 +1989,12 @@ function sameHeaders(a: Record, b: Record): bool * context's first, which answers it as above, at the URL a fallback gave it * too. What the test's continue and fallbacks ask (a URL, method, headers or * post data of their own) is what keploy's fetch of the script sends, and a - * Web Worker's seeding loads the app's script through the test's routes - * again, so they ask the same of it. A Shared Worker's seeding cannot - * (Chromium and WebKit route none of its loads): one whose script the + * Web Worker's seeding's load of the app's script gets what that fetch got, + * ahead of the test's routes, which so see the script once: a route that + * answers once (times), counts its loads or picks the worker's script by its + * destination sees what it sees with seeding off. A Shared Worker's seeding + * loads the app's script past every route (Chromium and WebKit route none + * of its loads): one whose script the * test's routes asked for otherwise than the browser sent it gets the answer * to what they asked for, whole, unseeded, and the test hears of it * (routed, 'continued'). @@ -1958,17 +2013,59 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // the test's routes asked of it (their continue's and fallbacks' options); // and the URLs it may reach the last route at (the app's, or one a // fallback of the test's gave it). - type Handed = { m: WorkerMark; fulfill: Route['fulfill']; sent: Record; asked: Asked }; + type Handed = { m: WorkerMark; at: string; fulfill: Route['fulfill']; sent: Record; method: string; asked: Asked }; const handed = new WeakMap(); const urls = new Set(); const ask = (h: Handed, o: Asked | undefined): void => { for (const [k, v] of Object.entries(o ?? {})) if (v !== undefined) (h.asked as Record)[k] = v; }; // Whether the test's routes asked for a worker's script otherwise than the - // browser sent it: a URL, method or post data of their own, or headers - // other than the browser's. - const otherwise = (h: Handed): boolean => Object.entries(h.asked) - .some(([k, v]) => k !== 'headers' || !sameHeaders(v as Record, h.sent)); + // browser sent it: headers other than the browser's, a URL other than the + // app's, a method other than the browser's, or post data. + const otherwise = (h: Handed): boolean => Object.entries(h.asked).some(([k, v]) => { + if (k === 'headers') return !sameHeaders(v as Record, h.sent); + if (k === 'url') return !sameUrl(String(v), h.m.url); + if (k === 'method') return v !== h.method; + return true; + }); + // The app's script a seeded Web Worker's seeding loads next (workerAnswer), + // kept from keploy's fetch for the worker's own load until the browser + // asks for it, and answered so ahead of the test's routes: the test's + // routes and the app's server see a Web Worker's script loaded once, as + // with seeding off. A seeding's load is told by its URL, the app's, and + // its Referer: the worker's marked URL, or, under a Referrer-Policy that + // sends less to the worker's own origin, what that policy sends (Chromium + // sends that for both kinds, Firefox and WebKit for a module worker), and + // by its kind (seedingLoad). Each is kept at each such Referer, with how + // many are kept at each URL. Chromium and WebKit route nothing a Shared + // Worker loads after its script, so none is kept for one. + type Kept = { kind: 'w' | 'm'; keys: string[]; answer: { status: number; headers: Record; body: Buffer } }; + const kept = new Map(); + const keptAt = new Map(); + const keyOf = (referer: string, url: string): string => `${referer}\n${noFragment(url)}`; + const keep = (h: Handed, kind: 'w' | 'm', answer: Kept['answer']): void => { + const policy = answer.headers['referrer-policy']; + const keys = [...new Set([keyOf(ownReferer(h.at), h.m.url), keyOf(ownReferer(h.at, policy), h.m.url)])]; + const k: Kept = { kind, keys, answer }; + for (const key of keys) kept.set(key, [...(kept.get(key) ?? []), k]); + const u = noFragment(h.m.url); + keptAt.set(u, (keptAt.get(u) ?? 0) + 1); + }; + // The kept script r is the seeding's load of, taken, or null. + const take = (r: Request): Kept['answer'] | null => { + const k = kept.get(keyOf(r.headers().referer ?? '', r.url()))?.find((x) => seedingLoad(r, x.kind)); + if (k === undefined) return null; + for (const key of k.keys) { + const rest = (kept.get(key) ?? []).filter((x) => x !== k); + if (rest.length > 0) kept.set(key, rest); + else kept.delete(key); + } + const u = noFragment(r.url()); + const n = (keptAt.get(u) ?? 1) - 1; + if (n > 0) keptAt.set(u, n); + else keptAt.delete(u); + return k.answer; + }; // A route's fetch with o is the request the test's routes asked for: o's // URL, method, headers and post data, or the request's (the URL the app // asked for, or one a fallback gave, and what fallbacks gave it). @@ -2002,6 +2099,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } status = got.status(); headers = headersOf(got.headers()); + if (h !== undefined && (m.kind === 'w' || m.kind === 'm')) keep(h, m.kind, { status, headers, body: await got.body() }); } catch { // The worker loads the app's script itself, and fails as it would. } @@ -2011,7 +2109,15 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = const r = route.request(); const m = workerMark(r.url()); if (m === null) { - await route.fallback(); + // A seeded Web Worker's seeding's load of the app's script: answered + // with what keploy fetched for the worker's own load. A worker's + // script load keploy handed on is none. + const k = handed.has(r) ? null : take(r); + if (k === null) await route.fallback(); + else { + await route.fulfill(k); + hooks.answered?.(r); + } return; } // A seeded Web Worker's word of a worker keploy cannot mark: heard, then @@ -2033,7 +2139,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await answer(route, m, own); return; } - const h: Handed = { m, fulfill: own, sent: r.headers(), asked: {} }; + const h: Handed = { m, at: r.url(), fulfill: own, sent: r.headers(), method: r.method(), asked: {} }; handed.set(r, h); urls.add(new URL(m.url).href); const fallback = route.fallback.bind(route); @@ -2067,7 +2173,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } await answer(route, h.m, h.fulfill, h); }; - const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM); + const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM) || keptAt.has(noFragment(u.href)); const lastUrl = (u: URL): boolean => urls.has(u.href); const route = context.route; const addLast = async (): Promise => { diff --git a/playwright/test/fixtures/once.spec.ts b/playwright/test/fixtures/once.spec.ts new file mode 100644 index 0000000..4ef8f3c --- /dev/null +++ b/playwright/test/fixtures/once.spec.ts @@ -0,0 +1,104 @@ +import fs from 'node:fs'; +import type { Page, Route } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER), seeded and with seeding +// off. Each test adds routes of its own that treat a worker script's loads +// apart: one that answers once (times: 1), one that picks the worker's +// script by its destination, one that counts its loads, then starts Web +// Workers at /ov/w.js (a needs x-auth: k; n and g need it too, under +// Referrer-Policy no-referrer and origin; p asks nothing) and keeps what +// each sent (its kind, location and draws), or 'error', under its title. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +test.use({ serviceWorkers: 'block' }); + +type Start = { url: string; type?: 'classic' | 'module'; shared?: boolean }; +const start = async (page: Page, list: Start[]): Promise => { + await page.goto(`${base}/ov`); + const got: unknown[] = []; + // One at a time, in order: each worker's place is its order. + for (const s of list) { + got.push(await page.evaluate((x) => new Promise((resolve) => { + if (x.shared) { + const w = new SharedWorker(x.url, { type: x.type ?? 'classic' }); + w.port.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + setTimeout(() => resolve('none'), 10000); + return; + } + const w = new Worker(x.url, { type: x.type ?? 'classic' }); + w.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + }), s)); + } + keep(test.info().title, got); +}; +const auth = (r: Route): { headers: Record } => ({ headers: { ...r.request().headers(), 'x-auth': 'k' } }); + +test('once on page', async ({ page }) => { + await page.route('**/ov/aw.js', (r) => r.continue(auth(r)), { times: 1 }); + await start(page, [{ url: '/ov/aw.js' }]); +}); + +test('once on context, module', async ({ context, page }) => { + await context.route('**/ov/aw.js', (r) => r.continue(auth(r)), { times: 1 }); + await start(page, [{ url: '/ov/aw.js', type: 'module' }]); +}); + +// The worker's script by its destination: sec-fetch-dest worker where the +// route sees it; Chromium shows the route none, and types it script. +test('by destination', async ({ page }) => { + await page.route('**/ov/aw.js', (r) => { + const q = r.request(); + const dest = q.headers()['sec-fetch-dest']; + return (dest === undefined ? q.resourceType() === 'script' : dest === 'worker') ? r.continue(auth(r)) : r.continue(); + }); + await start(page, [{ url: '/ov/aw.js' }]); +}); + +test('once under no-referrer', async ({ page }) => { + await page.route('**/ov/nw.js', (r) => r.continue(auth(r)), { times: 1 }); + await start(page, [{ url: '/ov/nw.js' }]); +}); + +test('once under no-referrer, module', async ({ page }) => { + await page.route('**/ov/nw.js', (r) => r.continue(auth(r)), { times: 1 }); + await start(page, [{ url: '/ov/nw.js', type: 'module' }]); +}); + +test('once under origin, module', async ({ page }) => { + await page.route('**/ov/gw.js', (r) => r.continue(auth(r)), { times: 1 }); + await start(page, [{ url: '/ov/gw.js', type: 'module' }]); +}); + +test('counted', async ({ page }) => { + let n = 0; + await page.route('**/ov/pw.js', (r) => { + n += 1; + return r.continue(); + }); + await start(page, [{ url: '/ov/pw.js' }, { url: '/ov/pw.js', type: 'module' }]); + keep('counted-n', n); +}); + +// A Web Worker whose script the test's route answers itself, then a Shared +// Worker at the same address whose script it sends on with a header: each +// is named. +test('answered then continued at one address', async ({ page }) => { + let n = 0; + await page.route('**/ov/as.js', (r) => { + n += 1; + return n === 1 ? r.fulfill({ contentType: 'text/javascript', body: 'postMessage({ kind: \'own\' });' }) : r.continue(auth(r)); + }); + await start(page, [{ url: '/ov/as.js' }, { url: '/ov/as.js', shared: true }]); +}); + +// A page route that falls back with the browser's own headers, then a +// context route that adds the one the server needs: the last one's win. +test('two routes', async ({ context, page }) => { + await context.route('**/ov/a*.js', (r) => r.continue(auth(r))); + await page.route('**/ov/a*.js', (r) => r.fallback({ headers: r.request().headers() })); + await start(page, [{ url: '/ov/aw.js' }, { url: '/ov/as.js', shared: true }]); +}); diff --git a/playwright/test/fixtures/overrides.spec.ts b/playwright/test/fixtures/overrides.spec.ts index 7c0813e..ebd42fb 100644 --- a/playwright/test/fixtures/overrides.spec.ts +++ b/playwright/test/fixtures/overrides.spec.ts @@ -7,7 +7,7 @@ import { test } from '../../dist'; // what the app's server needs for a worker's script: a header it answers // 401 without (/ov/a*.js), or a stand-in at another URL for a script it does // not have (/ov/o*.js, answered by /ov/m*.js). The last continues with the -// browser's own headers (/ov/p*.js). Then the test's page starts a Web +// browser's own headers, and its own URL and method (/ov/p*.js). Then the test's page starts a Web // Worker and a Shared Worker at those scripts and keeps what each sent (its // kind, location and draws), or 'error', under the test's title. const base = process.env.DRAWS_BASE ?? ''; @@ -76,3 +76,8 @@ test('browser headers on page', async ({ page }) => { await page.route('**/*', (r) => r.continue({ headers: r.request().headers() })); await start(page, 'p'); }); + +test('own url and method on page', async ({ page }) => { + await page.route('**/ov/p*.js', (r) => r.continue({ url: r.request().url(), method: r.request().method() })); + await start(page, 'p'); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index f5beb76..f4009a6 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, ownReferer, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -2068,7 +2068,7 @@ function routed() { let at = url; let over; let ended = false; - const request = { url: () => at, resourceType: () => as.type, headers: () => over?.headers ?? as.headers, postData: () => as.body ?? null }; + const request = { url: () => at, resourceType: () => as.type, headers: () => over?.headers ?? as.headers, method: () => as.method ?? 'GET', postData: () => as.body ?? null }; const route = { request: () => request, fetch: async (o = {}) => { @@ -2341,8 +2341,8 @@ test('workerRoutes sends a worker\'s script on with what the test\'s continue an const auth = (dest) => ({ ...sent(dest), 'x-auth': 'k' }); const seeded = (url, h, n, kind, own, tok = t) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body: workerAnswer(tok, n, kind, own) }]]; const whole = (url, h, body) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body }]]; - // A Web Worker's: keploy's fetch sends what the test's route asked, and - // its seeding then loads the app's script through the test's routes again. + // A Web Worker's: keploy's fetch sends what the test's route asked (its + // seeding's load of the app's script gets what that fetch got: a row below). const w = markWorker('http://h/w.js', 'w', t); mine = (route) => route.continue({ headers: auth('worker') }); assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 0, 'w', 'http://h/w.js')); @@ -2393,6 +2393,128 @@ test('workerRoutes sends a worker\'s script on with what the test\'s continue an assert.deepEqual(await ask(x, 'app()', 'sharedworker'), seeded('http://h/x.js', undefined, 1, 'x', 'http://h/x.js', tx)); assert.deepEqual(heard, [...Array(5).fill(['http://h/s.js', 'continued']), ['http://h/x.js', 'continued']]); assert.match(unplacedLine('t', 'http://h/s.js', 'continued'), /^keploy: in t, the Shared Worker at http:\/\/h\/s\.js draws the browser's own random numbers, and so does each worker it starts, so their draws may differ at replay: a route of the test's own sent the Shared Worker's script on with a URL, method, headers or post data of its own .* have that route continue or fall back with no options for the Shared Worker's script to seed it\.$/); + // Its own URL (fragment aside) or method given back, as the request has + // them, seeds it; another method, or post data, does not. + heard.length = 0; + mine = (route) => route.continue({ url: route.request().url(), method: route.request().method() }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), seeded('http://h/s.js', undefined, 8, 's', 'http://h/s.js', ts)); + mine = (route) => route.continue({ url: 'http://h/s.js#a' }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), seeded('http://h/s.js#a', undefined, 9, 's', 'http://h/s.js', ts)); + mine = (route) => route.continue({ method: 'POST' }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', undefined, 'app()')); + mine = (route) => route.continue({ postData: 'x' }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', undefined, 'app()')); + mine = (route) => route.continue({ url: 'not a url' }); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('not a url', undefined, 'app()')); + assert.deepEqual(heard, Array(3).fill(['http://h/s.js', 'continued'])); + // Two routes: the last one's options win. A page route's fallback with + // the browser's own headers, then a context route's continue that adds one. + heard.length = 0; + mine = (route) => route.fallback({ headers: sent('sharedworker') }); + await context.route(() => true, (route) => route.continue({ headers: auth('sharedworker') })); + assert.deepEqual(await ask(s, 'app()', 'sharedworker'), whole('http://h/s.js', auth('sharedworker'), 'app()')); + assert.deepEqual(heard, [['http://h/s.js', 'continued']]); +}); + +test('workerRoutes answers a seeded Web Worker\'s seeding\'s load of the app\'s script with what its fetch for the worker\'s own load got, ahead of the test\'s routes, once: told by its URL, its Referer (the worker\'s marked URL, or what the script\'s Referrer-Policy sends to its own origin) and its kind (importScripts for a classic worker, a static import for a module one); none for a script load keploy handed on, a Shared Worker\'s, or an answer not ok', async () => { + const { r, context } = routed(); + const answered = []; + await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); + const page = r.page(); + const hits = []; + let pass = false; + await page.route(() => true, (route) => { hits.push(route.request().url()); return pass ? route.fallback() : route.continue(); }); + const app = (policy, status = 200) => answerOf(status, 'app()', { 'content-type': 'text/javascript', ...(policy === undefined ? {} : { 'referrer-policy': policy }) }); + const start = (m, policy, status) => r.ask(m, app(policy, status), { type: 'script', headers: { referer: 'http://h/pg', 'sec-fetch-dest': workerMark(m).kind === 's' ? 'sharedworker' : 'worker' } }, page); + // The seeding's load: Chromium shows no destination and types it. + const load = (url, referer, type, dest) => r.ask(url, null, { type, headers: { ...(referer === null ? {} : { referer }), ...(dest === undefined ? {} : { 'sec-fetch-dest': dest }) } }, page); + const kept = (policy) => [['fulfill', { status: 200, headers: { 'content-type': 'text/javascript', ...(policy === undefined ? {} : { 'referrer-policy': policy }) }, body: 'app()' }]]; + const on = (url) => [['continue', url]]; + // A classic worker under its default policy: its importScripts, in each + // engine's terms, once; then a load there goes on. + const w = markWorker('http://h/w.js', 'w', 'd1'.repeat(16)); + assert.equal((await start(w))[0][0], 'fetch'); + assert.equal(r.match(new URL('http://h/w.js#f')), true); + assert.deepEqual(await load('http://h/w.js', w, 'other'), kept()); + assert.equal(r.match(new URL('http://h/w.js')), false); + assert.deepEqual(await load('http://h/w.js', w, 'other'), on('http://h/w.js')); + await start(w); + assert.deepEqual(await load('http://h/w.js', w, 'script', 'script'), kept()); + assert.deepEqual(hits, ['http://h/w.js', 'http://h/w.js', 'http://h/w.js']); + assert.deepEqual(answered, Array(4).fill('http://h/w.js')); + // Not a load of the seeding's kind, nor one with another Referer or at + // another URL: each goes on, and the seeding's load still gets it. + await start(w); + assert.deepEqual(await load('http://h/w.js', w, 'script'), on('http://h/w.js')); + assert.deepEqual(await load('http://h/w.js', w, 'script', 'worker'), on('http://h/w.js')); + assert.deepEqual(await load('http://h/w.js', 'http://h/pg', 'other'), on('http://h/w.js')); + assert.deepEqual(await load('http://h/w.js', null, 'other'), on('http://h/w.js')); + assert.deepEqual(await load('http://h/w.js?b=1', w, 'other'), on('http://h/w.js?b=1')); + assert.deepEqual(await load('http://h/w.js', w, 'other'), kept()); + // A module worker's static import of the app's script. + const m = markWorker('http://h/m.js', 'm', 'd2'.repeat(16)); + await start(m); + assert.deepEqual(await load('http://h/m.js', m, 'other'), on('http://h/m.js')); + assert.deepEqual(await load('http://h/m.js', m, 'script', 'script'), on('http://h/m.js')); + assert.deepEqual(await load('http://h/m.js', m, 'script'), kept()); + await start(m); + assert.deepEqual(await load('http://h/m.js', m, 'script', 'worker'), kept()); + // Under a policy that sends less to the worker's own origin: what it + // sends (Chromium), or the marked URL (Firefox and WebKit, for a classic + // worker); taken at one, it is gone from the other. + await start(w, 'no-referrer'); + assert.deepEqual(await load('http://h/w.js', null, 'other'), kept('no-referrer')); + assert.deepEqual(await load('http://h/w.js', w, 'script', 'script'), on('http://h/w.js')); + await start(w, 'no-referrer'); + assert.deepEqual(await load('http://h/w.js', w, 'script', 'script'), kept('no-referrer')); + assert.deepEqual(await load('http://h/w.js', null, 'other'), on('http://h/w.js')); + await start(m, 'origin'); + assert.deepEqual(await load('http://h/m.js', 'http://h/', 'script', 'script'), on('http://h/m.js')); + assert.deepEqual(await load('http://h/m.js', 'http://h/', 'script', 'worker'), kept('origin')); + // Two workers at one URL, each kept: each seeding's load gets one. + const w2 = markWorker('http://h/w.js', 'w', 'd3'.repeat(16)); + await start(w); + await start(w2); + assert.deepEqual(await load('http://h/w.js', w2, 'other'), kept()); + assert.deepEqual(await load('http://h/w.js', w, 'other'), kept()); + // Under no-referrer, with what each fetch got: one taken at its marked + // URL is gone from what no-referrer sends too, where the other's is. + const each = (m, body) => r.ask(m, answerOf(200, body, { 'content-type': 'text/javascript', 'referrer-policy': 'no-referrer' }), { type: 'script', headers: { referer: 'http://h/pg', 'sec-fetch-dest': 'worker' } }, page); + const got = async (...a) => (await load(...a))[0][1]?.body; + await each(w, 'one()'); + await each(w2, 'two()'); + assert.equal(await got('http://h/w.js', w, 'script', 'script'), 'one()'); + assert.equal(await got('http://h/w.js', null, 'other'), 'two()'); + assert.deepEqual(await load('http://h/w.js', null, 'other'), on('http://h/w.js')); + // A worker's script load keploy handed on is none, though it looks like + // a module worker's seeding's load: one at the URL of a worker kept under + // no-referrer, from a page under it too, is seeded. + await start(m, 'no-referrer'); + const m2 = markWorker('http://h/m.js', 'm', 'd4'.repeat(16)); + pass = true; + const second = await r.ask(m2, app('no-referrer'), { type: 'script', headers: {} }, page); + pass = false; + assert.equal(second[1][1].body, workerAnswer('d4'.repeat(16), 0, 'm', 'http://h/m.js')); + assert.deepEqual(await load('http://h/m.js', null, 'script'), kept('no-referrer')); + assert.deepEqual(await load('http://h/m.js', null, 'script'), kept('no-referrer')); + assert.deepEqual(await load('http://h/m.js', null, 'script'), on('http://h/m.js')); + // None for a Shared Worker's, or for an answer not ok. + const s = markWorker('http://h/s.js', 's', 'd5'.repeat(16)); + await start(s); + assert.equal(r.match(new URL('http://h/s.js')), false); + assert.deepEqual(await load('http://h/s.js', s, 'other'), on('http://h/s.js')); + const x = markWorker('http://h/x.js', 'w', 'd6'.repeat(16)); + assert.deepEqual(await start(x, undefined, 404), [['fetch', 'http://h/x.js'], ['fulfill', { status: 404, headers: { 'content-type': 'text/javascript' }, body: 'app()' }]]); + assert.equal(r.match(new URL('http://h/x.js')), false); + assert.deepEqual(await load('http://h/x.js', x, 'other'), on('http://h/x.js')); +}); + +test('ownReferer: the Referer a worker sends to its own origin under its script\'s Referrer-Policy, the last token that is a policy, in any case', () => { + const at = 'http://u:p@h:8/w.js?a=1&__keploy_worker=w.x#f'; + assert.equal(ownReferer(at), 'http://h:8/w.js?a=1&__keploy_worker=w.x'); + for (const p of ['', 'unsafe-url', 'same-origin', 'no-referrer-when-downgrade', 'origin-when-cross-origin', 'strict-origin-when-cross-origin', 'bogus', 'no-referrer, unsafe-url']) assert.equal(ownReferer(at, p), 'http://h:8/w.js?a=1&__keploy_worker=w.x', p); + for (const p of ['no-referrer', 'No-Referrer', 'origin, no-referrer', ' no-referrer , bogus', 'no-referrer,']) assert.equal(ownReferer(at, p), '', p); + for (const p of ['origin', 'strict-origin', 'no-referrer, STRICT-ORIGIN']) assert.equal(ownReferer(at, p), 'http://h:8/', p); }); test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module one whose policy refuses data: with its seeding in the answer; each request it fulfills is answered', async () => { diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 8fadc6a..8c4a46a 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -654,15 +654,17 @@ async function drawsServer() { // overrides.spec.ts's: /ov/aw.js and /ov/as.js answer 401 to a request // with no x-auth: k; /ov/ow.js and /ov/os.js are not there (404), and // their stand-ins /ov/mw.js and /ov/ms.js are; /ov/pw.js and /ov/ps.js - // ask nothing. Each Web Worker (w) and Shared Worker (s) sends its kind, + // ask nothing; once.spec.ts's /ov/nw.js and /ov/gw.js need x-auth: k too, + // under Referrer-Policy no-referrer and origin. Each Web Worker (w) and + // Shared Worker (s) sends its kind, // its location and what it drew. if (u.pathname === '/ov') { res.writeHead(200, { 'content-type': 'text/html' }); res.end(page('ov')); return; } if (u.pathname === '/ov/ow.js' || u.pathname === '/ov/os.js') { res.writeHead(404, { 'content-type': 'text/plain' }); res.end('none'); return; } - const ov = /^\/ov\/([amp])([ws])\.js$/.exec(u.pathname); + const ov = /^\/ov\/([ampng])([ws])\.js$/.exec(u.pathname); if (ov !== null) { - if (ov[1] === 'a' && req.headers['x-auth'] !== 'k') { res.writeHead(401, { 'content-type': 'text/plain' }); res.end('no auth'); return; } + if (ov[1] !== 'm' && ov[1] !== 'p' && req.headers['x-auth'] !== 'k') { res.writeHead(401, { 'content-type': 'text/plain' }); res.end('no auth'); return; } const sent = `{ kind: '${ov[1]}${ov[2]}', href: location.href, draws: ${DRAWS} }`; - res.writeHead(200, { 'content-type': 'text/javascript' }); + res.writeHead(200, { 'content-type': 'text/javascript', ...(ov[1] === 'n' ? { 'referrer-policy': 'no-referrer' } : ov[1] === 'g' ? { 'referrer-policy': 'origin' } : {}) }); res.end(ov[2] === 's' ? `onconnect = (e) => e.ports[0].postMessage(${sent});` : `postMessage(${sent});`); return; } @@ -1648,7 +1650,7 @@ for (const engine of ENGINES) { // overrides.spec.ts's tests, and the scripts each one's page starts its // workers at (/ov/w.js and /ov/s.js). const OV = [['continue headers on page', 'a'], ['continue headers on context', 'a'], ['continue url on page', 'o'], ['continue url on context', 'o'], - ['fallback headers on page', 'a'], ['fallback headers on context', 'a'], ['fallback url on page', 'o'], ['browser headers on page', 'p']]; + ['fallback headers on page', 'a'], ['fallback headers on context', 'a'], ['fallback url on page', 'o'], ['browser headers on page', 'p'], ['own url and method on page', 'p']]; // The marks a document at /ov of page t0 gives the workers overrides.spec.ts // starts there. @@ -1661,7 +1663,7 @@ function ovMarks(seed, k) { } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a worker whose script the test's own route, on its page or its context, continues or falls back with a header the app's server needs or with a stand-in's URL runs what that route asked for; a Web Worker's is seeded (its seeding loads the app's script through the test's routes again) at the app's URL; a Shared Worker's runs unseeded and is named in Chromium and WebKit (nothing a Shared Worker loads after its script is routed there), and is seeded when the route continues with the browser's own headers; Firefox's route reaches no Shared Worker's script, the test's neither, so there it fails as with no keploy`, async () => { + test(`seeded, in ${engine}: a worker whose script the test's own route, on its page or its context, continues or falls back with a header the app's server needs or with a stand-in's URL runs what that route asked for; a Web Worker's is seeded (keploy answers its seeding's load of the app's script with what it fetched so) at the app's URL; a Shared Worker's runs unseeded and is named in Chromium and WebKit (nothing a Shared Worker loads after its script is routed there), and is seeded when the route continues with the browser's own headers, or with its own URL and method; Firefox's route reaches no Shared Worker's script, the test's neither, so there it fails as with no keploy`, async () => { const j = journal(); const rec = await fakeAgent(j); try { @@ -1699,6 +1701,77 @@ for (const engine of ENGINES) { }); } +// once.spec.ts's tests, and the workers each one's page starts, in order +// (each script's URL, and w a classic Web Worker, m a module one, s a +// Shared Worker). +const ONCE = [['once on page', [['/ov/aw.js', 'w']]], ['once on context, module', [['/ov/aw.js', 'm']]], ['by destination', [['/ov/aw.js', 'w']]], + ['once under no-referrer', [['/ov/nw.js', 'w']]], ['once under no-referrer, module', [['/ov/nw.js', 'm']]], ['once under origin, module', [['/ov/gw.js', 'm']]], + ['counted', [['/ov/pw.js', 'w'], ['/ov/pw.js', 'm']]], ['answered then continued at one address', [['/ov/as.js', 'w'], ['/ov/as.js', 's']]], + ['two routes', [['/ov/aw.js', 'w'], ['/ov/as.js', 's']]]]; + +// The marks a document at /ov of page t0 gives the workers list starts. +function onceMarks(seed, list) { + const made = []; + class Worker { constructor(url) { made.push(workerMark(String(url))); } } + const code = list.map(([u, k]) => (k === 's' ? `new SharedWorker('${u}');` : `new Worker('${u}'${k === 'm' ? ', { type: \'module\' }' : ''});`)).join(' '); + documentAt(initScript(seed), '/ov', new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }).run(code); + return made; +} + +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a test's own route that treats a Web Worker's script loads apart (one that answers once, on its page or its context, one that picks the worker's script by its destination, one that counts its loads) sees what it sees with seeding off and the worker runs, seeded, at the app's URL: keploy answers the seeding's load of the app's script with what it fetched for the worker's own, under a Referrer-Policy that sends less too, so the app's server is asked for a Web Worker's script once; a Web Worker at one address whose script the test's route answers itself and a Shared Worker there whose script it sends on with a header are each named; of two routes, the last one's options count`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + try { + const names = [...ONCE.map(([t]) => t), 'counted-n']; + const r = await drawRun(j, { ...keployEnv(rec.url), PW_BROWSER: engine }, ['once.spec.ts'], names); + const off = await drawRun(j, { ...keployEnv(rec.url), PW_BROWSER: engine, KEPLOY_SEED_RANDOM: 'off' }, ['once.spec.ts'], names); + for (const [title, list] of ONCE) { + const marks = onceMarks(r.seeds[title], list); + const at = (u) => `http://127.0.0.1:${u}`; + const lines = []; + list.forEach(([u, k], i) => { + const [got, was] = [r.drew[title][i], off.drew[title][i]]; + const what = `${title} ${u} ${k}: ${JSON.stringify(got)} (off: ${JSON.stringify(was)})`; + if (k === 's') { + lines.push(sharedWorkerLine(title, at(u))); + // Firefox's route reaches no Shared Worker's script: it fails + // as with no keploy, and is named. + if (engine === 'firefox') { + assert.ok(['error', 'none'].includes(got), what); + lines.push(unplacedLine(title, at(u), 'shared')); + return; + } + assert.equal(got.kind, 'as', what); + assert.match(got.href, new RegExp(`^http://127\\.0\\.0\\.1:\\d+/ov/as\\.js\\?__keploy_worker=s\\.[0-9a-f]{32}$`), what); + lines.push(unplacedLine(title, at(u), 'continued')); + return; + } + if (title === 'answered then continued at one address') { + assert.deepEqual([got, was], [{ kind: 'own' }, { kind: 'own' }], what); + lines.push(unplacedLine(title, at(u), 'routed')); + return; + } + const kind = u.slice('/ov/'.length, -'.js'.length); + assert.equal(got.kind, kind, what); + assert.equal(was.kind, kind, what); + assert.match(got.href, new RegExp(`^http://127\\.0\\.0\\.1:\\d+${u.replace('.', '\\.')}$`), what); + assert.equal(marks[i].kind, k, what); + assert.deepEqual(got.draws, workerAt(`http://h${u}`, workerScript(marks[i].token, 0, k)).draws(), what); + }); + const said = r.tests[title].stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, `)).map((l) => l.replace(/:\d+\//g, ':/')); + assert.deepEqual(said.sort(), lines.sort(), title); + } + // The counting route saw each worker's script once, as with seeding + // off; the app's server was asked for the same scripts. + assert.deepEqual([r.drew['counted-n'], off.drew['counted-n']], [2, 2]); + const scripts = (seen) => seen.filter((u) => u.startsWith('/ov/')).sort(); + assert.deepEqual(scripts(r.seen), scripts(off.seen)); + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); + } finally { await rec.close(); } + }); +} + // The marks a document at /nw of page t0 gives the workers nested.spec.ts's // page starts, and the marks those Web Workers' seedings give theirs. function nestedMarks(seed) { From be324c1fbac64ce5451cdd8660a496296c53f3dd Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sun, 15 Nov 2026 06:27:44 +0530 Subject: [PATCH 46/73] fix(playwright): load a Web Worker's script once, its seeding first keploy's answer to a Web Worker's marked script load is now the app's script it fetched through the test's routes, with the worker's seeding loaded first from a URL keploy marks and answers itself (importScripts for a classic worker, after the app's own 'use strict' when it has one; an import ahead of the app's imports for a module worker), on the script's first line, then a sourceURL of the app's URL. The worker never loads the app's script a second time, so a route that answers once, counts loads or picks the worker's script by its destination, and the app's server, see one load under any referrer policy, as with seeding off. The kept scripts and the Referer matching are gone. Stacks through the app's script name its URL with no mark in each engine; the worker's location and an ErrorEvent's filename hide the mark. A module Web Worker's import.meta.url shows it. Signed-off-by: slayerjain --- playwright/src/random.ts | 278 ++++++++++++++++++++------------ playwright/test/random.test.mjs | 261 ++++++++++++++++-------------- playwright/test/rows.test.mjs | 12 +- 3 files changed, 321 insertions(+), 230 deletions(-) diff --git a/playwright/src/random.ts b/playwright/src/random.ts index d182d67..bd9e7c2 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1532,6 +1532,20 @@ ${DRAW_JS} const n = typeof o === 'string' ? o : o !== null && typeof o === 'object' && o.name !== undefined ? String(o.name) : ''; return hex(seed + '\\nshared worker\\n' + u.pathname + '\\n' + n); }); + // An error a Web Worker's script lets out reaches its Worker here naming + // the script it was thrown in (an ErrorEvent's filename): Firefox and + // WebKit name the worker's marked URL (workerAnswer), shown here with no + // mark, as Chromium names it. + try { + const E = g.ErrorEvent; + const d = typeof E === 'function' ? Object.getOwnPropertyDescriptor(E.prototype, 'filename') : undefined; + if (d && typeof d.get === 'function') { + const get = d.get; + const mark = new RegExp(${JSON.stringify(`[?&]${WORKER_PARAM}=[wm]\\.[0-9a-f]{32}(?![.0-9a-f])`)}, 'g'); + const v = { get filename() { const f = get.call(this); return typeof f === 'string' ? f.replace(mark, '') : f; } }; + Object.defineProperty(E.prototype, 'filename', { get: Object.getOwnPropertyDescriptor(v, 'filename').get, set: d.set, enumerable: d.enumerable, configurable: true }); + } + } catch (e) {} })(); `; } @@ -1540,21 +1554,22 @@ ${DRAW_JS} * A worker's script URL, marked (initScript), read back: its kind ('w' a * classic Web Worker's script, 'm' a module Web Worker's, 's' a classic * Shared Worker's, 'x' a module Shared Worker's, 'p' the seeding that the - * answer to 'm' imports, 't' a seeded Web Worker telling keploy the address + * answer to 'm' imports, 'q' the seeding that the answer to 'w' loads + * (importScripts), 't' a seeded Web Worker telling keploy the address * of a worker it started that keploy cannot mark: NESTED_JS), its token, - * the instance n of a token ('p'; -1 for the others), and + * the instance n of a token ('p' and 'q'; -1 for the others), and * the URL with no mark, as the app asked for it. null for a URL with no * mark keploy made. */ -export type WorkerMark = { kind: 'w' | 'm' | 's' | 'x' | 'p' | 't'; token: string; n: number; url: string }; +export type WorkerMark = { kind: 'w' | 'm' | 's' | 'x' | 'p' | 'q' | 't'; token: string; n: number; url: string }; export function workerMark(href: string): WorkerMark | null { const i = href.indexOf('#'); const b = i < 0 ? href : href.slice(0, i); - const m = /[?&]__keploy_worker=([wmsxpt])\.([0-9a-f]{32})(?:\.(\d{1,9}))?$/.exec(b); + const m = /[?&]__keploy_worker=([wmsxpqt])\.([0-9a-f]{32})(?:\.(\d{1,9}))?$/.exec(b); if (m === null) return null; const kind = m[1] as WorkerMark['kind']; - const numbered = kind === 'p'; + const numbered = kind === 'p' || kind === 'q'; if (numbered !== (m[3] !== undefined)) return null; return { kind, token: m[2], n: numbered ? Number(m[3]) : -1, url: b.slice(0, m.index) + (i < 0 ? '' : href.slice(i)) }; } @@ -1594,9 +1609,14 @@ ${STREAM_JS} // browser's own call throws in a seeded function or getter keeps, in V8 // and WebKit, a stack of its own, made as it was thrown, with the seeding's // frames: they are left out of it before it reaches the caller, and a - // stack with none, or that is no string, stays as it was. + // stack with none, or that is no string, stays as it was. A Web Worker's + // app's script runs at the worker's own marked URL, after its seeding + // (workerAnswer): a frame there is the app's, kept, with the mark taken + // off (each engine names the app's URL there already: the answer's + // sourceURL). const marked = new RegExp(${JSON.stringify(`[?&;]${WORKER_PARAM}=`)}); - const unmark = (s) => s.split('\\n').filter((l) => !marked.test(l)).join('\\n'); + const main = ${kind === 'w' || kind === 'm' ? `new RegExp(${JSON.stringify(`[?&]${WORKER_PARAM}=${kind}\\.${token}(?![.0-9a-f])`)}, 'g')` : 'null'}; + const unmark = (s) => s.split('\\n').map((l) => (main === null ? l : l.replace(main, ''))).filter((l) => !marked.test(l)).join('\\n'); const thrown = (e) => { try { const d = Object.getOwnPropertyDescriptor(e, 'stack'); @@ -1678,6 +1698,20 @@ ${DRAW_JS} Object.defineProperty(C.prototype, 'stack', { get: Object.getOwnPropertyDescriptor(v, 'stack').get, set: d.set, enumerable: d.enumerable, configurable: true }); } catch (e) {} } + // An error the app's script lets out names the script it was thrown in + // (an ErrorEvent's filename): Firefox and WebKit name a Web Worker's + // marked URL, shown here with no mark, as Chromium names it. + if (main !== null) { + try { + const E = g.ErrorEvent; + const d = typeof E === 'function' ? Object.getOwnPropertyDescriptor(E.prototype, 'filename') : undefined; + if (d && typeof d.get === 'function') { + const get = d.get; + const v = { get filename() { let f; try { f = get.call(this); } catch (e) { throw thrown(e); } return typeof f === 'string' ? f.replace(main, '') : f; } }; + Object.defineProperty(E.prototype, 'filename', { get: Object.getOwnPropertyDescriptor(v, 'filename').get, set: d.set, enumerable: d.enumerable, configurable: true }); + } + } catch (e) {} + } ${kind === 'w' || kind === 'm' ? NESTED_JS(token, reach) : ''}})(); `; } @@ -1768,10 +1802,94 @@ export function cspAllowsData(headers: Record): boolean { return true; } +/** + * Where keploy's first statement goes in a script's source: ahead of it, or + * after its hashbang line (#!, which only a source's very start may hold), + * so every line of the app's keeps its number. + */ +export function withPrefix(source: string, prefix: string): string { + if (!source.startsWith('#!')) return prefix + source; + const m = /\r\n|[\n\r\u2028\u2029]/.exec(source); + if (m === null) return `${source}\n${prefix}`; + const i = m.index + m[0].length; + return source.slice(0, i) + prefix + source.slice(i); +} + +/** + * Whether a classic script's directive prologue (the string statements it + * starts with, after a hashbang line, white space and comments, each ended + * by a semicolon, a line's end or the source's) holds 'use strict' written + * with no escape, which makes the whole script strict. + */ +export function useStrict(source: string): boolean { + const n = source.length; + const eol = /[\n\r\u2028\u2029]/; + // From at, past white space and comments; nl: whether a line ended there. + const skip = (at: number): { at: number; nl: boolean } => { + let nl = false; + for (;;) { + const c = source[at]; + if (c === undefined) return { at, nl }; + if (eol.test(c)) { nl = true; at++; } else if (/\s/.test(c)) at++; + else if (source.startsWith('//', at)) { + const e = source.slice(at).search(eol); + if (e < 0) return { at: n, nl }; + at += e; + } else if (source.startsWith('/*', at)) { + const e = source.indexOf('*/', at + 2); + if (e < 0) return { at: n, nl }; + if (eol.test(source.slice(at, e))) nl = true; + at = e + 2; + } else return { at, nl }; + } + }; + let i = 0; + if (source.startsWith('#!')) { + const m = eol.exec(source); + if (m === null) return false; + i = m.index; + } + for (;;) { + const s = skip(i); + const q = source[s.at]; + if (q !== '"' && q !== '\'') return false; + let j = s.at + 1; + while (j < n && source[j] !== q) { + if (source[j] === '\\') j++; + else if (eol.test(source[j])) return false; + j++; + } + if (j >= n) return false; + const raw = source.slice(s.at + 1, j); + const after = skip(j + 1); + if (source[after.at] === ';') i = after.at + 1; + else if (after.nl || after.at >= n) i = after.at; + else return false; + if (raw === 'use strict') return true; + } +} + /** * keploy's answer to a marked worker script ('w', 'm', 's' or 'x', for the - * instance n of token): the worker's seeding, then the app's own script, - * loaded from own, the URL the app asked for, with no mark. The app's + * instance n of token). A Web Worker's ('w', 'm') is the app's own script, + * source, as keploy fetched it, whose first statement loads the worker's + * seeding from a URL keploy marked ('q' for a classic worker, by + * importScripts, after a 'use strict' of the app's script's own when it has + * one, so it stays strict; 'p' for a module one, by an import ahead of every + * import of the app's, so it runs before any of them), on the app's first + * line, so each line keeps its number; then, unless the app's script names + * itself, a sourceURL of the URL the app asked for. The app's script is so + * loaded once, and runs at the worker's marked URL, after its seeding: a + * stack trace through it names the app's URL with no mark (each engine + * names a script by its sourceURL), and the worker's location and an + * ErrorEvent's filename hide the mark (workerScript); a module Web Worker's + * import.meta.url shows it, and the URLs its imports resolve against are + * the app's but for a fragment alone. When keploy could not fetch the + * app's script (source undefined), the worker gets its seeding, then loads + * the app's script itself, from own, as below. + * + * A Shared Worker's ('s', 'x') is the worker's seeding, then the app's own + * script, loaded from own, the URL the app asked for, with no mark. The app's * script so runs at its own URL: its import.meta.url, the URLs its imports * resolve against and a stack trace through it show no mark, and the * worker's location hides the one its script URL carries (workerScript). @@ -1781,9 +1899,9 @@ export function cspAllowsData(headers: Record): boolean { * before the worker takes any message or connection, and reports what the * app's script throws as the worker's own error, as an uncaught one at its * top level is (a Firefox stack, which goes on through the code that queued - * it, leaves the answer's frames out: workerScript). A module Web Worker - * imports its seeding (the route answers - * that import, marked 'p') and then the app's script. Chromium and WebKit + * it, leaves the answer's frames out: workerScript); so does a Web Worker + * whose script keploy could not fetch, and such a module Web Worker + * imports its seeding (marked 'p') and then the app's script. Chromium and WebKit * let the route reach none of what a Shared Worker loads after its script, * so a module Shared Worker imports its seeding as a data: module (its * media type carries keploy's mark as a parameter, so a stack leaves the @@ -1795,9 +1913,17 @@ export function cspAllowsData(headers: Record): boolean { * order, each in a microtask of its own, to the handlers the app's script * sets. */ -export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true, reach = false): string { +export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true, source?: string): string { const url = JSON.stringify(own); - if (kind === 'w' || kind === 's') return `${workerScript(token, n, kind, reach)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; + if ((kind === 'w' || kind === 'm') && source !== undefined) { + const first = kind === 'w' + ? `${useStrict(source) ? '"use strict";' : ''}importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});` + : `import ${JSON.stringify(markWorker(own, 'p', token, n))};`; + const named = /(?:\/\/|\/\*)[#@][ \t]*sourceURL[ \t]*=/.test(source); + return withPrefix(source, first) + (named ? '' : `\n//# sourceURL=${noFragment(own)}\n`); + } + if (kind === 'w') return `importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; + if (kind === 's') return `${workerScript(token, n, kind)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; if (kind === 'm') return `import ${JSON.stringify(markWorker(own, 'p', token, n))};\nimport ${url};\n`; const seeding = workerScript(token, n, 'x'); if (dataOk) return `import ${JSON.stringify(`data:text/javascript;${WORKER_PARAM}=x.${token};base64,${Buffer.from(seeding).toString('base64')}`)};\nimport ${url};\n`; @@ -1917,49 +2043,31 @@ function noFragment(url: string): string { return i < 0 ? url : url.slice(0, i); } -const POLICIES = new Set(['no-referrer', 'no-referrer-when-downgrade', 'same-origin', 'origin', 'strict-origin', 'origin-when-cross-origin', 'strict-origin-when-cross-origin', 'unsafe-url']); - -/** - * The Referer a worker whose script is at at sends with a request to its - * own origin, under policy, the Referrer-Policy its script was answered - * with (the last token it names that is a policy, in any case): none ('') - * under no-referrer, its origin under origin and strict-origin, else its - * URL with no fragment or user info. - */ -export function ownReferer(at: string, policy = ''): string { - let p = ''; - for (const t of policy.split(',')) { - const v = t.trim().toLowerCase(); - if (POLICIES.has(v)) p = v; - } - const u = new URL(noFragment(at)); - if (p === 'no-referrer') return ''; - if (p === 'origin' || p === 'strict-origin') return `${u.origin}/`; - u.username = ''; - u.password = ''; - return u.href; -} - /** - * Whether r is a load of the kind a Web Worker's seeding makes of the app's - * script (workerAnswer): importScripts for a classic one (sec-fetch-dest - * script where the route sees it; Chromium types it other), a static import - * for a module one (sec-fetch-dest worker; Chromium types it script). + * Whether the browser asks for r, at a URL keploy marked 'p' or 'q', as the + * seeding a Web Worker's answer loads (workerAnswer): a module worker's + * import of it ('p': as a worker's script, workerScriptLoad), a classic + * worker's importScripts of it ('q': sec-fetch-dest script where the route + * sees it; Chromium types it other), not as a request the worker's code + * makes there. */ -function seedingLoad(r: Request, kind: 'w' | 'm'): boolean { +function preludeLoad(r: Request, kind: 'p' | 'q'): boolean { + if (kind === 'p') return workerScriptLoad(r); const dest = r.headers()['sec-fetch-dest']; - if (dest !== undefined) return dest === (kind === 'w' ? 'script' : 'worker'); - return r.resourceType() === (kind === 'w' ? 'other' : 'script'); + if (dest !== undefined) return dest === 'script'; + return r.resourceType() === 'other'; } /** * Answers each marked worker script the test's context asks for with * workerAnswer, under the status and headers the app's server answered for * the script with no mark (a worker's policy comes from its script's - * answer), and a module Web Worker's seeding (marked 'p') with - * workerScript. A Web Worker's seeding's load of the app's script is - * answered with what keploy's fetch for that answer got (kept), so the app's - * server is asked for a Web Worker's script once, here; Chromium and WebKit + * answer), and a Web Worker's seeding (marked 'p' for a module one, 'q' for + * a classic one) with workerScript. A Web Worker's answer is the app's + * script keploy fetched for it, with the seeding's load ahead of it: the + * worker never loads the app's script again, so the test's routes and the + * app's server see a Web Worker's script loaded once, as with seeding off, + * under any referrer policy. Chromium and WebKit * route nothing a Shared Worker loads after its script, so it is asked for a * Shared Worker's twice: here, and by the worker, for the app's script. An answer * that is not ok goes to the worker as the server gave it; when keploy @@ -1989,10 +2097,9 @@ function seedingLoad(r: Request, kind: 'w' | 'm'): boolean { * context's first, which answers it as above, at the URL a fallback gave it * too. What the test's continue and fallbacks ask (a URL, method, headers or * post data of their own) is what keploy's fetch of the script sends, and a - * Web Worker's seeding's load of the app's script gets what that fetch got, - * ahead of the test's routes, which so see the script once: a route that - * answers once (times), counts its loads or picks the worker's script by its - * destination sees what it sees with seeding off. A Shared Worker's seeding + * Web Worker runs what that fetch got: a route that answers once (times), + * counts its loads or picks the worker's script by its destination sees what + * it sees with seeding off. A Shared Worker's seeding * loads the app's script past every route (Chromium and WebKit route none * of its loads): one whose script the * test's routes asked for otherwise than the browser sent it gets the answer @@ -2028,44 +2135,6 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = if (k === 'method') return v !== h.method; return true; }); - // The app's script a seeded Web Worker's seeding loads next (workerAnswer), - // kept from keploy's fetch for the worker's own load until the browser - // asks for it, and answered so ahead of the test's routes: the test's - // routes and the app's server see a Web Worker's script loaded once, as - // with seeding off. A seeding's load is told by its URL, the app's, and - // its Referer: the worker's marked URL, or, under a Referrer-Policy that - // sends less to the worker's own origin, what that policy sends (Chromium - // sends that for both kinds, Firefox and WebKit for a module worker), and - // by its kind (seedingLoad). Each is kept at each such Referer, with how - // many are kept at each URL. Chromium and WebKit route nothing a Shared - // Worker loads after its script, so none is kept for one. - type Kept = { kind: 'w' | 'm'; keys: string[]; answer: { status: number; headers: Record; body: Buffer } }; - const kept = new Map(); - const keptAt = new Map(); - const keyOf = (referer: string, url: string): string => `${referer}\n${noFragment(url)}`; - const keep = (h: Handed, kind: 'w' | 'm', answer: Kept['answer']): void => { - const policy = answer.headers['referrer-policy']; - const keys = [...new Set([keyOf(ownReferer(h.at), h.m.url), keyOf(ownReferer(h.at, policy), h.m.url)])]; - const k: Kept = { kind, keys, answer }; - for (const key of keys) kept.set(key, [...(kept.get(key) ?? []), k]); - const u = noFragment(h.m.url); - keptAt.set(u, (keptAt.get(u) ?? 0) + 1); - }; - // The kept script r is the seeding's load of, taken, or null. - const take = (r: Request): Kept['answer'] | null => { - const k = kept.get(keyOf(r.headers().referer ?? '', r.url()))?.find((x) => seedingLoad(r, x.kind)); - if (k === undefined) return null; - for (const key of k.keys) { - const rest = (kept.get(key) ?? []).filter((x) => x !== k); - if (rest.length > 0) kept.set(key, rest); - else kept.delete(key); - } - const u = noFragment(r.url()); - const n = (keptAt.get(u) ?? 1) - 1; - if (n > 0) keptAt.set(u, n); - else keptAt.delete(u); - return k.answer; - }; // A route's fetch with o is the request the test's routes asked for: o's // URL, method, headers and post data, or the request's (the URL the app // asked for, or one a fallback gave, and what fallbacks gave it). @@ -2074,8 +2143,8 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await own(o); hooks.answered?.(route.request()); }; - if (m.kind === 'p') { - await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, 'm', reach) }); + if (m.kind === 'p' || m.kind === 'q') { + await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, m.kind === 'p' ? 'm' : 'w', reach) }); return; } const n = counts.get(m.token) ?? 0; @@ -2087,6 +2156,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = const whole = (m.kind === 's' || m.kind === 'x') && h !== undefined && otherwise(h); let status = 200; let headers: Record = { 'content-type': 'text/javascript' }; + let source: string | undefined; try { const got = await route.fetch(o); if (whole || !got.ok()) { @@ -2099,25 +2169,19 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } status = got.status(); headers = headersOf(got.headers()); - if (h !== undefined && (m.kind === 'w' || m.kind === 'm')) keep(h, m.kind, { status, headers, body: await got.body() }); + // A worker's script is read as UTF-8 whatever its charset says, as + // the browser reads it; keploy's answer is UTF-8 too. + if (m.kind === 'w' || m.kind === 'm') source = new TextDecoder().decode(await got.body()); } catch { // The worker loads the app's script itself, and fails as it would. } - await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers), reach) }); + await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers), source) }); }; const first = async (route: Route): Promise => { const r = route.request(); const m = workerMark(r.url()); if (m === null) { - // A seeded Web Worker's seeding's load of the app's script: answered - // with what keploy fetched for the worker's own load. A worker's - // script load keploy handed on is none. - const k = handed.has(r) ? null : take(r); - if (k === null) await route.fallback(); - else { - await route.fulfill(k); - hooks.answered?.(r); - } + await route.fallback(); return; } // A seeded Web Worker's word of a worker keploy cannot mark: heard, then @@ -2130,12 +2194,12 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = hooks.answered?.(r); return; } - if (!workerScriptLoad(r)) { + const own = route.fulfill.bind(route); + if (m.kind === 'p' || m.kind === 'q' ? !preludeLoad(r, m.kind) : !workerScriptLoad(r)) { await route.fallback({ url: m.url }); return; } - const own = route.fulfill.bind(route); - if (m.kind === 'p') { + if (m.kind === 'p' || m.kind === 'q') { await answer(route, m, own); return; } @@ -2173,7 +2237,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } await answer(route, h.m, h.fulfill, h); }; - const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM) || keptAt.has(noFragment(u.href)); + const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM); const lastUrl = (u: URL): boolean => urls.has(u.href); const route = context.route; const addLast = async (): Promise => { diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index f4009a6..5f9dc95 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, ownReferer, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -1920,7 +1920,6 @@ test('a Web Worker\'s seeding starts each Web Worker it starts at its script URL assert.deepEqual(a.posted, [[to, { method: 'POST', body: '["nested","http://h/d/b.js"]' }], [to, { method: 'POST', body: '["nested","https://h/s.js"]' }], [to, { method: 'POST', body: '["worker","blob:"]' }]], kind); } - assert.deepEqual(workerAnswer(t, 0, 'w', 'http://h/w.js', true, true).split('queueMicrotask(')[0], workerScript(t, 0, 'w', true)); assert.notEqual(workerScript(t, 0, 'w', true), workerScript(t, 0, 'w')); }); @@ -1944,9 +1943,11 @@ test('a Web Worker\'s seeding starts a Web Worker keploy cannot mark as the app test('a worker\'s seeding: an Error\'s or a DOMException\'s stack read through its prototype\'s getter (Firefox) leaves out each frame at a URL whose query has keploy\'s mark, any kind; its setter, flags and a value that is no string are kept; a stack that is no getter stays as it was', () => { const t = 'ab'.repeat(16); const href = markWorker('http://h/w.js', 'w', t); - const kept = ['@http://h/w.js:1:2', `h@http://h/w.js?a__keploy_worker=w.${t}:7:8`]; - const frames = [kept[0], `VoidFunction*@${href}:114:15`, `g@http://h/w.js?b=1&__keploy_worker=o.${t}:5:6`, `f@${markWorker('http://h/m.js', 'p', t, 0)}:3:4`, kept[1], - `promise callback*@${markWorker('http://h/s.js?q=1', 's', 'ef'.repeat(16))}:9:1`, '']; + const kept = ['@http://h/w.js:1:2', `h@http://h/w.js?a__keploy_worker=w.${t}:7:8`, 'app@http://h/w.js:4:5', `k@http://h/w.js?__keploy_worker=w.${'ef'.repeat(16)}:2:2`]; + // The app's script runs at the worker's own marked URL (href): its frame + // there is kept, with no mark; one at another worker's mark is not this one's. + const frames = [kept[0], `VoidFunction*@${markWorker('http://h/w.js', 'q', t, 0)}:114:15`, `g@http://h/w.js?b=1&__keploy_worker=o.${t}:5:6`, `f@${markWorker('http://h/m.js', 'p', t, 0)}:3:4`, kept[1], + `promise callback*@${markWorker('http://h/s.js?q=1', 's', 'ef'.repeat(16))}:9:1`, `app@${href}:4:5`, `${kept[3].replace('w.', 'm.')}`, '']; class DOMException {} const w = workerAt(href, '', { DOMException }); w.run('for (const C of [Error, DOMException]) Object.defineProperty(C.prototype, \'stack\', { get() { return this.s; }, set(v) { this.set = v; }, enumerable: false, configurable: true });'); @@ -1954,7 +1955,7 @@ test('a worker\'s seeding: an Error\'s or a DOMException\'s stack read through i // V8 gives each error its own stack; an object made from the prototype // reads it through the prototype's getter, as Firefox's errors do. for (const C of ['Error', 'DOMException']) { - assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.s = ${JSON.stringify(frames.join('\n'))}; return e.stack; })()`), [...kept, ''].join('\n'), C); + assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.s = ${JSON.stringify(frames.join('\n'))}; return e.stack; })()`), [kept[0], kept[1], kept[2], ''].join('\n'), C); assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.stack = 'x'; return e.set; })()`), 'x', C); assert.equal(w.run(`(() => { const e = Object.create(${C}.prototype); e.s = 7; return e.stack; })()`), 7, C); const d = w.run(`Object.getOwnPropertyDescriptor(${C}.prototype, 'stack')`); @@ -1971,9 +1972,9 @@ test('a worker\'s seeding: an Error\'s or a DOMException\'s stack read through i test('a worker\'s seeding: an error the browser\'s own call throws in a seeded getRandomValues or randomUUID, or in the location\'s href, search or toString, reaches the caller with each frame at a URL keploy marked (a query\'s mark or a data: module\'s parameter) left out of its own stack, through its setter or as its value, flags kept; a stack with no such frame, one that is no string, none of its own and a thrown value that is no object stay as they were; a page\'s errors go on as they are', () => { const t = 'cd'.repeat(16); const href = markWorker('http://h/w.js', 'w', t); - const kept = ['TypeError: no', ' at f (http://h/w.js:1:2)', ` at g (http://h/w.js?a__keploy_worker=w.${t}:7:8)`, '']; - const stack = [kept[0], ` at Crypto.getRandomValues (${href}:30:5)`, kept[1], ` at x (data:text/javascript;__keploy_worker=x.${t};base64,QUJD:1:9)`, - ` at y (${markWorker('http://h/m.js', 'p', t, 0)}:3:4)`, kept[2], kept[3]].join('\n'); + const kept = ['TypeError: no', ' at f (http://h/w.js:1:2)', ` at g (http://h/w.js?a__keploy_worker=w.${t}:7:8)`, ' at app (http://h/w.js:8:9)', '']; + const stack = [kept[0], ` at Crypto.getRandomValues (${markWorker('http://h/w.js', 'q', t, 0)}:30:5)`, kept[1], ` at x (data:text/javascript;__keploy_worker=x.${t};base64,QUJD:1:9)`, + ` at y (${markWorker('http://h/m.js', 'p', t, 0)}:3:4)`, kept[2], ` at app (${href}:8:9)`, kept[4]].join('\n'); // What the browser's own calls throw, in turn. const throws = []; const raise = () => { throw throws.shift(); }; @@ -2109,10 +2110,14 @@ test('workerRoutes answers a marked classic worker script with its seeding then, assert.equal(r.match(new URL('http://h/w.js?a=1')), false); const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; assert.deepEqual(await r.ask(markWorker('http://h/w.js?a=1#f', 'w', t), answerOf(201, 'app()')), [['fetch', 'http://h/w.js?a=1#f'], - ['fulfill', { status: 201, headers, body: `${workerScript(t, 0, 'w')}queueMicrotask(importScripts.bind(globalThis, "http://h/w.js?a=1#f"));\n` }]]); + ['fulfill', { status: 201, headers, body: `importScripts(${JSON.stringify(markWorker('http://h/w.js?a=1#f', 'q', t, 0))});app()\n//# sourceURL=http://h/w.js?a=1\n` }]]); const second = await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')); - assert.equal(second[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1')); - assert.equal(workerAnswer(t, 1, 'w', 'http://h/w.js?a=1'), `${workerScript(t, 1, 'w')}queueMicrotask(importScripts.bind(globalThis, "http://h/w.js?a=1"));\n`); + assert.equal(second[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1', true, 'app()')); + // A worker's script is read as UTF-8, a BOM taken off, whatever its charset says. + const latin = await r.ask(markWorker('http://h/w.js', 'w', t), { ...answerOf(200, ''), body: async () => Buffer.from([0xef, 0xbb, 0xbf, 0x61, 0xc3, 0xa9]) }); + assert.equal(latin[1][1].body, workerAnswer(t, 2, 'w', 'http://h/w.js', true, 'a\u00e9')); + // With no source (keploy could not fetch it), its seeding, then the app's script loaded in a microtask. + assert.equal(workerAnswer(t, 1, 'w', 'http://h/w.js?a=1'), `importScripts(${JSON.stringify(markWorker('http://h/w.js?a=1', 'q', t, 1))});queueMicrotask(importScripts.bind(globalThis, "http://h/w.js?a=1"));\n`); }); test('workerRoutes answers a module worker script with an import of its seeding then one of the app\'s own; its seeding is workerScript; an answer not ok goes as given; one it cannot fetch gets its seeding with no header of the app\'s; a malformed mark falls back', async () => { @@ -2121,7 +2126,8 @@ test('workerRoutes answers a module worker script with an import of its seeding await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); const t = '23'.repeat(16); const did = await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'export {}')); - assert.equal(did[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};\nimport "http://h/m.js";\n`); + assert.equal(did[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};export {}\n//# sourceURL=http://h/m.js\n`); + assert.equal(workerAnswer(t, 0, 'm', 'http://h/m.js'), `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};\nimport "http://h/m.js";\n`); assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0)), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 0, 'm') }]]); assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'w', t), answerOf(404, 'no')), [['fetch', 'http://h/m.js'], ['fulfill', { status: 404, headers: { 'content-type': 'text/javascript', 'x-a': '1' }, body: 'no' }]]); @@ -2153,13 +2159,23 @@ test('workerRoutes answers a marked URL as a worker\'s script only when the brow // The worker's script, as each engine loads it: its first start is n 0. const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), as('xhr', 'worker')), [['fetch', 'http://h/w.js?a=1'], - ['fulfill', { status: 200, headers, body: workerAnswer(t, 0, 'w', 'http://h/w.js?a=1') }]]); - assert.equal((await r.ask(w, answerOf(200, 'app()'), as('script', 'worker')))[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1')); - assert.equal((await r.ask(w, answerOf(200, 'app()'), as('script')))[1][1].body, workerAnswer(t, 2, 'w', 'http://h/w.js?a=1')); + ['fulfill', { status: 200, headers, body: workerAnswer(t, 0, 'w', 'http://h/w.js?a=1', true, 'app()') }]]); + assert.equal((await r.ask(w, answerOf(200, 'app()'), as('script', 'worker')))[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1', true, 'app()')); + assert.equal((await r.ask(w, answerOf(200, 'app()'), as('script')))[1][1].body, workerAnswer(t, 2, 'w', 'http://h/w.js?a=1', true, 'app()')); + // A classic worker's seeding, at its q mark, as each engine loads it by + // importScripts (Chromium: other; Firefox: script; WebKit: xhr, dest + // script); what its code asks for there goes on with no mark. + const q = markWorker('http://h/w.js?a=1', 'q', t, 2); + for (const a of [as('other'), as('script', 'script'), as('xhr', 'script')]) { + assert.deepEqual(await r.ask(q, undefined, a), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 2, 'w') }]], JSON.stringify(a)); + } + for (const a of [as('fetch'), as('xhr'), as('script'), as('fetch', 'empty'), as('script', 'worker')]) { + assert.deepEqual(await r.ask(q, undefined, a), [['fallback', { url: 'http://h/w.js?a=1' }]], JSON.stringify(a)); + } const s = '89'.repeat(16); assert.equal((await r.ask(markWorker('http://h/s.js', 's', s), answerOf(200, 'app()'), as('xhr', 'sharedworker')))[1][1].body, workerAnswer(s, 0, 's', 'http://h/s.js')); assert.deepEqual(await r.ask(markWorker('http://h/m.js', 'p', t, 0), undefined, as('script', 'worker')), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(t, 0, 'm') }]]); - assert.equal(answered.length, 5); + assert.equal(answered.length, 8); }); test('workerRoutes answers a seeded Web Worker\'s word of a worker keploy cannot mark (marked t) itself, with no body, and hears its address; one with no address is answered and names none; neither reaches a route of the test\'s', async () => { @@ -2193,7 +2209,20 @@ test('cspAllowsData: no policy, or each policy whose script or worker source lis test('workerAnswer: a classic Web or Shared Worker runs its seeding, then imports the app\'s script at its URL in a microtask it queues, with none of its own code on the stack; its location shows no mark; a module one imports its seeding as a data: module, then the app\'s script', () => { const t = '45'.repeat(16); - for (const kind of ['s', 'w']) { + // A classic Web Worker whose script keploy could not fetch loads its + // seeding at its q mark first, then the app's script, in a microtask. + const wh = markWorker('http://h/w.js?q=1', 'w', t); + const wq = markWorker('http://h/w.js?q=1', 'q', t, 2); + const imp = []; + const later = []; + const ww = workerAt(wh, '', { importScripts: function (...u) { imp.push(u); if (u[0] === wq) ww.run(workerScript(t, 2, 'w')); }, queueMicrotask: (f) => { later.push(f); } }); + ww.run(workerAnswer(t, 2, 'w', 'http://h/w.js?q=1')); + assert.deepEqual([imp, later.length], [[[wq]], 1]); + later[0](); + assert.deepEqual(imp, [[wq], ['http://h/w.js?q=1']]); + assert.deepEqual(ww.draws(), workerAt(wh, workerScript(t, 2, 'w')).draws()); + assert.equal(ww.run('location.href'), 'http://h/w.js?q=1'); + for (const kind of ['s']) { const href = markWorker('http://h/s.js?q=1', kind, t); const imported = []; const queued = []; @@ -2292,7 +2321,7 @@ test('workerRoutes keeps keploy\'s route ahead of every route the test adds to i const pass = async (route, q) => { seen.push(q.url()); await route.continue(); }; await page.route(() => true, pass); const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; - const seeded = (n) => [['fetch', 'http://h/w.js?a=1'], ['fulfill', { status: 200, headers, body: workerAnswer(t, n, 'w', 'http://h/w.js?a=1') }]]; + const seeded = (n) => [['fetch', 'http://h/w.js?a=1'], ['fulfill', { status: 200, headers, body: workerAnswer(t, n, 'w', 'http://h/w.js?a=1', true, 'app()') }]]; assert.deepEqual(await r.ask(w, answerOf(200, 'app()'), undefined, page), seeded(0)); assert.deepEqual(seen, ['http://h/w.js?a=1']); assert.deepEqual(answered, ['http://h/w.js?a=1']); @@ -2339,21 +2368,21 @@ test('workerRoutes sends a worker\'s script on with what the test\'s continue an const ask = (m, body, dest) => r.ask(m, answerOf(200, body), { type: 'script', headers: { 'sec-fetch-dest': dest, accept: '*/*' } }, page); const sent = (dest) => ({ 'sec-fetch-dest': dest, accept: '*/*' }); const auth = (dest) => ({ ...sent(dest), 'x-auth': 'k' }); - const seeded = (url, h, n, kind, own, tok = t) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body: workerAnswer(tok, n, kind, own) }]]; + const seeded = (url, h, n, kind, own, tok = t, src = undefined) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body: workerAnswer(tok, n, kind, own, true, src) }]]; const whole = (url, h, body) => [h === undefined ? ['fetch', url] : ['fetch', url, h], ['fulfill', { status: 200, headers, body }]]; // A Web Worker's: keploy's fetch sends what the test's route asked (its // seeding's load of the app's script gets what that fetch got: a row below). const w = markWorker('http://h/w.js', 'w', t); mine = (route) => route.continue({ headers: auth('worker') }); - assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 0, 'w', 'http://h/w.js')); + assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 0, 'w', 'http://h/w.js', t, 'app()')); mine = (route) => route.continue({ url: 'http://h/stand.js' }); - assert.deepEqual(await ask(w, 'stand()', 'worker'), seeded('http://h/stand.js', undefined, 1, 'w', 'http://h/w.js')); + assert.deepEqual(await ask(w, 'stand()', 'worker'), seeded('http://h/stand.js', undefined, 1, 'w', 'http://h/w.js', t, 'stand()')); // Fallbacks: keploy's last route answers it with what they gave, at the // URL one gave too. mine = (route) => route.fallback({ headers: auth('worker') }); - assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 2, 'w', 'http://h/w.js')); + assert.deepEqual(await ask(w, 'app()', 'worker'), seeded('http://h/w.js', auth('worker'), 2, 'w', 'http://h/w.js', t, 'app()')); mine = (route) => route.fallback({ url: 'http://h/stand.js' }); - assert.deepEqual(await ask(w, 'stand()', 'worker'), seeded('http://h/stand.js', undefined, 3, 'w', 'http://h/w.js')); + assert.deepEqual(await ask(w, 'stand()', 'worker'), seeded('http://h/stand.js', undefined, 3, 'w', 'http://h/w.js', t, 'stand()')); assert.deepEqual(heard, []); // A Shared Worker's: the browser's own headers (in another case), an // option left undefined, or none, seed it. @@ -2416,106 +2445,100 @@ test('workerRoutes sends a worker\'s script on with what the test\'s continue an assert.deepEqual(heard, [['http://h/s.js', 'continued']]); }); -test('workerRoutes answers a seeded Web Worker\'s seeding\'s load of the app\'s script with what its fetch for the worker\'s own load got, ahead of the test\'s routes, once: told by its URL, its Referer (the worker\'s marked URL, or what the script\'s Referrer-Policy sends to its own origin) and its kind (importScripts for a classic worker, a static import for a module one); none for a script load keploy handed on, a Shared Worker\'s, or an answer not ok', async () => { +test('workerRoutes answers a Web Worker\'s script with the app\'s script it fetched, its seeding\'s load ahead of it, under any Referrer-Policy: the test\'s route sees the worker\'s script once, keploy answers its seeding at its p or q mark only, and a load at the app\'s URL after it is the app\'s own, to the test\'s routes; an answer not ok goes as given; a Shared Worker whose script a route continues at another query gets that answer, whole, and is named', async () => { const { r, context } = routed(); const answered = []; await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); const page = r.page(); const hits = []; - let pass = false; - await page.route(() => true, (route) => { hits.push(route.request().url()); return pass ? route.fallback() : route.continue(); }); - const app = (policy, status = 200) => answerOf(status, 'app()', { 'content-type': 'text/javascript', ...(policy === undefined ? {} : { 'referrer-policy': policy }) }); - const start = (m, policy, status) => r.ask(m, app(policy, status), { type: 'script', headers: { referer: 'http://h/pg', 'sec-fetch-dest': workerMark(m).kind === 's' ? 'sharedworker' : 'worker' } }, page); - // The seeding's load: Chromium shows no destination and types it. - const load = (url, referer, type, dest) => r.ask(url, null, { type, headers: { ...(referer === null ? {} : { referer }), ...(dest === undefined ? {} : { 'sec-fetch-dest': dest }) } }, page); - const kept = (policy) => [['fulfill', { status: 200, headers: { 'content-type': 'text/javascript', ...(policy === undefined ? {} : { 'referrer-policy': policy }) }, body: 'app()' }]]; - const on = (url) => [['continue', url]]; - // A classic worker under its default policy: its importScripts, in each - // engine's terms, once; then a load there goes on. - const w = markWorker('http://h/w.js', 'w', 'd1'.repeat(16)); - assert.equal((await start(w))[0][0], 'fetch'); - assert.equal(r.match(new URL('http://h/w.js#f')), true); - assert.deepEqual(await load('http://h/w.js', w, 'other'), kept()); - assert.equal(r.match(new URL('http://h/w.js')), false); - assert.deepEqual(await load('http://h/w.js', w, 'other'), on('http://h/w.js')); - await start(w); - assert.deepEqual(await load('http://h/w.js', w, 'script', 'script'), kept()); - assert.deepEqual(hits, ['http://h/w.js', 'http://h/w.js', 'http://h/w.js']); - assert.deepEqual(answered, Array(4).fill('http://h/w.js')); - // Not a load of the seeding's kind, nor one with another Referer or at - // another URL: each goes on, and the seeding's load still gets it. - await start(w); - assert.deepEqual(await load('http://h/w.js', w, 'script'), on('http://h/w.js')); - assert.deepEqual(await load('http://h/w.js', w, 'script', 'worker'), on('http://h/w.js')); - assert.deepEqual(await load('http://h/w.js', 'http://h/pg', 'other'), on('http://h/w.js')); - assert.deepEqual(await load('http://h/w.js', null, 'other'), on('http://h/w.js')); - assert.deepEqual(await load('http://h/w.js?b=1', w, 'other'), on('http://h/w.js?b=1')); - assert.deepEqual(await load('http://h/w.js', w, 'other'), kept()); - // A module worker's static import of the app's script. - const m = markWorker('http://h/m.js', 'm', 'd2'.repeat(16)); - await start(m); - assert.deepEqual(await load('http://h/m.js', m, 'other'), on('http://h/m.js')); - assert.deepEqual(await load('http://h/m.js', m, 'script', 'script'), on('http://h/m.js')); - assert.deepEqual(await load('http://h/m.js', m, 'script'), kept()); - await start(m); - assert.deepEqual(await load('http://h/m.js', m, 'script', 'worker'), kept()); - // Under a policy that sends less to the worker's own origin: what it - // sends (Chromium), or the marked URL (Firefox and WebKit, for a classic - // worker); taken at one, it is gone from the other. - await start(w, 'no-referrer'); - assert.deepEqual(await load('http://h/w.js', null, 'other'), kept('no-referrer')); - assert.deepEqual(await load('http://h/w.js', w, 'script', 'script'), on('http://h/w.js')); - await start(w, 'no-referrer'); - assert.deepEqual(await load('http://h/w.js', w, 'script', 'script'), kept('no-referrer')); - assert.deepEqual(await load('http://h/w.js', null, 'other'), on('http://h/w.js')); - await start(m, 'origin'); - assert.deepEqual(await load('http://h/m.js', 'http://h/', 'script', 'script'), on('http://h/m.js')); - assert.deepEqual(await load('http://h/m.js', 'http://h/', 'script', 'worker'), kept('origin')); - // Two workers at one URL, each kept: each seeding's load gets one. - const w2 = markWorker('http://h/w.js', 'w', 'd3'.repeat(16)); - await start(w); - await start(w2); - assert.deepEqual(await load('http://h/w.js', w2, 'other'), kept()); - assert.deepEqual(await load('http://h/w.js', w, 'other'), kept()); - // Under no-referrer, with what each fetch got: one taken at its marked - // URL is gone from what no-referrer sends too, where the other's is. - const each = (m, body) => r.ask(m, answerOf(200, body, { 'content-type': 'text/javascript', 'referrer-policy': 'no-referrer' }), { type: 'script', headers: { referer: 'http://h/pg', 'sec-fetch-dest': 'worker' } }, page); - const got = async (...a) => (await load(...a))[0][1]?.body; - await each(w, 'one()'); - await each(w2, 'two()'); - assert.equal(await got('http://h/w.js', w, 'script', 'script'), 'one()'); - assert.equal(await got('http://h/w.js', null, 'other'), 'two()'); - assert.deepEqual(await load('http://h/w.js', null, 'other'), on('http://h/w.js')); - // A worker's script load keploy handed on is none, though it looks like - // a module worker's seeding's load: one at the URL of a worker kept under - // no-referrer, from a page under it too, is seeded. - await start(m, 'no-referrer'); - const m2 = markWorker('http://h/m.js', 'm', 'd4'.repeat(16)); - pass = true; - const second = await r.ask(m2, app('no-referrer'), { type: 'script', headers: {} }, page); - pass = false; - assert.equal(second[1][1].body, workerAnswer('d4'.repeat(16), 0, 'm', 'http://h/m.js')); - assert.deepEqual(await load('http://h/m.js', null, 'script'), kept('no-referrer')); - assert.deepEqual(await load('http://h/m.js', null, 'script'), kept('no-referrer')); - assert.deepEqual(await load('http://h/m.js', null, 'script'), on('http://h/m.js')); - // None for a Shared Worker's, or for an answer not ok. - const s = markWorker('http://h/s.js', 's', 'd5'.repeat(16)); - await start(s); - assert.equal(r.match(new URL('http://h/s.js')), false); - assert.deepEqual(await load('http://h/s.js', s, 'other'), on('http://h/s.js')); + await page.route(() => true, (route) => { hits.push(route.request().url()); return route.continue(); }); + const app = (policy, status = 200, body = 'app()') => answerOf(status, body, { 'content-type': 'text/javascript', ...(policy === undefined ? {} : { 'referrer-policy': policy }) }); + const start = (m, policy, status, body) => r.ask(m, app(policy, status, body), { type: 'script', headers: { referer: 'http://h/pg', 'sec-fetch-dest': 'worker' } }, page); + const load = (url, type, dest) => r.ask(url, null, { type, headers: dest === undefined ? {} : { 'sec-fetch-dest': dest } }, page); + const headers = (policy) => ({ 'content-type': 'text/javascript', ...(policy === undefined ? {} : { 'referrer-policy': policy }) }); + const tw = 'd1'.repeat(16); + const tm = 'd2'.repeat(16); + const w = markWorker('http://h/w.js', 'w', tw); + const m = markWorker('http://h/m.js', 'm', tm); + for (const [i, policy] of [undefined, 'no-referrer', 'origin', 'strict-origin'].entries()) { + assert.deepEqual(await start(w, policy), [['fetch', 'http://h/w.js'], ['fulfill', { status: 200, headers: headers(policy), body: `importScripts(${JSON.stringify(markWorker('http://h/w.js', 'q', tw, i))});app()\n//# sourceURL=http://h/w.js\n` }]], String(policy)); + assert.deepEqual(await start(m, policy, 200, 'import "./a.js";\nb()'), [['fetch', 'http://h/m.js'], ['fulfill', { status: 200, headers: headers(policy), body: `import ${JSON.stringify(markWorker('http://h/m.js', 'p', tm, i))};import "./a.js";\nb()\n//# sourceURL=http://h/m.js\n` }]], String(policy)); + } + assert.deepEqual(hits, Array(4).fill(['http://h/w.js', 'http://h/m.js']).flat()); + // Nothing is kept: a load at the app's URL, of any kind, is no seeding's. + for (const u of ['http://h/w.js', 'http://h/m.js']) assert.equal(r.match(new URL(u)), false); + for (const [type, dest] of [['other'], ['script', 'script'], ['script'], ['script', 'worker']]) { + assert.deepEqual(await load('http://h/w.js', type, dest), [['continue', 'http://h/w.js']]); + } + // The seedings, at their marks, are keploy's, past the test's routes. + assert.deepEqual(await load(markWorker('http://h/w.js', 'q', tw, 1), 'other'), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(tw, 1, 'w') }]]); + assert.deepEqual(await load(markWorker('http://h/m.js', 'p', tm, 3), 'script', 'worker'), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(tm, 3, 'm') }]]); + assert.equal(hits.length, 12); + assert.equal(answered.length, 10); + // An answer not ok goes to the worker as the server gave it. const x = markWorker('http://h/x.js', 'w', 'd6'.repeat(16)); - assert.deepEqual(await start(x, undefined, 404), [['fetch', 'http://h/x.js'], ['fulfill', { status: 404, headers: { 'content-type': 'text/javascript' }, body: 'app()' }]]); - assert.equal(r.match(new URL('http://h/x.js')), false); - assert.deepEqual(await load('http://h/x.js', x, 'other'), on('http://h/x.js')); + assert.deepEqual(await start(x, undefined, 404), [['fetch', 'http://h/x.js'], ['fulfill', { status: 404, headers: headers(), body: 'app()' }]]); + // A Shared Worker's script a route continues at another query: the + // answer to that, whole, unseeded, and named. + const two = routed(); + const heard = []; + await workerRoutes(two.context, { routed: (at, why) => heard.push([at, why]) }); + const p2 = two.r.page(); + await p2.route(() => true, (route) => route.continue({ url: 'http://h/s.js?v=2' })); + const s = markWorker('http://h/s.js', 's', 'd5'.repeat(16)); + assert.deepEqual(await two.r.ask(s, app(), { type: 'script', headers: { 'sec-fetch-dest': 'sharedworker' } }, p2), [['fetch', 'http://h/s.js?v=2'], ['fulfill', { status: 200, headers: headers(), body: 'app()' }]]); + assert.deepEqual(heard, [['http://h/s.js', 'continued']]); + // Where the route reaches a Web Worker's own workers (reach: WebKit), + // the seeding it serves at a p or q mark starts them seeded too. + const three = routed(); + await workerRoutes(three.context, {}, true); + assert.notEqual(workerScript(tw, 0, 'w', true), workerScript(tw, 0, 'w')); + assert.deepEqual(await three.r.ask(markWorker('http://h/w.js', 'q', tw, 0), undefined, { type: 'other', headers: {} }), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(tw, 0, 'w', true) }]]); + assert.deepEqual(await three.r.ask(markWorker('http://h/m.js', 'p', tm, 0), undefined, { type: 'script', headers: {} }), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(tm, 0, 'm', true) }]]); +}); + +test('workerAnswer: a Web Worker\'s is the app\'s script with its seeding\'s load first, on its first line (after a hashbang line), strict when the app\'s directive prologue says \'use strict\', then a sourceURL of the app\'s URL unless the app\'s script names one; useStrict reads a prologue as the language does', () => { + const t = 'ee'.repeat(16); + const q = JSON.stringify(markWorker('http://h/w.js?a=1#f', 'q', t, 4)); + const p = JSON.stringify(markWorker('http://h/m.js', 'p', t, 4)); + const tail = '\n//# sourceURL=http://h/w.js?a=1\n'; + const w = (src) => workerAnswer(t, 4, 'w', 'http://h/w.js?a=1#f', true, src); + assert.equal(w('a()\nb()'), `importScripts(${q});a()\nb()${tail}`); + assert.equal(w('\'use strict\';\na()'), `"use strict";importScripts(${q});'use strict';\na()${tail}`); + assert.equal(w('#!/x\r\n"use strict"\na()'), `#!/x\r\n"use strict";importScripts(${q});"use strict"\na()${tail}`); + assert.equal(w('#!/x'), `#!/x\nimportScripts(${q});${tail}`); + assert.equal(w('a()\n//# sourceURL=mine.js'), `importScripts(${q});a()\n//# sourceURL=mine.js`); + assert.equal(w('a() /*@ sourceURL=mine.js */'), `importScripts(${q});a() /*@ sourceURL=mine.js */`); + assert.equal(workerAnswer(t, 4, 'm', 'http://h/m.js', true, 'import "./a.js";\nb()'), `import ${p};import "./a.js";\nb()\n//# sourceURL=http://h/m.js\n`); + assert.equal(workerAnswer(t, 4, 'm', 'http://h/m.js', true, '#!/x\nb()'), `#!/x\nimport ${p};b()\n//# sourceURL=http://h/m.js\n`); + // Each line of the app's keeps its number. + assert.equal(w('a\nb\nc').split('\n').slice(0, 3).map((l) => l.slice(-1)).join(''), 'abc'); + for (const [src, strict] of [ + ['\'use strict\'', true], ['"use strict"; a()', true], ['/* c */ // d\n "use asm"\n\'use strict\'\nx', true], ['\ufeff "a"; "b";\n"use strict";', true], + ['#!/usr/bin/env node\n\'use strict\';', true], ['"use strict" /* c\n */ a()', true], + ['"use strict" + a', false], ['a(); "use strict";', false], ['"use\\x20strict";', false], ['"a" + b; "use strict";', false], + ['', false], ['"use strict', false], ['"use\nstrict"', false], ['#!/x', false], ['`use strict`;', false], ['// "use strict"', false], ['"x\\"y"; "use strict"', true], + ]) assert.equal(useStrict(src), strict, JSON.stringify(src)); + assert.equal(withPrefix('a', 'P;'), 'P;a'); + assert.equal(withPrefix('#!x\u2028a', 'P;'), '#!x\u2028P;a'); +}); + +test('a Web Worker\'s seeding: an ErrorEvent\'s filename at the worker\'s own marked URL shows it with no mark; another, one that is no string and the getter\'s own error stay as they were; a Shared Worker\'s seeding leaves it as it is', () => { + const t = 'ab'.repeat(16); + for (const kind of ['w', 'm', 's']) { + const href = markWorker('http://h/w.js?a=1', kind, t); + class ErrorEvent {} + const w = workerAt(href, '', { ErrorEvent }); + w.run('Object.defineProperty(ErrorEvent.prototype, \'filename\', { get() { if (!(this instanceof ErrorEvent)) throw new TypeError(\'Illegal invocation\'); return this.f; }, enumerable: true, configurable: true });'); + w.run(workerScript(t, 0, kind)); + const read = (f) => w.run(`(() => { const e = new ErrorEvent(); e.f = ${JSON.stringify(f)}; return e.filename; })()`); + assert.equal(read(href), kind === 's' ? href : 'http://h/w.js?a=1', kind); + assert.equal(read(`${href}#h`), kind === 's' ? `${href}#h` : 'http://h/w.js?a=1#h', kind); + for (const f of ['http://h/o.js', markWorker('http://h/w.js?a=1', kind === 'w' ? 'm' : 'w', t), markWorker('http://h/w.js', kind, 'cd'.repeat(16)), 7]) assert.equal(read(f), f, `${kind} ${f}`); + assert.equal(w.run('(() => { try { return Object.getOwnPropertyDescriptor(ErrorEvent.prototype, \'filename\').get.call({}); } catch (e) { return e.message; } })()'), 'Illegal invocation', kind); + } }); -test('ownReferer: the Referer a worker sends to its own origin under its script\'s Referrer-Policy, the last token that is a policy, in any case', () => { - const at = 'http://u:p@h:8/w.js?a=1&__keploy_worker=w.x#f'; - assert.equal(ownReferer(at), 'http://h:8/w.js?a=1&__keploy_worker=w.x'); - for (const p of ['', 'unsafe-url', 'same-origin', 'no-referrer-when-downgrade', 'origin-when-cross-origin', 'strict-origin-when-cross-origin', 'bogus', 'no-referrer, unsafe-url']) assert.equal(ownReferer(at, p), 'http://h:8/w.js?a=1&__keploy_worker=w.x', p); - for (const p of ['no-referrer', 'No-Referrer', 'origin, no-referrer', ' no-referrer , bogus', 'no-referrer,']) assert.equal(ownReferer(at, p), '', p); - for (const p of ['origin', 'strict-origin', 'no-referrer, STRICT-ORIGIN']) assert.equal(ownReferer(at, p), 'http://h:8/', p); -}); test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module one whose policy refuses data: with its seeding in the answer; each request it fulfills is answered', async () => { const { r, context } = routed(); @@ -2524,13 +2547,13 @@ test('workerRoutes answers a Shared Worker\'s script with workerAnswer, a module const t = '67'.repeat(16); const headers = { 'content-type': 'text/javascript', 'x-a': '1' }; const s0 = markWorker('http://h/s.js', 's', t); - assert.deepEqual(await r.ask(s0, answerOf(200, 'app()')), [['fetch', 'http://h/s.js'], ['fulfill', { status: 200, headers, body: workerAnswer(t, 0, 's', 'http://h/s.js') }]]); + assert.deepEqual(await r.ask(s0, answerOf(200, '\ufeffapp()')), [['fetch', 'http://h/s.js'], ['fulfill', { status: 200, headers, body: workerAnswer(t, 0, 's', 'http://h/s.js') }]]); assert.equal((await r.ask(s0, answerOf(200, 'app()')))[1][1].body, workerAnswer(t, 1, 's', 'http://h/s.js')); const x0 = markWorker('http://h/m.js', 'x', t.replace('6', '8')); assert.equal((await r.ask(x0, answerOf(200, 'export {}')))[1][1].body, workerAnswer(t.replace('6', '8'), 0, 'x', 'http://h/m.js', true)); const policy = { 'content-type': 'text/javascript', 'content-security-policy': "script-src 'self'" }; const x1 = markWorker('http://h/c.js', 'x', t.replace('6', '9')); - assert.deepEqual(await r.ask(x1, answerOf(200, 'export {}', policy)), [['fetch', 'http://h/c.js'], ['fulfill', { status: 200, headers: policy, body: workerAnswer(t.replace('6', '9'), 0, 'x', 'http://h/c.js', false) }]]); + assert.deepEqual(await r.ask(x1, answerOf(200, '\ufeffexport {}', policy)), [['fetch', 'http://h/c.js'], ['fulfill', { status: 200, headers: policy, body: workerAnswer(t.replace('6', '9'), 0, 'x', 'http://h/c.js', false) }]]); assert.notEqual(workerAnswer(t, 0, 'x', 'http://h/c.js', false), workerAnswer(t, 0, 'x', 'http://h/c.js', true)); // A classic one is not refused by a policy: it imports the app's script. const d0 = markWorker('http://h/d.js', 's', t.replace('6', 'a')); @@ -2549,7 +2572,7 @@ test('a worker\'s seeding: its fetch and Request ask for its script\'s own URL w // A Request that keeps what it was made with, and a fetch that keeps what // it was given; each refuses 'bad' with an error whose stack has a frame // at a marked URL. - const marked = (m) => { const e = {}; Object.defineProperty(e, 'stack', { value: `E: ${m}\n at f (${href}:1:2)\n at g (http://h/a.js:3:4)`, writable: true, configurable: true }); return e; }; + const marked = (m) => { const e = {}; Object.defineProperty(e, 'stack', { value: `E: ${m}\n at f (${href.replace('=w.', '=q.')}.0:1:2)\n at g (http://h/a.js:3:4)`, writable: true, configurable: true }); return e; }; const given = []; const fakes = () => ({ URL, diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 8c4a46a..05cde6b 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1536,9 +1536,11 @@ for (const engine of ENGINES) { assert.match(w.shared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/shared\.js$/); assert.match(w.mshared.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); for (const c of [w.c1, w.c2, w['c1+2']]) assert.match(c.href, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); - // The app's script runs at its own URL: its import.meta.url and a - // stack through it show no mark. - assert.match(w.mod.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mod\.js$/); + // A Shared Worker's app's script runs at its own URL: its + // import.meta.url shows no mark; a module Web Worker's runs at its + // marked URL, which its import.meta.url shows (workerAnswer). A stack + // through either shows no mark. + assert.equal(w.mod.meta, markWorker(w.mod.href, 'm', mod.token)); assert.match(w.mshared.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); for (const c of [w.c1, w.c2, w['c1+2']]) assert.match(c.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); // No code of keploy's is on a stack through the app's script, at its @@ -1818,7 +1820,9 @@ for (const engine of ENGINES) { if (seeds || k === 'a' || k === 'am') assert.deepEqual(w[k].draws, model, `${k} ${facts}`); else assert.notDeepEqual(w[k].draws, model, `${k} ${facts}`); } - for (const k of ['am', 'bm']) assert.equal(w[k].meta, w[k].href); + // A seeded module Web Worker runs the app's script at its marked URL: + // its import.meta.url shows the mark (workerAnswer). + for (const k of ['am', 'bm']) assert.equal(w[k].meta, seeds || k === 'am' ? markWorker(w[k].href, 'm', m[k].token) : w[k].href, k); const said = r.tests.nested.stdout.split('\n').filter((l) => l.startsWith('keploy: in nested, ')).map((l) => l.replace(/:\d+\//, ':/')); const at = (p) => `http://127.0.0.1:/nw/${p}`; assert.deepEqual(said.sort(), [unplacedLine('nested', 'blob:', 'worker'), sharedWorkerLine('nested', at('s.js')), From 29b01144331d32484915cbc50942176acd5f966a Mon Sep 17 00:00:00 2001 From: slayerjain Date: Sun, 15 Nov 2026 06:36:57 +0530 Subject: [PATCH 47/73] test(playwright): load a worker's script once under a document's policy Rows in each engine, seeded and with seeding off: a document whose referrer policy is no-referrer, origin or strict-origin, set by a meta tag or by a header on the HTML alone, starts a classic and a module Web Worker whose script a route sends on once with the header the server needs (times: 1), and two whose loads a route counts. Each runs seeded at the app's URL, the counting route sees one load per worker, and the app's server is asked for each Web Worker's script once, as with seeding off. A row for the error event: an error a seeded Web Worker throws, classic and module, names the app's script and the line it was thrown on, in the worker and in the page, as with seeding off. Signed-off-by: slayerjain --- playwright/test/fixtures/once.spec.ts | 46 +++++++++++++++++++- playwright/test/rows.test.mjs | 62 +++++++++++++++++++++------ 2 files changed, 94 insertions(+), 14 deletions(-) diff --git a/playwright/test/fixtures/once.spec.ts b/playwright/test/fixtures/once.spec.ts index 4ef8f3c..7e37e7c 100644 --- a/playwright/test/fixtures/once.spec.ts +++ b/playwright/test/fixtures/once.spec.ts @@ -9,14 +9,16 @@ import { test } from '../../dist'; // Workers at /ov/w.js (a needs x-auth: k; n and g need it too, under // Referrer-Policy no-referrer and origin; p asks nothing) and keeps what // each sent (its kind, location and draws), or 'error', under its title. +// The doc tests start them from a document under a referrer policy of its +// own, at /ovd// (a meta tag, or a header on the HTML only). const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); test.use({ serviceWorkers: 'block' }); type Start = { url: string; type?: 'classic' | 'module'; shared?: boolean }; -const start = async (page: Page, list: Start[]): Promise => { - await page.goto(`${base}/ov`); +const start = async (page: Page, list: Start[], doc = '/ov'): Promise => { + await page.goto(`${base}${doc}`); const got: unknown[] = []; // One at a time, in order: each worker's place is its order. for (const s of list) { @@ -102,3 +104,43 @@ test('two routes', async ({ context, page }) => { await page.route('**/ov/a*.js', (r) => r.fallback({ headers: r.request().headers() })); await start(page, [{ url: '/ov/aw.js' }, { url: '/ov/as.js', shared: true }]); }); + +// A document under each referrer policy, set by a meta tag or by a header +// on the HTML only, starts a classic Web Worker at /ov/aw.js and a module +// one at /ov/tw.js, each sent on once with the header the server needs +// (times: 1), then a classic and a module one at /ov/pw.js, whose loads a +// route counts (kept as -n). +for (const how of ['meta', 'header']) { + for (const policy of ['no-referrer', 'origin', 'strict-origin']) { + test(`doc ${how} ${policy}`, async ({ page }) => { + let n = 0; + await page.route('**/ov/aw.js', (r) => r.continue(auth(r)), { times: 1 }); + await page.route('**/ov/tw.js', (r) => r.continue(auth(r)), { times: 1 }); + await page.route('**/ov/pw.js', (r) => { + n += 1; + return r.continue(); + }); + await start(page, [{ url: '/ov/aw.js' }, { url: '/ov/tw.js', type: 'module' }, { url: '/ov/pw.js' }, { url: '/ov/pw.js', type: 'module' }], `/ovd/${how}/${policy}`); + keep(`${test.info().title}-n`, n); + }); + } +} + +// A classic and a module Web Worker at /ov/ew.js throw after they start: +// kept is the filename and line the error event names in the worker (as +// the worker posts it) and in the page. +test('error event', async ({ page }) => { + await page.goto(`${base}/ov`); + const got: unknown[] = []; + for (const type of ['classic', 'module'] as const) { + got.push(await page.evaluate((t) => new Promise((resolve) => { + const seen: Record<string, unknown> = {}; + const done = (): void => { if (seen.in !== undefined && seen.page !== undefined) resolve(seen); }; + const w = new Worker('/ov/ew.js', { type: t }); + w.onmessage = (e) => { seen.in = e.data; done(); }; + w.onerror = (e) => { seen.page = { filename: e.filename, lineno: e.lineno }; e.preventDefault(); done(); }; + setTimeout(() => resolve(seen), 10000); + }), type)); + } + keep(test.info().title, got); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 05cde6b..9e5c7d0 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -655,12 +655,26 @@ async function drawsServer() { // with no x-auth: k; /ov/ow.js and /ov/os.js are not there (404), and // their stand-ins /ov/mw.js and /ov/ms.js are; /ov/pw.js and /ov/ps.js // ask nothing; once.spec.ts's /ov/nw.js and /ov/gw.js need x-auth: k too, - // under Referrer-Policy no-referrer and origin. Each Web Worker (w) and - // Shared Worker (s) sends its kind, - // its location and what it drew. + // under Referrer-Policy no-referrer and origin, and /ov/tw.js needs it + // with none. Each Web Worker (w) and Shared Worker (s) sends its kind, + // its location and what it drew. /ovd/<how>/<policy> is /ov's document + // under that referrer policy, by a meta tag or a header on it alone. if (u.pathname === '/ov') { res.writeHead(200, { 'content-type': 'text/html' }); res.end(page('ov')); return; } + const ovd = /^\/ovd\/(meta|header)\/(no-referrer|origin|strict-origin)$/.exec(u.pathname); + if (ovd !== null) { + res.writeHead(200, { 'content-type': 'text/html', ...(ovd[1] === 'header' ? { 'referrer-policy': ovd[2] } : {}) }); + res.end(ovd[1] === 'meta' ? page('ov').replace('<head>', `<head><meta name="referrer" content="${ovd[2]}">`) : page('ov')); + return; + } if (u.pathname === '/ov/ow.js' || u.pathname === '/ov/os.js') { res.writeHead(404, { 'content-type': 'text/plain' }); res.end('none'); return; } - const ov = /^\/ov\/([ampng])([ws])\.js$/.exec(u.pathname); + // /ov/ew.js: a worker that posts what its error event names, then throws + // on its second line. + if (u.pathname === '/ov/ew.js') { + res.writeHead(200, { 'content-type': 'text/javascript' }); + res.end('self.addEventListener(\'error\', (e) => postMessage({ kind: \'ew-in\', filename: e.filename, lineno: e.lineno }));\nsetTimeout(() => { throw new Error(\'boom\'); });'); + return; + } + const ov = /^\/ov\/([ampngt])([ws])\.js$/.exec(u.pathname); if (ov !== null) { if (ov[1] !== 'm' && ov[1] !== 'p' && req.headers['x-auth'] !== 'k') { res.writeHead(401, { 'content-type': 'text/plain' }); res.end('no auth'); return; } const sent = `{ kind: '${ov[1]}${ov[2]}', href: location.href, draws: ${DRAWS} }`; @@ -1705,31 +1719,34 @@ for (const engine of ENGINES) { // once.spec.ts's tests, and the workers each one's page starts, in order // (each script's URL, and w a classic Web Worker, m a module one, s a -// Shared Worker). +// Shared Worker), and the doc tests' document paths (/ov for the others). +const DOCS = ['meta', 'header'].flatMap((how) => ['no-referrer', 'origin', 'strict-origin'].map((policy) => [how, policy])); +const DOC_PATH = Object.fromEntries(DOCS.map(([how, policy]) => [`doc ${how} ${policy}`, `/ovd/${how}/${policy}`])); const ONCE = [['once on page', [['/ov/aw.js', 'w']]], ['once on context, module', [['/ov/aw.js', 'm']]], ['by destination', [['/ov/aw.js', 'w']]], ['once under no-referrer', [['/ov/nw.js', 'w']]], ['once under no-referrer, module', [['/ov/nw.js', 'm']]], ['once under origin, module', [['/ov/gw.js', 'm']]], ['counted', [['/ov/pw.js', 'w'], ['/ov/pw.js', 'm']]], ['answered then continued at one address', [['/ov/as.js', 'w'], ['/ov/as.js', 's']]], - ['two routes', [['/ov/aw.js', 'w'], ['/ov/as.js', 's']]]]; + ['two routes', [['/ov/aw.js', 'w'], ['/ov/as.js', 's']]], + ...Object.keys(DOC_PATH).map((t) => [t, [['/ov/aw.js', 'w'], ['/ov/tw.js', 'm'], ['/ov/pw.js', 'w'], ['/ov/pw.js', 'm']]])]; -// The marks a document at /ov of page t0 gives the workers list starts. -function onceMarks(seed, list) { +// The marks a document at doc of page t0 gives the workers list starts. +function onceMarks(seed, list, doc = '/ov') { const made = []; class Worker { constructor(url) { made.push(workerMark(String(url))); } } const code = list.map(([u, k]) => (k === 's' ? `new SharedWorker('${u}');` : `new Worker('${u}'${k === 'm' ? ', { type: \'module\' }' : ''});`)).join(' '); - documentAt(initScript(seed), '/ov', new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }).run(code); + documentAt(initScript(seed), doc, new Map(), 't0', { globals: { URL, Worker, SharedWorker: Worker } }).run(code); return made; } for (const engine of ENGINES) { - test(`seeded, in ${engine}: a test's own route that treats a Web Worker's script loads apart (one that answers once, on its page or its context, one that picks the worker's script by its destination, one that counts its loads) sees what it sees with seeding off and the worker runs, seeded, at the app's URL: keploy answers the seeding's load of the app's script with what it fetched for the worker's own, under a Referrer-Policy that sends less too, so the app's server is asked for a Web Worker's script once; a Web Worker at one address whose script the test's route answers itself and a Shared Worker there whose script it sends on with a header are each named; of two routes, the last one's options count`, async () => { + test(`seeded, in ${engine}: a test's own route that treats a Web Worker's script loads apart (one that answers once, on its page or its context, one that picks the worker's script by its destination, one that counts its loads) sees what it sees with seeding off and the worker runs, seeded, at the app's URL: the worker's script is loaded once, keploy's seeding ahead of it, so the app's server is asked for a Web Worker's script once, under a script's Referrer-Policy and under a document's (no-referrer, origin, strict-origin, by a meta tag or a header), classic and module; an error a seeded Web Worker throws names the app's script and its line, in the worker and in the page; a Web Worker at one address whose script the test's route answers itself and a Shared Worker there whose script it sends on with a header are each named; of two routes, the last one's options count`, async () => { const j = journal(); const rec = await fakeAgent(j); try { - const names = [...ONCE.map(([t]) => t), 'counted-n']; + const names = [...ONCE.map(([t]) => t), 'counted-n', ...Object.keys(DOC_PATH).map((t) => `${t}-n`), 'error event']; const r = await drawRun(j, { ...keployEnv(rec.url), PW_BROWSER: engine }, ['once.spec.ts'], names); const off = await drawRun(j, { ...keployEnv(rec.url), PW_BROWSER: engine, KEPLOY_SEED_RANDOM: 'off' }, ['once.spec.ts'], names); for (const [title, list] of ONCE) { - const marks = onceMarks(r.seeds[title], list); + const marks = onceMarks(r.seeds[title], list, DOC_PATH[title]); const at = (u) => `http://127.0.0.1:<port>${u}`; const lines = []; list.forEach(([u, k], i) => { @@ -1767,6 +1784,27 @@ for (const engine of ENGINES) { // The counting route saw each worker's script once, as with seeding // off; the app's server was asked for the same scripts. assert.deepEqual([r.drew['counted-n'], off.drew['counted-n']], [2, 2]); + // Under each document's referrer policy too: the counting route saw + // its two workers' script twice, the app's server was asked once for + // each Web Worker's script (tw.js only by the doc tests; pw.js by them + // and counted), as with seeding off. + for (const t of Object.keys(DOC_PATH)) assert.deepEqual([r.drew[`${t}-n`], off.drew[`${t}-n`]], [2, 2], t); + const asked = (seen, u) => seen.filter((s) => s === u).length; + assert.deepEqual([asked(r.seen, '/ov/tw.js'), asked(off.seen, '/ov/tw.js')], [DOCS.length, DOCS.length]); + assert.deepEqual([asked(r.seen, '/ov/pw.js'), asked(off.seen, '/ov/pw.js')], [2 + 2 * DOCS.length, 2 + 2 * DOCS.length]); + // An error a seeded Web Worker throws, classic and module, names the + // app's script and the line it was thrown on, in the worker and in + // the page, as with seeding off: no mark. + const port = (x) => JSON.parse(JSON.stringify(x).replace(/:\d+\//g, ':<port>/')); + const ew = 'http://127.0.0.1:<port>/ov/ew.js'; + [0, 1].forEach((i) => { + const [got, was] = [port(r.drew['error event'][i]), port(off.drew['error event'][i])]; + const what = `error event ${i === 0 ? 'classic' : 'module'}: ${JSON.stringify(got)} (off: ${JSON.stringify(was)})`; + assert.deepEqual(got, was, what); + assert.deepEqual([got.in.kind, got.in.filename, got.page.filename], ['ew-in', ew, ew], what); + assert.equal(got.in.lineno, 2, what); + }); + assert.deepEqual(r.tests['error event'].stdout.split('\n').filter((l) => l.startsWith('keploy: in error event, ')), []); const scripts = (seen) => seen.filter((u) => u.startsWith('/ov/')).sort(); assert.deepEqual(scripts(r.seen), scripts(off.seen)); assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), []); From ece38a07bf7a826533bbd53f5ffcfe98e8c99a29 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 06:51:49 +0530 Subject: [PATCH 48/73] docs(playwright): say a Web Worker's script loads once, its seeding first The README's Workers paragraph said a Web Worker's answer was its seeding, which loaded the app's script again, and how keploy told that second load apart. Say what the code does: a Web Worker's answer is the app's script, with its seeding loaded ahead of it on its first line and a sourceURL of the app's URL; the test's routes and the app's server see the script loaded once under any referrer policy; an error event's filename shows no mark; a module Web Worker's import.meta.url is its marked URL; line numbers are kept and the first line's columns shift. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/playwright/README.md b/playwright/README.md index df202dc..710c127 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request (the app's server is asked for the script with no mark) with the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: its `import.meta.url`, the URLs its imports resolve against and a stack trace through it show no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), and the worker's `location` shows none. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them), and keploy answers a Web Worker's seeding's load of the app's script with the script it got so, ahead of the test's routes: the test's routes and the app's server see a Web Worker's script loaded once, as with seeding off. keploy tells that load by its URL, its `Referer` (the worker's marked URL, or what the script's `Referrer-Policy` sends to its own origin: none under `no-referrer`, the origin under `origin` and `strict-origin`) and its kind (`importScripts` for a classic worker, a static import for a module one), and a worker's script load it handed on the test's routes is never one. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker's seeding's load of the app's script gets the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); From 2d8d48199e7d661995debb5aeaf7c8ce752ce597 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 07:12:53 +0530 Subject: [PATCH 49/73] fix(playwright): stop waiting on a terminated Web Worker's script load Chromium reports no end, ever, for the script load of a Web Worker that a page terminates while that script loads: no requestfinished, no requestfailed and no 'worker' event. Settle waited for it, so the test hung until its timeout failed it, seeding on or off (0.1.0 and 0.1.1). The init script every keploy test's context runs (now workersScript) wraps the page's Worker as it wraps SharedWorker: each document numbers its Web Workers at a URL (resolved, no fragment) in the order it starts them, asks the test for its document number at its first one, and its terminate() reports the document, the URL and the worker's place. In Chromium, settle keeps each frame's script requests by URL (keploy's mark aside) per document, a new one at each commit with an answer, and drops the terminated worker's load, now or when it is reported. A load whose URL keploy marked is left to keploy's route; a document that terminates a worker as it unloads names its own document. Firefox and WebKit go on loading the script and report its end, which settle waits for. keploy's route now counts a worker's script as answered once its fulfill has run, even when that fulfill throws (the browser dropped the request of a terminated worker), so a seeded test does not wait on it either. Rows: unit rows for the page's Worker (numbering, terminate reports, kept constructor and terminate) and for settle's drop (place, frame, document, mark, engine); browser rows in each engine, seeded and off, for a classic and a module Web Worker terminated while their scripts load and for two Web Workers at one URL with the second terminated (the first one's load is still waited for). The module Web Worker row now also shows that URLs built from import.meta.url with a relative path or a query carry no mark, and one with a fragment alone keeps it. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/src/index.ts | 22 ++- playwright/src/random.ts | 10 +- playwright/src/settle.ts | 201 ++++++++++++++++++++---- playwright/test/fixtures/settle.spec.ts | 24 +++ playwright/test/rows.test.mjs | 49 +++++- playwright/test/unit.test.mjs | 120 +++++++++++++- 6 files changed, 378 insertions(+), 48 deletions(-) diff --git a/playwright/src/index.ts b/playwright/src/index.ts index ab6ced8..ccbd879 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -67,7 +67,7 @@ import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; -import { Inflight, sharedWorkerLine, sharedWorkerScript } from './settle'; +import { Inflight, sharedWorkerLine, workersScript } from './settle'; export { expect, KeployError }; export { seedFaker } from './random'; @@ -184,7 +184,9 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ // The lines that wait for a parent's parser to end, each said at the // test's end if it has not. const waiting = new Set<() => void>(); - const inflight = new Inflight(context); + // Chromium reports no end for the script load of a Web Worker + // terminated while it loads; its document's word drops it. + const inflight = new Inflight(context, browserName === 'chromium'); if (s.seed !== '') { // A document the seeding cannot place draws the browser's own numbers // and says so here, once for each address and reason in the test. @@ -265,18 +267,26 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ } // Seeded or not: each Shared Worker a page starts is one settle does not // wait on, and the test says so once for each script address (origin - // and path). + // and path); a Web Worker a page terminates drops its script load if + // the browser reports no end for it (workersScript). const sharedReport = `__keploy_shared_worker_${++sharedReports}`; const sharedTold = new Set<string>(); - await context.exposeBinding(sharedReport, (source, url: unknown) => { + await context.exposeBinding(sharedReport, (source, kind: unknown, url: unknown, n: unknown, k: unknown) => { + if (kind === 'd') return inflight.webWorkerDocument(source.frame); + if (kind === 't') { + inflight.webWorkerStopped(Number(url), String(n), Number(k)); + return undefined; + } + if (kind !== 's') return undefined; const u = String(url); inflight.sharedWorker(source.frame, u); const at = pageAddress(u.split('?')[0]); - if (sharedTold.has(at)) return; + if (sharedTold.has(at)) return undefined; sharedTold.add(at); process.stdout.write(`${sharedWorkerLine(name, at)}\n`); + return undefined; }); - await context.addInitScript({ content: sharedWorkerScript(sharedReport) }); + await context.addInitScript({ content: workersScript(sharedReport) }); const seeded = s.seed !== ''; const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; if (context.pages().length > 0) opened(); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index bd9e7c2..d92209b 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -2139,9 +2139,15 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // URL, method, headers and post data, or the request's (the URL the app // asked for, or one a fallback gave, and what fallbacks gave it). const answer = async (route: Route, m: WorkerMark, own: Route['fulfill'], h?: Handed, o: Asked = {}): Promise<void> => { + // Once keploy's fulfill has run, nothing of the request is in flight: + // one that throws (the worker was terminated while keploy fetched its + // script, and the browser dropped the request) ends it too. const fulfill = async (o: Parameters<Route['fulfill']>[0]): Promise<void> => { - await own(o); - hooks.answered?.(route.request()); + try { + await own(o); + } finally { + hooks.answered?.(route.request()); + } }; if (m.kind === 'p' || m.kind === 'q') { await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, m.kind === 'p' ? 'm' : 'w', reach) }); diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 3410f25..be69a7d 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -17,8 +17,22 @@ export class Inflight { // The script URLs, with no fragment, that each frame started a Shared // Worker at, for the test's life (sharedWorker). private readonly shared = new Map<Frame | null, Set<string>>(); + // Chromium (stops): each frame's documents, the one it has now last; in + // each, its script requests by URL (no fragment, no keploy mark), in the + // order reported, and the places of the Web Workers there it terminated. + private readonly docs = new Map<Frame, Doc[]>(); + // Each document's number (webWorkerDocument), the document it names. + private readonly numbered = new Map<number, Doc>(); + private documents = 0; - constructor(context: BrowserContext) { + /** + * stops: the browser reports no end for the script load of a Web Worker + * terminated while its script loads (Chromium: no requestfinished or + * requestfailed, ever), so settle drops that load when its document says + * the worker was terminated (webWorkerStopped). Firefox and WebKit go on + * loading it and report its end, which settle waits for. + */ + constructor(context: BrowserContext, private readonly stops = false) { context.on('request', (r) => { if (this.ended.has(r)) return; let frame: Frame | null = null; @@ -26,6 +40,7 @@ export class Inflight { try { frame = r.frame(); } catch { frame = null; } const nav = r.isNavigationRequest(); if (!nav && (this.shared.get(frame)?.has(noFragment(r.url())) || sharedScript(r.url()))) return; + if (this.stops && frame !== null && !nav && r.resourceType() === 'script' && this.loaded(frame, r)) return; this.open.set(r, { frame, nav, n: ++this.reported }); }); context.on('requestfinished', (r) => this.done(r)); @@ -60,7 +75,9 @@ export class Inflight { // route change) has no navigation request and replaces nothing. p.on('framenavigated', (f) => { // A commit with no answer (a route change) is numbered 0: it drops - // nothing. + // nothing. One with an answer is a new document: the Web Workers it + // starts are its own (stops). + if (this.stops && loading.has(f)) this.docs.get(f)?.push(newDoc()); const answered = loading.get(f) ?? 0; loading.delete(f); this.drop((o) => o.frame === f && !o.nav && o.n <= answered); @@ -76,7 +93,10 @@ export class Inflight { // which the browser goes on sending only until the test closes its // context), or the frame's own document request, still loading. Each // frame that goes is reported apart, a frame inside it too. - p.on('framedetached', (f) => this.drop((o) => o.frame === f)); + p.on('framedetached', (f) => { + this.docs.delete(f); + this.drop((o) => o.frame === f); + }); }; context.pages().forEach(watch); context.on('page', watch); @@ -114,7 +134,7 @@ export class Inflight { /** * A document of frame started a Shared Worker whose script is at url - * (sharedWorkerScript). Settle never waits on a request of that frame at + * (workersScript). Settle never waits on a request of that frame at * that URL (fragment aside) that is no navigation, reported before this or * after, for the rest of the test: the worker's script is one, and Chromium * reports no end for it, as Playwright does not follow Shared Workers. The @@ -131,6 +151,58 @@ export class Inflight { this.drop((o, r) => !o.nav && o.frame === frame && noFragment(r.url()) === u); } + /** + * The document frame has now asks its number, which it gives with each Web + * Worker it terminates (webWorkerStopped): a document that terminates one + * after the frame committed the next (in its unload handler) names its + * own. It asks as it starts, before any script of its own runs. + */ + webWorkerDocument(frame: Frame | null): number { + const n = ++this.documents; + if (this.stops && frame !== null) this.numbered.set(n, this.docOf(frame)); + return n; + } + + /** + * The document numbered doc terminated the k-th Web Worker (from 1) it + * started at url (no fragment): with stops, settle drops that worker's + * script load (the k-th script request the document's frame reported at + * that URL while the document was its own), now or when it is reported, + * if keploy's route does not answer it (a URL keploy marked: answered + * ends it). A load that ended, or a URL the document started no Web + * Worker at, drops nothing. + */ + webWorkerStopped(doc: number, url: string, k: number): void { + const d = this.numbered.get(doc); + if (d === undefined || !Number.isInteger(k) || k < 1) return; + const u = noFragment(url); + const at = loadsAt(d, u); + at.stopped.add(k); + const r = at.loads[k - 1]; + if (r !== undefined && workerMark(r.url()) === null) this.done(r); + } + + // Keeps r, a script request of frame, in its document's loads: true when + // it is the load of a Web Worker its document terminated already, which + // settle never waits on. + private loaded(frame: Frame, r: Request): boolean { + const m = workerMark(r.url()); + // keploy's own loads (a seeding, a word of a worker) start no worker. + if (m !== null && m.kind !== 'w' && m.kind !== 'm') return false; + const at = loadsAt(this.docOf(frame), noFragment(m === null ? r.url() : m.url)); + at.loads.push(r); + return m === null && at.stopped.has(at.loads.length); + } + + private docOf(frame: Frame): Doc { + let ds = this.docs.get(frame); + if (ds === undefined) { + ds = [newDoc()]; + this.docs.set(frame, ds); + } + return ds[ds.length - 1]; + } + private done(r: Request): void { if (this.open.delete(r)) this.wake(); } @@ -151,6 +223,21 @@ export class Inflight { } } +type Doc = Map<string, { loads: Request[]; stopped: Set<number> }>; + +function newDoc(): Doc { + return new Map(); +} + +function loadsAt(d: Doc, url: string): { loads: Request[]; stopped: Set<number> } { + let at = d.get(url); + if (at === undefined) { + at = { loads: [], stopped: new Set() }; + d.set(url, at); + } + return at; +} + function pageOf(f: Frame): Page | null { try { return f.page(); } catch { return null; } } @@ -173,47 +260,99 @@ function noFragment(url: string): string { /** * The init script every keploy test's context runs, seeded or not, after - * the seeding's (initScript): the page's SharedWorker, which starts each - * worker as the app asked, with the same arguments, then calls the binding - * report (exposed on the context, which this script takes off the window) - * with the script's URL, resolved against the document's base, with no - * fragment, so settle never waits on that script (Inflight.sharedWorker). + * the seeding's (initScript), which calls the binding report (exposed on + * the context, which this script takes off the window) with a kind first: + * - the page's SharedWorker starts each worker as the app asked, then + * reports ('s', url) the script's URL, resolved against the document's + * base, with no fragment, so settle never waits on that script + * (Inflight.sharedWorker); + * - the page's Worker starts each Web Worker as the app asked and numbers + * it k among the document's Web Workers at that URL (resolved, no + * fragment), from 1; at the document's first, it asks ('d') its number; + * its terminate() then reports ('t', number, url, k), once for each + * worker, so settle drops a script load the browser reports no end for + * (Inflight.webWorkerStopped). * A worker the browser refuses to start (a URL it cannot read) throws the - * browser's own error and reports nothing. It keeps new, instanceof, the - * constructor, name and length of the browser's own. + * browser's own error and is neither reported nor numbered. Each keeps new, + * instanceof, the constructor, name and length of the browser's own; + * terminate keeps its name, length and the property's flags. The page's + * own numbers are not drawn on. */ -export function sharedWorkerScript(report: string): string { +export function workersScript(report: string): string { const r = JSON.stringify(report); return `(() => { const g = globalThis; let tell = null; try { const f = g[${r}]; if (typeof f === 'function') tell = f; delete g[${r}]; } catch (e) { tell = null; } - const Real = g.SharedWorker; - if (typeof Real !== 'function' || tell === null) return; + if (tell === null) return; const U = g.URL; const construct = Reflect.construct; + const apply = Reflect.apply; const slice = Array.prototype.slice; - const W = { SharedWorker: function (url) { - // Called without new, it throws as the browser's own does. - if (!new.target) return Real.apply(this, arguments); - const w = construct(Real, slice.call(arguments), new.target); + const quiet = (p) => { if (p && typeof p.catch === 'function') p.catch(() => {}); return p; }; + // The script's URL as the browser resolves it, with no fragment. + const href = (url) => { + const d = g.document; + const u = new U(String(url), d ? d.baseURI : g.location.href); + u.hash = ''; + return u.href; + }; + // Puts a constructor in place of the browser's own at name, which calls + // after with each worker it starts and the worker's script URL. + const wrap = (name, after) => { + const Real = g[name]; + if (typeof Real !== 'function') return null; + const W = { [name]: function (url) { + // Called without new, it throws as the browser's own does. + if (!new.target) return apply(Real, this, arguments); + const w = construct(Real, slice.call(arguments), new.target); + try { after(w, href(url)); } catch (e) {} + return w; + } }[name]; + try { + W.prototype = Real.prototype; + Object.defineProperty(Real.prototype, 'constructor', { value: W, writable: true, enumerable: false, configurable: true }); + Object.setPrototypeOf(W, Object.getPrototypeOf(Real)); + const d = Object.getOwnPropertyDescriptor(g, name); + Object.defineProperty(g, name, { value: W, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); + } catch (e) { + // A window that takes no property: its workers are as the browser's. + return null; + } + return Real.prototype; + }; + wrap('SharedWorker', (w, u) => { quiet(tell('s', u)); }); + // Each Web Worker's URL and place k there, until it is terminated. + const places = new WeakMap(); + const get = WeakMap.prototype.get; + const set = WeakMap.prototype.set; + const del = WeakMap.prototype.delete; + const counts = Object.create(null); + let doc = null; + const P = wrap('Worker', (w, u) => { + const k = (counts[u] || 0) + 1; + counts[u] = k; + apply(set, places, [w, [u, k]]); + if (doc === null) doc = quiet(tell('d')); + }); + const d = P ? Object.getOwnPropertyDescriptor(P, 'terminate') : undefined; + if (!d || typeof d.value !== 'function') return; + const stop = d.value; + const T = { terminate: function () { + const out = apply(stop, this, arguments); try { - const d = g.document; - const u = new U(String(url), d ? d.baseURI : g.location.href); - u.hash = ''; - const p = tell(u.href); - if (p && typeof p.catch === 'function') p.catch(() => {}); + const at = apply(get, places, [this]); + if (at !== undefined && doc !== null) { + apply(del, places, [this]); + doc.then((n) => { quiet(tell('t', n, at[0], at[1])); }, () => {}); + } } catch (e) {} - return w; - } }.SharedWorker; + return out; + } }.terminate; try { - W.prototype = Real.prototype; - Object.defineProperty(Real.prototype, 'constructor', { value: W, writable: true, enumerable: false, configurable: true }); - Object.setPrototypeOf(W, Object.getPrototypeOf(Real)); - const d = Object.getOwnPropertyDescriptor(g, 'SharedWorker'); - Object.defineProperty(g, 'SharedWorker', { value: W, writable: d ? d.writable : true, enumerable: d ? d.enumerable : false, configurable: true }); + Object.defineProperty(P, 'terminate', { value: T, writable: d.writable, enumerable: d.enumerable, configurable: true }); } catch (e) { - // A window that takes no property: its Shared Workers are as the browser's. + // A prototype that takes no property: its terminate is the browser's. } })(); `; diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index 4a6b193..ea6bf2c 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -262,3 +262,27 @@ test('a page that starts a Shared Worker', async ({ page }) => { })); if (said !== 'up') throw new Error(`the Shared Worker said ${said}`); }); +test('a Web Worker its page terminates while its script loads', async ({ page }) => { + await page.goto(`${base}/page`); + const sent = page.waitForRequest(/\/heldw\.m?js(\?|$)/); + await page.evaluate(() => { + const g = globalThis as unknown as { w: Worker[] }; + g.w = [new Worker('/heldw.js'), new Worker('/heldw.mjs', { type: 'module' })]; + }); + await sent; + await reached(); + await page.evaluate(() => { (globalThis as unknown as { w: Worker[] }).w.forEach((w) => w.terminate()); }); +}); +test('two Web Workers at one URL, the second terminated while their scripts load', async ({ page }) => { + await page.goto(`${base}/page`); + const sent = page.waitForRequest(/\/heldw\.js(\?|$)/); + await page.evaluate(() => { + const g = globalThis as unknown as { w: Worker[]; up: Promise<unknown> }; + g.w = [new Worker('/heldw.js'), new Worker('/heldw.js')]; + // The first worker's script runs once the server answers it. + g.up = new Promise((resolve) => { g.w[0].onmessage = (e) => resolve(e.data); }); + }); + await sent; + await reached(); + await page.evaluate(() => { (globalThis as unknown as { w: Worker[] }).w[1].terminate(); }); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 9e5c7d0..f83c37e 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -70,6 +70,16 @@ async function pageServer(j) { // A Shared Worker that answers each page that connects. if (req.url === '/shared.js?v=1') { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end("onconnect = (e) => { e.ports[0].postMessage('up'); };"); return; } if (req.url === '/nocontent') { res.writeHead(204); res.end(); return; } + // A Web Worker's script, classic or module, answered only once /release + // is called. + if (req.url === '/heldw.js' || req.url === '/heldw.mjs') { + j.add({ page: 'heldw sent' }); + saw(); + res.on('close', () => { if (!res.writableFinished) j.add({ page: 'heldw cut' }); }); + res.on('finish', () => j.add({ page: 'heldw answered' })); + void released.then(() => { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end('postMessage("up");'); }); + return; + } if (req.url === '/resetcontent') { res.writeHead(205); res.end(); return; } // A disposition's type is case-insensitive (RFC 6266): the browser downloads "Attachment" too. if (req.url === '/download') { res.writeHead(200, { 'content-type': 'application/octet-stream', 'content-disposition': 'Attachment; filename="d.bin"' }); res.end('bytes'); return; } @@ -448,6 +458,36 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { }); } +// A Web Worker the page terminates while its script loads: Chromium reports +// no end for that load, ever (and no 'worker' event), so settle drops it on +// the page's word of the terminate; Firefox and WebKit go on loading it and +// report its end, which settle waits for (the server answers once settle's +// step has begun). Of two Web Workers at one URL, the one terminated is +// dropped and the other's load waited for. Seeded, keploy's route fetches +// the worker's script and ends the load when its fulfill has run. A settle +// that waits on the terminated load in Chromium ends the test by its +// timeout (the config's 30 s): no wall-clock bound in the row. +for (const engine of ['chromium', 'firefox', 'webkit']) { + for (const seed of ['seeded', 'off']) { + test(`settle, in ${engine}, ${seed}: a test whose page terminates a Web Worker (classic and module) while its script loads ends, and of two Web Workers at one URL the one terminated is not waited for and the other is`, async (t) => { + for (const title of ['a Web Worker its page terminates while its script loads', 'two Web Workers at one URL, the second terminated while their scripts load']) { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), ...(seed === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}), SLOW_BASE: p.url, NO_RELEASE: '1', RELEASE_AT_STEP: `${p.url}/release`, PW_BROWSER: engine }); + const r = tests[title]; + assert.equal(r.status, 'passed', `${title}\n${out}`); + const order = j.events.map((e) => (e.page ? e.page : e.agent.path)); + // The surviving worker's load is answered before the test ends. + if (title.startsWith('two')) assert.ok(order.indexOf('heldw answered') >= 0 && order.indexOf('heldw answered') < order.indexOf('/agent/scope/end'), order.join(', ')); + t.diagnostic(`${title}: ${order.filter((e) => e.startsWith('heldw') || e === '/agent/scope/end').join(', ')}; steps: ${r.steps.filter((x) => x.startsWith('keploy: wait for')).join(' | ')}`); + } finally { await p.close(); await a.close(); } + } + }); + } +} + // Settle rows where a call is still waited for: the page server answers it // only once settle's step has begun (the rows' reporter asks /release then), // so the test ends by settle's own wait, with no deadline on its body. The @@ -690,7 +730,7 @@ async function drawsServer() { // and sends what its own code gets at its script's URL (OWN). '/wk/ded.js': `importScripts('dep.js'); onmessage = (e) => postMessage({ kind: 'ded-early', data: e.data }); postMessage({ kind: 'ded', draws: ${DRAWS}, href: location.href, dep: self.dep, stack: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('ded')} ${REFS}.then((v) => postMessage({ kind: 'ded-ref', ...v }));`, '/wk/dep.js': 'self.dep = \'dep\';', - '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, top: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('mod')} ${REFS}.then((v) => postMessage({ kind: 'mod-ref', ...v }));`, + '/wk/mod.js': `import { dep } from './lib.mjs'; self.onmessage = (e) => postMessage({ kind: 'mod-early', data: e.data }); postMessage({ kind: 'mod', draws: ${DRAWS}, href: location.href, dep, meta: import.meta.url, rel: new URL('./lib.mjs', import.meta.url).href, up: new URL('../wk/data.json?v=2', import.meta.url).href, frag: new URL('#part', import.meta.url).href, top: String(new Error('at').stack), errs: ${ERRS} }); ${OWN('mod')} ${REFS}.then((v) => postMessage({ kind: 'mod-ref', ...v }));`, '/wk/lib.mjs': 'export const dep = \'lib\';', '/wk/shared.js': `const ref = ${REFS}; onconnect = (e) => { e.ports[0].postMessage({ kind: 'shared', draws: ${DRAWS}, href: location.href, stack: String(new Error('at').stack), errs: ${ERRS} }); ref.then((v) => e.ports[0].postMessage({ kind: 'shared-ref', ...v })); };`, // Module Shared Workers: one plain, one (started by two names, c1 @@ -1555,6 +1595,13 @@ for (const engine of ENGINES) { // marked URL, which its import.meta.url shows (workerAnswer). A stack // through either shows no mark. assert.equal(w.mod.meta, markWorker(w.mod.href, 'm', mod.token)); + // A URL the module builds from its import.meta.url with a relative + // path, or a query of its own, is the app's, with no mark (the path is + // the app's); one with a fragment alone keeps the URL's query, mark + // and all, as code that compares the exact URL sees it. + assert.equal(w.mod.rel, w.mod.href.replace(/mod\.js$/, 'lib.mjs')); + assert.equal(w.mod.up, w.mod.href.replace(/mod\.js$/, 'data.json?v=2')); + assert.equal(w.mod.frag, `${w.mod.meta}#part`); assert.match(w.mshared.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/mshared\.js$/); for (const c of [w.c1, w.c2, w['c1+2']]) assert.match(c.meta, /^http:\/\/127\.0\.0\.1:\d+\/wk\/cshared\.js$/); // No code of keploy's is on a stack through the app's script, at its diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index eb8644f..6878b4c 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -8,7 +8,7 @@ import os from 'node:os'; import path from 'node:path'; import { AgentClient, KeployError } from '../dist/agent.js'; import { decide } from '../dist/join.js'; -import { Inflight, sharedWorkerScript } from '../dist/settle.js'; +import { Inflight, workersScript } from '../dist/settle.js'; import { markWorker } from '../dist/random.js'; import { EventEmitter } from 'node:events'; import vm from 'node:vm'; @@ -363,23 +363,23 @@ test('settle: a request of the frame that started a Shared Worker at its script\ assert.deepEqual([woke, one.size], [true, 0]); }); -test('sharedWorkerScript: the page\'s SharedWorker starts each worker as asked, then reports its script URL against the document\'s base with no fragment; the binding leaves the window; a worker the browser refuses throws its own error and reports nothing; new, instanceof, constructor, name and length kept; with no binding the browser\'s own stays', () => { +test('workersScript: the page\'s SharedWorker starts each worker as asked, then reports its script URL against the document\'s base with no fragment; the binding leaves the window; a worker the browser refuses throws its own error and reports nothing; new, instanceof, constructor, name and length kept; with no binding the browser\'s own stays', () => { const made = []; const told = []; class Base {} class SharedWorker extends Base { constructor(url, o) { super(); if (String(url) === 'http://[' || String(url).endsWith('/refused.js')) throw new SyntaxError('the browser\'s own'); made.push([url, o]); } } - const ctx = { URL, document: { baseURI: 'http://h/a/p' }, location: { href: 'http://h/b/p' }, __r: (u) => { told.push(u); return Promise.reject(new Error('gone')); } }; + const ctx = { URL, document: { baseURI: 'http://h/a/p' }, location: { href: 'http://h/b/p' }, __r: (k, u) => { told.push(`${k} ${u}`); return Promise.reject(new Error('gone')); } }; // As a window's: not enumerable; the wrapper keeps the property's flags. Object.defineProperty(ctx, 'SharedWorker', { value: SharedWorker, writable: true, enumerable: false, configurable: true }); vm.createContext(ctx); - vm.runInContext(sharedWorkerScript('__r'), ctx); + vm.runInContext(workersScript('__r'), ctx); assert.equal(vm.runInContext('typeof __r', ctx), 'undefined'); const o = { name: 'n', type: 'module' }; ctx.o = o; vm.runInContext('globalThis.w = new SharedWorker("s.js?q=1#f", o)', ctx); assert.deepEqual(made, [['s.js?q=1#f', o]]); assert.equal(made[0][1], o); - assert.deepEqual(told, ['http://h/a/s.js?q=1']); + assert.deepEqual(told, ['s http://h/a/s.js?q=1']); assert.equal(vm.runInContext('w instanceof SharedWorker && SharedWorker.prototype.constructor === SharedWorker && SharedWorker.name === "SharedWorker" && SharedWorker.length === 1', ctx), true); assert.equal(vm.runInContext('const d = Object.getOwnPropertyDescriptor(globalThis, "SharedWorker"); JSON.stringify([d.enumerable, d.writable, d.configurable])', ctx), '[false,true,true]'); assert.equal(Object.getPrototypeOf(ctx.SharedWorker), Base); @@ -389,19 +389,123 @@ test('sharedWorkerScript: the page\'s SharedWorker starts each worker as asked, // A window with no document yet: its location. delete ctx.document; vm.runInContext('new SharedWorker("s.js")', ctx); - assert.deepEqual(told, ['http://h/a/s.js?q=1', 'http://h/b/s.js']); + assert.deepEqual(told, ['s http://h/a/s.js?q=1', 's http://h/b/s.js']); // A report that throws: the worker is still the app's. const c2 = { URL, SharedWorker, location: { href: 'http://h/p' }, __r: () => { throw new Error('gone'); } }; vm.createContext(c2); - vm.runInContext(sharedWorkerScript('__r'), c2); + vm.runInContext(workersScript('__r'), c2); assert.equal(vm.runInContext('new SharedWorker("/s.js") instanceof SharedWorker', c2), true); // No binding: the browser's own stays. const c3 = { URL, SharedWorker, location: { href: 'http://h/p' } }; vm.createContext(c3); - vm.runInContext(sharedWorkerScript('__r'), c3); + vm.runInContext(workersScript('__r'), c3); assert.equal(c3.SharedWorker, SharedWorker); }); +test('workersScript: the page\'s Worker starts each Web Worker as asked and numbers it among the document\'s Web Workers at its URL (resolved, no fragment), from 1; the first asks the document\'s number; terminate() calls the browser\'s own, then reports the number, URL and place once for each worker; a refused worker is not numbered; a terminate of no worker throws the browser\'s own error; called without new, Worker throws as the browser\'s does; new, instanceof, constructor, name, length and terminate\'s name, length and flags kept', async () => { + const told = []; + const stopped = []; + class Base {} + class Worker extends Base { + constructor(url) { super(); if (String(url).endsWith('/refused.js')) throw new SyntaxError('the browser\'s own'); } + } + Object.defineProperty(Worker.prototype, 'terminate', { value: function terminate() { if (!(this instanceof Worker)) throw new TypeError('Illegal invocation'); stopped.push(this); }, writable: true, enumerable: true, configurable: true }); + const ctx = { URL, document: { baseURI: 'http://h/a/p' }, location: { href: 'http://h/b/p' }, __r: (...a) => { told.push(a); return Promise.resolve(a[0] === 'd' ? 7 : undefined); } }; + Object.defineProperty(ctx, 'Worker', { value: Worker, writable: true, enumerable: false, configurable: true }); + vm.createContext(ctx); + vm.runInContext(workersScript('__r'), ctx); + assert.throws(() => vm.runInContext('new Worker("/refused.js")', ctx), (e) => e.message === 'the browser\'s own'); + assert.deepEqual(told, []); + vm.runInContext('globalThis.a = new Worker("w.js#x"); globalThis.b = new Worker("/a/w.js"); globalThis.c = new Worker("v.js")', ctx); + // The document's number is asked once, at its first Web Worker. + assert.deepEqual(told, [['d']]); + vm.runInContext('b.terminate(); b.terminate(); a.terminate(); c.terminate()', ctx); + assert.deepEqual(stopped, [ctx.b, ctx.b, ctx.a, ctx.c]); + await ticks(); + assert.deepEqual(told.slice(1), [['t', 7, 'http://h/a/w.js', 2], ['t', 7, 'http://h/a/w.js', 1], ['t', 7, 'http://h/a/v.js', 1]]); + assert.throws(() => vm.runInContext('Worker.prototype.terminate.call({})', ctx), (e) => e.message === 'Illegal invocation'); + assert.throws(() => vm.runInContext('Worker("/w.js")', ctx), TypeError); + assert.equal(stopped.length, 4); + assert.equal(vm.runInContext('a instanceof Worker && Worker.prototype.constructor === Worker && Worker.name === "Worker" && Worker.length === 1 && Worker.prototype.terminate.name === "terminate" && Worker.prototype.terminate.length === 0', ctx), true); + assert.equal(vm.runInContext('const d = Object.getOwnPropertyDescriptor(Worker.prototype, "terminate"); JSON.stringify([d.enumerable, d.writable, d.configurable])', ctx), '[true,true,true]'); + // A document whose number keploy could not give: its terminate reports + // nothing, and stops the worker. + const t2 = []; + const c2 = { URL, Worker, location: { href: 'http://h/p' }, __r: (...a) => { t2.push(a); return Promise.reject(new Error('gone')); } }; + vm.createContext(c2); + vm.runInContext(workersScript('__r'), c2); + vm.runInContext('new Worker("/w.js").terminate()', c2); + await ticks(); + assert.deepEqual(t2, [['d']]); + assert.equal(stopped.length, 5); +}); + +test('settle (Chromium): the script load of a Web Worker its document terminated is dropped, the k-th script request of its frame at that URL (fragment and keploy\'s mark aside) while the document was the frame\'s, reported before the word or after; a load that ended, another place, another frame, another document of the frame (its commit came between), a URL keploy marked for its own route (answered ends it) and another engine are waited for', async () => { + const ctx = fakeContext(); + const inflight = new Inflight(ctx, true); + const page = new EventEmitter(); + ctx.emit('page', page); + const f = { page: () => page }; + const g = { page: () => page }; + const load = (frame, url, type = 'script') => ({ frame: () => frame, isNavigationRequest: () => false, method: () => 'GET', url: () => url, resourceType: () => type }); + const doc = inflight.webWorkerDocument(f); + const one = load(f, 'http://h/w.js#a'); + const two = load(f, 'http://h/w.js'); + const fetched = load(f, 'http://h/w.js', 'fetch'); + const other = load(g, 'http://h/w.js'); + for (const r of [one, fetched, two, other]) ctx.emit('request', r); + assert.equal(inflight.size, 4); + let settled = false; + inflight.settled().then(() => { settled = true; }); + // The first ended; the second is dropped when its worker is terminated. + ctx.emit('requestfinished', one); + inflight.webWorkerStopped(doc, 'http://h/w.js', 1); + assert.equal(inflight.size, 3); + inflight.webWorkerStopped(doc, 'http://h/w.js#z', 2); + assert.equal(inflight.size, 2); + // A place the frame reported no load for yet: dropped when reported. + inflight.webWorkerStopped(doc, 'http://h/w.js', 3); + ctx.emit('request', load(f, 'http://h/w.js')); + assert.equal(inflight.size, 2); + // A word with no document, or a place that is no number: nothing. + inflight.webWorkerStopped(doc + 9, 'http://h/w.js', 4); + inflight.webWorkerStopped(doc, 'http://h/w.js', Number.NaN); + inflight.webWorkerStopped(doc, 'http://h/w.js', 0); + ctx.emit('request', load(f, 'http://h/w.js')); + assert.equal(inflight.size, 3); + // keploy's mark for a Web Worker's script counts as its URL, but keploy's + // route ends it; its seeding's own load counts nowhere. + const t = 'ab'.repeat(16); + const marked = load(f, markWorker('http://h/m.js', 'w', t)); + ctx.emit('request', load(f, markWorker('http://h/m.js', 'q', t, 0))); + ctx.emit('request', marked); + ctx.emit('request', load(f, 'http://h/m.js')); + inflight.webWorkerStopped(doc, 'http://h/m.js', 1); + assert.equal(inflight.size, 6); + inflight.webWorkerStopped(doc, 'http://h/m.js', 2); + assert.equal(inflight.size, 5); + // The frame commits a new document: its Web Workers are numbered anew, + // and the old document's word names the old one's loads. + ctx.emit('response', { request: () => ({ isNavigationRequest: () => true, frame: () => f }), status: () => 200, headers: () => ({}) }); + page.emit('framenavigated', f); + const fresh = load(f, 'http://h/w.js'); + ctx.emit('request', fresh); + const now = inflight.size; + inflight.webWorkerStopped(doc, 'http://h/w.js', 1); + assert.equal(inflight.size, now); + const next = inflight.webWorkerDocument(f); + inflight.webWorkerStopped(next, 'http://h/w.js', 1); + assert.equal(inflight.size, now - 1); + assert.equal(settled, false); + // Another engine: no drop. + const ctx2 = fakeContext(); + const off = new Inflight(ctx2); + const d2 = off.webWorkerDocument(f); + ctx2.emit('request', load(f, 'http://h/w.js')); + off.webWorkerStopped(d2, 'http://h/w.js', 1); + assert.equal(off.size, 1); +}); + test('settle: a cancel that comes while settle already waits wakes it (its page closes; then, for a service worker\'s call, the test closes its context)', async () => { for (const worker of [false, true]) { const c = worker ? 'a page\'s call and a service worker\'s' : 'a page\'s call'; From d989949efa4e559f27e109a5fede5bc15c68c4cf Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 07:12:53 +0530 Subject: [PATCH 50/73] docs(playwright): say what a terminated worker and import.meta.url do The README's "a call does not end" case says how settle treats the script load of a Web Worker a page terminates while it loads (Chromium reports no end for it, so the page's word drops it; Firefox and WebKit report the end, which settle waits for). A new case says that a seeded module Web Worker's import.meta.url is its marked URL, that URLs built from it with a relative path or a query of their own carry no mark, and that only code reading the exact URL, or adding a fragment alone, sees it, the same at record and at replay. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/playwright/README.md b/playwright/README.md index 710c127..8432a9a 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -98,7 +98,8 @@ keploy seeds, places and waits for what it can reach in the test browser and the - `given` needs the agent route `POST /agent/scope/given` (keploy/keploy#5016): keploy v3.6.192 and later serve it; with the enterprise keploy, that is an enterprise release on keploy v3.6.192 or later. An older keploy answers that route with a plain 404, and `given` fails the test with `keploy: this keploy agent answered 404 to POST /agent/scope/given, so it cannot take the given value <name>: given needs keploy v3.6.192 or later (keploy/keploy#5016), or an enterprise release on keploy v3.6.192 or later; an older keploy answers 404 there`. Claim, begin, settle and end do not need that route. - Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser and test process draw their own random numbers, and each test says so (`keploy: keploy answered the begin of <test> with no seed, ...`, above). Claim, begin, settle and end work with it as before. - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). -- **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. +- **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. +- **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: - a popup opened with noopener, as a link with `target="_blank"` opens one, or opened other than by `window.open` (a form, or a link with `rel="opener"`, that targets a new window); From e540b8e661cd2e79221be00e7eb5993cee443697 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 07:57:08 +0530 Subject: [PATCH 51/73] test(playwright): settle waits for terminated worker loads off Chromium The settle row for Web Workers a page terminates while their scripts load now also checks that every such load the server got is answered before the test ends in Firefox and WebKit, seeded or with seeding off: they go on loading it. Chromium ends the load at once, and, seeded, keploy's fetch of it is cut too. A unit row checks that a commit with no answer (a route change inside the page) keeps the document, so its word of a terminated worker still drops that load, and that a load at a URL keploy marked is waited for even when its place was terminated before it was reported. webWorkerStopped no longer checks that the place is a whole number from 1: no load is ever at another place, so it dropped nothing either way. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/src/settle.ts | 7 ++++--- playwright/test/rows.test.mjs | 9 +++++++++ playwright/test/unit.test.mjs | 22 ++++++++++++++++++++++ 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index be69a7d..b8b2982 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -169,12 +169,13 @@ export class Inflight { * script load (the k-th script request the document's frame reported at * that URL while the document was its own), now or when it is reported, * if keploy's route does not answer it (a URL keploy marked: answered - * ends it). A load that ended, or a URL the document started no Web - * Worker at, drops nothing. + * ends it). A load that ended, a URL the document started no Web Worker + * at, or a place that is no whole number from 1 (no load is there), + * drops nothing. */ webWorkerStopped(doc: number, url: string, k: number): void { const d = this.numbered.get(doc); - if (d === undefined || !Number.isInteger(k) || k < 1) return; + if (d === undefined) return; const u = noFragment(url); const at = loadsAt(d, u); at.stopped.add(k); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index f83c37e..ebf7b06 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -481,6 +481,15 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { const order = j.events.map((e) => (e.page ? e.page : e.agent.path)); // The surviving worker's load is answered before the test ends. if (title.startsWith('two')) assert.ok(order.indexOf('heldw answered') >= 0 && order.indexOf('heldw answered') < order.indexOf('/agent/scope/end'), order.join(', ')); + // Firefox and WebKit, seeded or off, go on loading a terminated + // worker's script and settle waits for it: every load the server + // got is answered before the test ends. Chromium ends the load + // at once (seeded, keploy's fetch of it is cut too). + if (engine !== 'chromium') { + const before = order.slice(0, order.indexOf('/agent/scope/end')); + const n = (e) => before.filter((x) => x === e).length; + assert.ok(n('heldw sent') > 0 && n('heldw answered') === n('heldw sent'), `${title}: ${order.join(', ')}`); + } t.diagnostic(`${title}: ${order.filter((e) => e.startsWith('heldw') || e === '/agent/scope/end').join(', ')}; steps: ${r.steps.filter((x) => x.startsWith('keploy: wait for')).join(' | ')}`); } finally { await p.close(); await a.close(); } } diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 6878b4c..9e49eb9 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -506,6 +506,28 @@ test('settle (Chromium): the script load of a Web Worker its document terminated assert.equal(off.size, 1); }); +test('settle (Chromium): a commit with no answer (a route change inside the page) keeps the document, so its word still drops the load of a Web Worker it terminated; a load at a URL keploy marked for its own route is waited for though its place was terminated before it was reported (keploy\'s route ends it)', async () => { + const ctx = fakeContext(); + const inflight = new Inflight(ctx, true); + const page = new EventEmitter(); + ctx.emit('page', page); + const f = { page: () => page }; + const load = (url) => ({ frame: () => f, isNavigationRequest: () => false, method: () => 'GET', url: () => url, resourceType: () => 'script' }); + const doc = inflight.webWorkerDocument(f); + ctx.emit('request', load('http://h/a.js')); + page.emit('framenavigated', f); + ctx.emit('request', load('http://h/w.js')); + assert.equal(inflight.size, 2); + inflight.webWorkerStopped(doc, 'http://h/w.js', 1); + assert.equal(inflight.size, 1); + // Its place terminated first, then the load, at a URL keploy marked. + inflight.webWorkerStopped(doc, 'http://h/m.js', 1); + ctx.emit('request', load(markWorker('http://h/m.js', 'w', 'cd'.repeat(16)))); + assert.equal(inflight.size, 2); + ctx.emit('request', load('http://h/m.js')); + assert.equal(inflight.size, 3); +}); + test('settle: a cancel that comes while settle already waits wakes it (its page closes; then, for a service worker\'s call, the test closes its context)', async () => { for (const worker of [false, true]) { const c = worker ? 'a page\'s call and a service worker\'s' : 'a page\'s call'; From 71ecf15550f7031560db1b8c17203caab7be73d1 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 08:39:30 +0530 Subject: [PATCH 52/73] fix(playwright): fail a route that unroutes itself as without keploy A page route of the test's own that takes itself off as it gets a Web Worker's script load, then continues it, fails with Playwright's error "route.continue: Route is already handled!" when seeding is off: the page has no route left, so Playwright sent the load on already. Seeded, keploy's own route on the page kept the page intercepted, so the continue went on and the test passed. keploy's routes stay on the page and on the context. When the test's unroute or unrouteAll, with any behavior, leaves a page no route of its own, keploy does what Playwright does there before the call returns: it answers each worker script load a route taken off still held, seeded, fetching the script once, and sends any other request on, through Playwright's own answers that leave the request to the route holding it. That route's continue, fulfill or abort then fails at the browser's end with Playwright's own error (dropped after 'ignoreErrors'), and its fallback goes on, as with no keploy. A load keploy or the test began to answer is waited for. The context's last route does the same for a request from no page or from a page with no route of the test's. The test's unrouteAll takes off the test's routes alone, with its behavior, and keeps keploy's, so the page or context is never left unrouted: Playwright never sends a request on with keploy's mark. With 'wait' it waits for the test's routes, as Playwright does. Where Playwright has no such way, keploy's routes are added again after it, as before. Rows: unit rows on a fake of Playwright's routing (its routes' handlers and the requests each holds, its unrouteAll path, its answers past its API, and one without them) for the routes kept, each of continue, fulfill, abort and fallback after unroute and unrouteAll with each behavior, a request other than a worker's, a load the test began to answer, a page that keeps a route of the test's, 'wait' from outside the route, and the context's last route from a page with and without a route of the test's and from no page; a browser row in each engine, seeded and with seeding off: after unroute a continue fails with Playwright's error and a fallback passes, after unrouteAll() a fulfill fails, after 'ignoreErrors' an abort passes, 'wait' from the test waits for the route's continue; the app's server is asked for each script once, never with the mark, and each passing test's worker draws its seeding's numbers. The README says what an unroute and unrouteAll do, and, under "What the run says when", that in Chromium a page's script at a terminated Web Worker's URL can shift which load settle stops waiting on. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 3 +- playwright/src/random.ts | 215 +++++++++++++++-- playwright/test/fixtures/unroute.spec.ts | 70 ++++++ playwright/test/random.test.mjs | 291 ++++++++++++++++++++++- playwright/test/rows.test.mjs | 53 +++++ 5 files changed, 608 insertions(+), 24 deletions(-) create mode 100644 playwright/test/fixtures/unroute.spec.ts diff --git a/playwright/README.md b/playwright/README.md index 8432a9a..81cc3a9 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and again after `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -99,6 +99,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser and test process draw their own random numbers, and each test says so (`keploy: keploy answered the begin of <test> with no seed, ...`, above). Claim, begin, settle and end work with it as before. - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). - **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. +- **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: diff --git a/playwright/src/random.ts b/playwright/src/random.ts index d92209b..3970ac1 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1985,18 +1985,57 @@ type Routes = Pick<BrowserContext, 'route' | 'unroute' | 'unrouteAll'>; * replaces is taken off, so no request finds neither. After the test's * unrouteAll, again comes first, then keploy's. Playwright runs a page's * routes before its context's, and the last added first. + * + * Playwright's unrouteAll takes keploy's route off too, and while target + * has no route Playwright sends each request it gets, and each one a route + * still held, on as the browser asked for it: a worker's, with keploy's + * mark. So where Playwright shows its list of target's routes and its own + * way to take some of them off, the test's unrouteAll takes off the test's + * routes alone (with the behavior it gives), and keploy's stay. Elsewhere + * keploy's are added again after it (again, then keploy's). + * + * When the test's unroute or unrouteAll leaves target no route of its own, + * Playwright stops intercepting target's requests and sends on each one a + * route it took off still held. keploy's route stays, so left does that, + * given the test's routes it took off. */ -function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Route) => Promise<void>, again: () => Promise<unknown> = async () => {}): Promise<void> { +function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Route) => Promise<void>, o: { again?: () => Promise<unknown>; left?: (removed: Held[]) => Promise<void>; also?: unknown } = {}): Promise<void> { const route = target.route; + const unroute = target.unroute; const unrouteAll = target.unrouteAll; + const inner = target as unknown as Inner; let now: ((route: Route) => Promise<void>) | null = null; + // keploy's handlers on target: each one keepFirst added, and also. + const mine = new WeakSet<object>(); + if (typeof o.also === 'function') mine.add(o.also); const add = async (): Promise<void> => { const h = (r: Route): Promise<void> => handler(r); const was = now; now = h; + mine.add(h); await route.call(target, url, h); - if (was !== null) await target.unroute(url, was); + if (was !== null) await unroute.call(target, url, was); + }; + const ours = (x: Held | null): boolean => typeof x?.handler === 'function' && mine.has(x.handler); + // Playwright's list of target's routes, or null where it shows none. + const list = (): Held[] | null => (Array.isArray(inner._routes) ? (inner._routes as Held[]) : null); + // Whether the test has a route of its own on target (true where + // Playwright shows no list). + const theirs = (): boolean => { + const rs = list(); + return rs === null || rs.some((x) => !ours(x)); }; + theirsOf.set(target, theirs); + const after = async (before: Held[] | null): Promise<void> => { + if (o.left === undefined || before === null || theirs()) return; + const rs = list() ?? []; + await o.left(before.filter((x) => !rs.includes(x) && !ours(x))); + }; + target.unroute = async function (this: Routes, ...args: Parameters<Routes['unroute']>) { + const before = list()?.slice() ?? null; + await unroute.apply(this, args); + await after(before); + } as Routes['unroute']; target.route = async function (this: Routes, ...args: Parameters<Routes['route']>) { const added = route.apply(this, args); const ahead = add(); @@ -2007,13 +2046,40 @@ function keepFirst(target: Routes, url: (u: URL) => boolean, handler: (route: Ro } } as Routes['route']; target.unrouteAll = async function (this: Routes, ...args: Parameters<Routes['unrouteAll']>) { - await unrouteAll.apply(this, args); - await again(); - await add(); + const rs = list(); + const before = rs?.slice() ?? null; + if (rs !== null && typeof inner._unrouteInternal === 'function') { + const kept = rs.filter(ours); + await inner._unrouteInternal.call(target, rs.filter((x) => !kept.includes(x)), kept, args[0]?.behavior); + if (typeof inner._disposeHarRouters === 'function') inner._disposeHarRouters.call(target); + } else { + await unrouteAll.apply(this, args); + await (o.again ?? (async () => {}))(); + await add(); + } + await after(before); } as Routes['unrouteAll']; return add(); } +/** + * Playwright's own, past its API (none of it there in some versions): a + * page's or context's list of its routes (_routes), its way to take some + * of them off, keeping the rest (_unrouteInternal, which unrouteAll calls + * with all of them), and the routes from HARs it lets go of with them. + */ +type Inner = { + _routes?: unknown; + _unrouteInternal?: (removed: Held[], remaining: Held[], behavior?: string) => Promise<void>; + _disposeHarRouters?: () => void; +}; + +/** A route in Playwright's list: its handler, and the requests it is handling. */ +type Held = { handler?: unknown; _activeInvocations?: Iterable<{ route?: Route }> }; + +/** Whether the test has a route of its own on a page or context keploy routes. */ +const theirsOf = new WeakMap<object, () => boolean>(); + /** What a test's route.continue or route.fallback asks a request be sent with. */ type Asked = NonNullable<Parameters<Route['continue']>[0]>; @@ -2116,11 +2182,14 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = return out; }; // The worker script loads handed on to the test's routes: each one's mark, - // the route's own fulfill, the headers the browser sent it with, and what - // the test's routes asked of it (their continue's and fallbacks' options); - // and the URLs it may reach the last route at (the app's, or one a - // fallback of the test's gave it). - type Handed = { m: WorkerMark; at: string; fulfill: Route['fulfill']; sent: Record<string, string>; method: string; asked: Asked }; + // the route's own methods, the headers the browser sent it with, what the + // test's routes asked of it (their continue's and fallbacks' options), + // whether keploy or the test began to answer it (started; end once that + // ended), and whether keploy answered it as a route of the test's that + // held it was taken off (gone: release); and the URLs it may reach the + // last route at (the app's, or one a fallback of the test's gave it). + type Own = { continue: Route['continue']; fallback: Route['fallback']; fulfill: Route['fulfill']; abort: Route['abort'] }; + type Handed = { m: WorkerMark; own: Own; sent: Record<string, string>; method: string; asked: Asked; started: boolean; gone: boolean; ended: () => void; end: Promise<void> }; const handed = new WeakMap<Request, Handed>(); const urls = new Set<string>(); const ask = (h: Handed, o: Asked | undefined): void => { @@ -2147,8 +2216,10 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await own(o); } finally { hooks.answered?.(route.request()); + h?.ended(); } }; + if (h !== undefined) h.started = true; if (m.kind === 'p' || m.kind === 'q') { await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, m.kind === 'p' ? 'm' : 'w', reach) }); return; @@ -2200,24 +2271,48 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = hooks.answered?.(r); return; } - const own = route.fulfill.bind(route); if (m.kind === 'p' || m.kind === 'q' ? !preludeLoad(r, m.kind) : !workerScriptLoad(r)) { await route.fallback({ url: m.url }); return; } if (m.kind === 'p' || m.kind === 'q') { - await answer(route, m, own); + await answer(route, m, route.fulfill.bind(route)); return; } - const h: Handed = { m, at: r.url(), fulfill: own, sent: r.headers(), method: r.method(), asked: {} }; - handed.set(r, h); + const h = hand(route, m, r.headers(), r.method(), {}); + await h.own.fallback({ url: m.url }); + }; + // Keeps route, a worker script's load handed on to the test's routes, as + // keploy's: its continue has keploy answer it (answer), its fallback goes + // on (at the URL it gives, which the context's routes then match), its + // fulfill answers it unseeded and the test hears of it (routed), its abort + // aborts it. Once keploy or the test began to answer it, each of the four + // waits until that ended, then does what Playwright's own does: continue, + // fulfill and abort, and a fallback after the test's own answer, fail + // with the error of a route already handled; a fallback after keploy + // answered it as its route was taken off (gone) goes on, as Playwright's + // does after it sent the request on. + const hand = (route: Route, m: WorkerMark, sent: Record<string, string>, method: string, asked: Asked): Handed => { + let ended = (): void => {}; + const end = new Promise<void>((resolve) => { ended = resolve; }); + const own: Own = { continue: route.continue.bind(route), fallback: route.fallback.bind(route), fulfill: route.fulfill.bind(route), abort: route.abort.bind(route) }; + const h: Handed = { m, own, sent, method, asked, started: false, gone: false, ended, end }; + handed.set(route.request(), h); urls.add(new URL(m.url).href); - const fallback = route.fallback.bind(route); route.continue = async (o) => { + if (h.started) { + await h.end; + return own.continue(o); + } ask(h, o); - await answer(route, m, own, h, o); + await answer(route, m, own.fulfill, h, o); }; route.fallback = async (o) => { + if (h.gone) return own.fallback(o).catch(() => {}); + if (h.started) { + await h.end; + return own.fallback(o); + } ask(h, o); if (o?.url !== undefined) { try { @@ -2226,14 +2321,35 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // Not a URL: Playwright refuses it. } } - await fallback(o); + await own.fallback(o); }; route.fulfill = async (o) => { + if (h.started) { + await h.end; + return own.fulfill(o); + } + h.started = true; const u = new URL(m.url); hooks.routed?.(u.origin + u.pathname, 'routed'); - await own(o); + try { + await own.fulfill(o); + } finally { + h.ended(); + } + }; + route.abort = async (e) => { + if (h.started) { + await h.end; + return own.abort(e); + } + h.started = true; + try { + await own.abort(e); + } finally { + h.ended(); + } }; - await fallback({ url: m.url }); + return h; }; const last = async (route: Route): Promise<void> => { const h = handed.get(route.request()); @@ -2241,7 +2357,62 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await route.fallback(); return; } - await answer(route, h.m, h.fulfill, h); + // One a route of the test's answered already (it ignored the error of + // its own answer after: unrouteAll's 'ignoreErrors') goes on, as it + // goes on with no keploy. + if (h.started) { + await h.end; + await h.own.fallback(); + return; + } + await answer(route, h.m, h.own.fulfill, h); + }; + // What Playwright does with the requests a route of the test's still held + // when the test took the last route of a page, or of its context, off: + // it sends each one on as the browser asked for it, so the route's own + // continue, fulfill or abort after that fails (a route already handled). + // keploy's route stays there, so keploy does it, and the test's unroute + // or unrouteAll returns once each is answered: a worker script's load + // keploy handed on that no one began to answer gets keploy's answer + // (seeded, at the app's URL, gone), one that keploy or the test began to + // answer is waited for, and any other request is continued. Of the + // context's, only one from no page or from a page with no route of the + // test's: Playwright sent the rest through their page. Where Playwright + // has them, keploy answers through its own answers past its API, which, + // as Playwright's own sending on, leave the request to the route that + // holds it: that route's continue, fulfill or abort then fails at the + // browser's end, with the error it gets with no keploy, and its fallback + // goes on to the next route. + type Quiet = { _innerFulfill?: Route['fulfill']; _innerContinue?: (fallback: boolean) => Promise<unknown> }; + const release = async (removed: Held[], sent: (r: Request) => boolean = () => true): Promise<void> => { + const ends: Promise<unknown>[] = []; + for (const x of removed) { + for (const { route } of x?._activeInvocations ?? []) { + if (route === undefined || !sent(route.request())) continue; + const h = handed.get(route.request()); + const q = route as unknown as Quiet; + if (h === undefined) { + const sendOn = q._innerContinue; + ends.push((typeof sendOn === 'function' ? sendOn.call(route, true) : route.continue()).catch(() => {})); + } else if (!h.started) { + h.gone = true; + const fulfill = q._innerFulfill; + ends.push(answer(route, h.m, typeof fulfill === 'function' ? (o) => fulfill.call(route, o) : h.own.fulfill, h).catch(() => {})); + } else { + ends.push(h.end); + } + } + } + await Promise.all(ends); + }; + const noPageRoute = (r: Request): boolean => { + let p: Page | null = null; + try { + p = r.frame().page(); + } catch { + // A Service Worker's request, or one whose frame is gone: no page. + } + return p === null || !(theirsOf.get(p)?.() ?? false); }; const marked = (u: URL): boolean => u.searchParams.has(WORKER_PARAM); const lastUrl = (u: URL): boolean => urls.has(u.href); @@ -2250,9 +2421,9 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await route.call(context, lastUrl, last); }; await addLast(); - await keepFirst(context, marked, first, addLast); + await keepFirst(context, marked, first, { again: addLast, left: (removed) => release(removed, noPageRoute), also: last }); const onPage = (p: Page): void => { - keepFirst(p, marked, first).catch(() => { + keepFirst(p, marked, first, { left: (removed) => release(removed) }).catch(() => { // A page that closed as it opened: none of its requests is left. }); }; diff --git a/playwright/test/fixtures/unroute.spec.ts b/playwright/test/fixtures/unroute.spec.ts new file mode 100644 index 0000000..e876574 --- /dev/null +++ b/playwright/test/fixtures/unroute.spec.ts @@ -0,0 +1,70 @@ +import fs from 'node:fs'; +import type { Route } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER), seeded and with seeding +// off. A route of the test's own page takes itself off as it gets a Web +// Worker's script load (the page's last route: unroute, or unrouteAll with +// a behavior), then answers the load: Playwright sends the load on as the +// page has no route left, so a continue, fulfill or abort after that fails +// ('Route is already handled!'; after 'ignoreErrors' the error is dropped) +// and a fallback does nothing more. unrouteAll's 'wait' from the test, +// while the route holds the load, waits for the route, whose continue +// sends it. The worker's message (its kind, location and draws) is kept +// under the test's title. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +test.use({ serviceWorkers: 'block' }); + +const work = (): Promise<unknown> => new Promise((resolve) => { + const w = new Worker('/ov/pw.js'); + w.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); +}); + +for (const how of ['continue', 'fallback'] as const) { + test(`unroute then ${how}`, async ({ page }) => { + await page.goto(`${base}/ov`); + const h = async (r: Route): Promise<void> => { + await page.unroute('**/ov/pw.js', h); + await (how === 'continue' ? r.continue() : r.fallback()); + }; + await page.route('**/ov/pw.js', h); + keep(test.info().title, await page.evaluate(() => new Promise((resolve) => { + const w = new Worker('/ov/pw.js'); + w.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + }))); + }); +} + +for (const [behavior, how] of [['default', 'fulfill'], ['ignoreErrors', 'abort']] as const) { + test(`unrouteAll ${behavior} then ${how}`, async ({ page }) => { + await page.goto(`${base}/ov`); + await page.route('**/ov/pw.js', async (r: Route): Promise<void> => { + await page.unrouteAll({ behavior }); + await (how === 'fulfill' ? r.fulfill({ contentType: 'text/javascript', body: 'postMessage("own")' }) : r.abort()); + }); + keep(test.info().title, await page.evaluate(work)); + }); +} + +test('unrouteAll wait then continue', async ({ page }) => { + await page.goto(`${base}/ov`); + let reached = (): void => {}; + const at = new Promise<void>((resolve) => { reached = resolve; }); + let go = (): void => {}; + const going = new Promise<void>((resolve) => { go = resolve; }); + await page.route('**/ov/pw.js', async (r: Route): Promise<void> => { + reached(); + await going; + await r.continue(); + }); + const done = page.evaluate(work); + await at; + const off = page.unrouteAll({ behavior: 'wait' }); + go(); + await off; + keep(test.info().title, await done); +}); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 5f9dc95..e69d194 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -2056,8 +2056,12 @@ function routed() { const listeners = []; context.pages = () => pages.slice(); context.on = (event, f) => { if (event === 'page') listeners.push(f); }; - r.page = () => { + // list: the page shows its routes as Playwright's page does (_routes, + // each with its handler), which keploy reads to know whether the test + // has a route there. + r.page = (list = false) => { const p = target(); + if (list) Object.defineProperty(p, '_routes', { get: () => p.routes }); pages.push(p); for (const f of listeners) f(p); return p; @@ -2640,3 +2644,288 @@ test('a worker\'s seeding: its fetch and Request ask for its script\'s own URL w assert.deepEqual([workerAt(href, workerScript(t, 0, 'w'), rest).run('typeof fetch + typeof Request'), workerAt(href, workerScript(t, 0, 'w'), rest2).run('typeof fetch + typeof Request')], [`undefined${typeof noFetch}`, `${typeof noRequest}undefined`]); }); + +test('workerRoutes keeps its route on a page and on the context through the test\'s unroute and unrouteAll: with Playwright\'s own way to take some routes off, unrouteAll takes the test\'s alone, with the behavior it gives, and lets go of its HARs; without it, keploy\'s are added again, ahead', async () => { + const { context, page, log } = played(); + await workerRoutes(context); + const p = page(); + await turn(); + const k = p._routes[0]; + assert.equal(p._routes.length, 1); + const h = async (route) => route.continue(); + const g = async (route) => route.continue(); + await p.route(uwAt, h); + await p.route(uwAt, g); + assert.deepEqual(p._routes.map((x) => x.handler).slice(1), [g, h]); + await p.unroute(uwAt, g); + await p.unroute(uwAt, h); + assert.equal(p._routes.length, 1); + for (const behavior of [undefined, 'default', 'wait', 'ignoreErrors']) { + await p.route(uwAt, h); + log.length = 0; + await p.unrouteAll(behavior === undefined ? undefined : { behavior }); + assert.deepEqual(log, [['page', 'off', 1, 1, behavior]], String(behavior)); + assert.equal(p._routes.length, 1, String(behavior)); + assert.equal(p._routes[0].match(new URL(markWorker('http://h/uw.js', 'w', 'f1'.repeat(16)))), true); + } + assert.equal(p.hars, 4); + assert.notEqual(p._routes[0], k); + await context.route(uwAt, h); + log.length = 0; + await context.unrouteAll({ behavior: 'wait' }); + // keploy's two routes there (its first and its last) stay. + assert.deepEqual(log, [['context', 'off', 1, 2, 'wait']]); + assert.equal(context.hars, 1); + // Without Playwright's own way: keploy's added again after the test's. + const { r, context: c2 } = routed(); + await workerRoutes(c2); + const q = r.page(true); + await turn(); + await q.route('**/w.js', h); + await q.unrouteAll(); + assert.equal(q.routes.length, 1); + assert.equal(q.routes[0].match(new URL(markWorker('http://h/w.js', 'w', 'f1'.repeat(16)))), true); + const n = c2.routes.length; + await c2.unrouteAll(); + assert.equal(c2.routes.length, n); +}); + +// A fake of Playwright's own routing, past its API: a page and the context +// keep their routes as Playwright does (_routes: each one's matcher, its +// handler and the requests it is handling now), take some off with +// _unrouteInternal ('wait' waits for what they handle; after +// 'ignoreErrors' an error a handler throws is dropped and the request goes +// on to the next route) and let go of their HARs. A request is handled +// once, as by Playwright: each route gets it to handle in turn, and once +// one answered it, continue, fulfill, abort and fallback throw Playwright's +// error; with inner, Playwright's own answers past its API (_innerFulfill, +// _innerContinue: sent on) answer it and leave it to the route that holds +// it, whose continue, fulfill or abort then fails at the browser's end +// (route.<call>: ...). What the request saw is in did: keploy's fetch, its seeded answer, +// another answer, an error a handler threw ('threw', or 'dropped' after +// 'ignoreErrors'), a handler that returned with no answer ('unanswered'), +// or the URL the browser sent it to when no route answered it ('sent'). +const uwAt = (u) => u.pathname === '/uw.js'; +function played(inner = true) { + const log = []; + const listeners = []; + const target = (name) => { + const t = { _routes: [], hars: 0 }; + t.route = async (match, handler) => { t._routes.unshift({ match, url: match, handler, _activeInvocations: new Set() }); }; + t.unroute = async (match, handler) => { + const removed = t._routes.filter((x) => x.match === match && (handler === undefined || x.handler === handler)); + await t._unrouteInternal(removed, t._routes.filter((x) => !removed.includes(x)), 'default'); + }; + t._unrouteInternal = async (removed, remaining, behavior) => { + t._routes = remaining; + log.push([name, 'off', removed.length, remaining.length, behavior]); + for (const x of removed) x.drop = behavior === 'ignoreErrors'; + if (behavior === 'wait') await Promise.all(removed.flatMap((x) => [...x._activeInvocations].map((i) => i.complete))); + }; + t._disposeHarRouters = () => { t.hars++; }; + t.unrouteAll = async (o) => { await t._unrouteInternal(t._routes, [], o?.behavior); t._disposeHarRouters(); }; + return t; + }; + const context = target('context'); + const pages = []; + context.pages = () => pages.slice(); + context.on = (event, f) => { if (event === 'page') listeners.push(f); }; + const page = () => { + const p = target('page'); + pages.push(p); + for (const f of listeners) f(p); + return p; + }; + // A request from page p (null: from no page) at url; fetch's answer waits + // for held. + const send = (url, p, held = Promise.resolve()) => { + const did = []; + let at = url; + let handling = false; + let here = false; + let answered = false; + const request = { url: () => at, resourceType: () => 'script', headers: () => ({}), method: () => 'GET', postData: () => null, frame: () => { if (p === null) throw new Error('no frame'); return { page: () => p }; } }; + const kind = (o) => (String(o.body).startsWith('importScripts(') ? 'seeded' : 'fulfill'); + const serve = (call, what) => { + if (answered) throw new Error(`route.${call}: Route is already handled!`); + answered = true; + did.push(what); + }; + const take = (call, what) => { + if (!handling) throw new Error('Route is already handled!'); + serve(call, what); + handling = false; + here = true; + }; + const route = { + request: () => request, + fetch: async () => { did.push('fetch'); await held; return answerOf(200, 'app()'); }, + fulfill: async (o) => take('fulfill', kind(o)), + continue: async () => take('continue', 'continue'), + abort: async () => take('abort', 'abort'), + fallback: async (o) => { + if (!handling) throw new Error('Route is already handled!'); + handling = false; + if (o?.url !== undefined) at = o.url; + }, + }; + if (inner) { + route._innerFulfill = async (o) => serve('fulfill', kind(o)); + route._innerContinue = async () => serve('continue', ['sent', at]); + } + const end = (async () => { + for (const t of p === null ? [context] : [p, context]) { + for (const x of t._routes.slice()) { + if (!x.match(new URL(at))) continue; + handling = true; + here = false; + let complete = () => {}; + const inv = { route, complete: new Promise((res) => { complete = res; }) }; + x._activeInvocations.add(inv); + try { + await x.handler(route, request); + } catch (e) { + did.push([x.drop ? 'dropped' : 'threw', e.message]); + if (x.drop) continue; + return; + } finally { + x._activeInvocations.delete(inv); + complete(); + } + if (here) return; + if (handling) { + did.push('unanswered'); + return; + } + } + } + if (!answered) did.push(['sent', at]); + })(); + return { did, end }; + }; + return { context, page, send, log }; +} + +// The test's route on a page takes a Web Worker's script load keploy handed +// on, then the test takes that route off (the page's last of the test's: +// unroute, or unrouteAll with behavior), then answers the load (how), or +// before (first). +async function takenOff(how, { first = false, off = (p, t) => p.unroute(uwAt, t), other = false, at = markWorker('http://h/uw.js', 'w', 'e1'.repeat(16)), inner = true } = {}) { + const { context, page, send } = played(inner); + await workerRoutes(context); + const p = page(); + await turn(); + if (other) await p.route(() => true, (route) => route.fallback()); + let release = () => {}; + const held = new Promise((res) => { release = res; }); + let early = null; + const call = (route) => (how === 'fulfill' ? route.fulfill({ body: 'own' }) : route[how]()); + const t = async (route) => { + const done = first ? call(route) : null; + const going = off(p, t); + await turn(); + release(); + await going; + early = x.did.slice(); + await (first ? done : call(route)); + }; + await p.route(uwAt, t); + const x = send(at, p, held); + await x.end; + return { did: x.did, early }; +} + +test('workerRoutes, when the test takes off a page\'s last route of its own while that route holds a worker script\'s load keploy handed on: keploy, whose route stays on the page, answers it, seeded, before the unroute or unrouteAll returns, as Playwright sends such a request on; the route\'s continue, fulfill or abort after that fails with Playwright\'s error, and its fallback goes on, as with no keploy', async () => { + for (const how of ['continue', 'fulfill', 'abort']) { + const x = await takenOff(how); + assert.deepEqual(x.early, ['fetch', 'seeded'], how); + assert.deepEqual(x.did, ['fetch', 'seeded', ['threw', `route.${how}: Route is already handled!`]], how); + } + const f = await takenOff('fallback'); + assert.deepEqual(f.did, ['fetch', 'seeded']); + // unrouteAll with any behavior; after 'ignoreErrors' the error is dropped. + for (const [behavior, how] of [[undefined, 'continue'], ['default', 'fulfill'], ['ignoreErrors', 'abort'], ['ignoreErrors', 'fallback']]) { + const x = await takenOff(how, { off: (p) => p.unrouteAll(behavior === undefined ? undefined : { behavior }) }); + assert.deepEqual(x.early, ['fetch', 'seeded'], `${behavior} ${how}`); + const after = how === 'fallback' ? [] : [[behavior === 'ignoreErrors' ? 'dropped' : 'threw', `route.${how}: Route is already handled!`]]; + assert.deepEqual(x.did, ['fetch', 'seeded', ...after], `${behavior} ${how}`); + } + // Any other request the route held is sent on, as Playwright sends it. + const o = await takenOff('continue', { at: 'http://h/uw.js' }); + assert.deepEqual(o.did, [['sent', 'http://h/uw.js'], ['threw', 'route.continue: Route is already handled!']]); + // A Playwright with no answers past its API: keploy answers through the + // route's own, so the route's later call fails at once, and a fallback + // does nothing. + const c = await takenOff('continue', { inner: false }); + assert.deepEqual(c.did, ['fetch', 'seeded', ['threw', 'Route is already handled!']]); + const b = await takenOff('fallback', { inner: false }); + assert.deepEqual(b.did, ['fetch', 'seeded']); + const n = await takenOff('continue', { inner: false, at: 'http://h/uw.js' }); + assert.deepEqual(n.did, ['continue', ['threw', 'Route is already handled!']]); +}); + +test('workerRoutes, when the test takes a route off that began to answer a worker script\'s load (continue, fulfill, abort), or while the page keeps a route of the test\'s, or with unrouteAll\'s \'wait\' from outside the route: that answer goes on as with no keploy, once, and the unroute waits for it', async () => { + for (const how of ['continue', 'fulfill', 'abort']) { + const x = await takenOff(how, { first: true }); + const own = how === 'continue' ? ['fetch', 'seeded'] : [how]; + assert.deepEqual(x.early, own, how); + assert.deepEqual(x.did, own, how); + } + // Another route of the test's stays on the page: Playwright sends + // nothing on, so the route's continue answers it. + const kept = await takenOff('continue', { other: true }); + assert.deepEqual(kept.early, []); + assert.deepEqual(kept.did, ['fetch', 'seeded']); + // unrouteAll's 'wait', from outside the route: it waits for the route, + // whose continue answers the load. + const { context, page, send } = played(); + await workerRoutes(context); + const p = page(); + await turn(); + let go = () => {}; + const going = new Promise((res) => { go = res; }); + let reached = () => {}; + const at = new Promise((res) => { reached = res; }); + await p.route(uwAt, async (route) => { reached(); await going; await route.continue(); }); + const x = send(markWorker('http://h/uw.js', 'w', 'e2'.repeat(16)), p); + await at; + const off = p.unrouteAll({ behavior: 'wait' }); + await turn(); + assert.deepEqual(x.did, []); + go(); + await off; + await x.end; + assert.deepEqual(x.did, ['fetch', 'seeded']); +}); + +test('workerRoutes, when the test takes its context\'s last route off while it holds a worker script\'s load: keploy answers one from a page with no route of the test\'s, or from no page (Playwright sends those on), and leaves one from a page that has one (Playwright sent it through that page) to the route', async () => { + for (const [from, sent] of [['bare', true], ['routed', false], ['none', true]]) { + const { context, page, send } = played(); + await workerRoutes(context); + const p = page(); + await turn(); + if (from === 'routed') await p.route(() => true, (route) => route.fallback()); + let early = null; + const c = async (route) => { + await context.unroute(uwAt, c); + early = x.did.slice(); + await route.continue(); + }; + await context.route(uwAt, c); + const x = send(from === 'none' ? 'http://h/uw.js' : markWorker('http://h/uw.js', 'w', 'e3'.repeat(16)), from === 'none' ? null : p); + await x.end; + assert.deepEqual(early, sent ? (from === 'none' ? [['sent', 'http://h/uw.js']] : ['fetch', 'seeded']) : [], from); + const after = sent ? [['threw', 'route.continue: Route is already handled!']] : []; + assert.deepEqual(x.did, [...(from === 'none' ? [['sent', 'http://h/uw.js']] : ['fetch', 'seeded']), ...after], from); + } +}); + +test('workerRoutes: a request at a worker script\'s URL that keploy handed nothing at reaches the context\'s routes past keploy\'s, unchanged', async () => { + const { r, context } = routed(); + await workerRoutes(context); + const p = r.page(true); + await p.route(() => true, (route) => route.fallback()); + assert.deepEqual(await r.ask(markWorker('http://h/k.js', 'w', 'f2'.repeat(16)), answerOf(200, 'app()'), { type: 'script', headers: {} }, p), [['fetch', 'http://h/k.js'], ['fulfill', { status: 200, headers: { 'content-type': 'text/javascript', 'x-a': '1' }, body: `importScripts(${JSON.stringify(markWorker('http://h/k.js', 'q', 'f2'.repeat(16), 0))});app()\n//# sourceURL=http://h/k.js\n` }]]); + // The app's own fetch at that URL, from another page with no route. + assert.deepEqual(await r.ask('http://h/k.js', null, { type: 'fetch', headers: {} }, r.page(true)), [['fallback']]); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index ebf7b06..faf45f5 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -2492,3 +2492,56 @@ test('seeded: after a test that throws, and after one that times out, the worker assert.deepEqual(begins.map((b) => [b.name, b.random, /^[0-9a-f]{32}$/.test(b.seed)]), [['throws', 'seeded', true], ['times out', 'seeded', true]]); } finally { await rec.close(); } }); + +// A route of the test's own page that takes itself off as it gets a Web +// Worker's script load (unroute, unrouteAll with each behavior), then +// answers it (Playwright sent it on already: the page has no route left), +// and unrouteAll's 'wait' from the test while the route holds the load +// (unroute.spec.ts): seeded, each test fails with Playwright's own error, +// or passes, as with seeding off; the app's server is asked for each +// worker's script once, never with keploy's mark; seeded, the worker of each +// test that passes draws its seeding's numbers. +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a page route that takes itself off (unroute, unrouteAll with each behavior), then continues, fulfills or aborts a Web Worker's script load, fails the test with Playwright's own error, as with seeding off (after 'ignoreErrors' it passes), one that falls back passes, and unrouteAll's 'wait' waits for the route's continue; the app's server is asked for each script once, never with the mark, and each worker runs, seeded`, async () => { + // The tests that pass, whose worker's message is kept. + const passing = ['unroute then fallback', 'unrouteAll ignoreErrors then abort', 'unrouteAll wait then continue']; + const j = journal(); + const rec = await fakeAgent(j); + try { + const runs = {}; + for (const mode of ['seeded', 'off']) { + const d = await drawsServer(); + const from = j.events.length; + try { + const { tests, tmp, out } = await playwright(['unroute.spec.ts'], { ...keployEnv(rec.url), PW_BROWSER: engine, DRAWS_BASE: d.url, ...(mode === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }); + const begins = j.events.slice(from).filter((e) => e.agent && e.agent.path === '/agent/scope/begin').map((e) => e.agent.body); + runs[mode] = { tests, tmp, out, seen: d.seen.slice(), seeds: Object.fromEntries(begins.map((b) => [b.name, b.seed])) }; + } finally { await d.close(); } + } + for (const mode of ['seeded', 'off']) { + const { tests, tmp, out, seen } = runs[mode]; + const c = tests['unroute then continue']; + assert.equal(c.status, 'failed', `${mode}\n${out}`); + assert.match(c.error, /route\.continue: Route is already handled!/, `${mode}: ${c.error}`); + const u = tests['unrouteAll default then fulfill']; + assert.equal(u.status, 'failed', `${mode}\n${out}`); + assert.match(u.error, /route\.fulfill: Route is already handled!/, `${mode}: ${u.error}`); + for (const title of passing) { + assert.equal(tests[title].status, 'passed', `${mode} ${title}: ${tests[title].error}\n${out}`); + } + for (const title of passing) { + const got = JSON.parse(fs.readFileSync(path.join(tmp, `${title}.json`), 'utf8')); + assert.equal(got.kind, 'pw', `${mode} ${title}`); + assert.match(got.href, /^http:\/\/127\.0\.0\.1:\d+\/ov\/pw\.js$/, `${mode} ${title}`); + } + assert.deepEqual(seen.filter((u) => u.startsWith('/ov/')), Array(5).fill('/ov/pw.js'), mode); + assert.deepEqual(seen.filter((u) => u.includes('__keploy_worker')), [], mode); + } + for (const title of passing) { + const [mark] = onceMarks(runs.seeded.seeds[title], [['/ov/pw.js', 'w']]); + const got = JSON.parse(fs.readFileSync(path.join(runs.seeded.tmp, `${title}.json`), 'utf8')); + assert.deepEqual(got.draws, workerAt('http://h/ov/pw.js', workerScript(mark.token, 0, 'w')).draws(), title); + } + } finally { await rec.close(); } + }); +} From 9695ee0dc9afb3e0226ab04b9ad657709baa5d5a Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 09:39:01 +0530 Subject: [PATCH 53/73] fix(playwright): keep a worker strict when its prologue holds U+2028 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit keploy puts the load of a classic Web Worker's seeding first in the app's script, after a 'use strict' of the app's own when its directive prologue has one, so the script stays strict. The prologue scan ended at a line terminator inside a string, but U+2028 and U+2029 may stand in a string literal: a script that starts "a
b"; "use strict"; is strict, and keploy's answer made it sloppy. The scan now ends a string only at its closing quote; a raw line feed or carriage return in one is a syntax error, which the browser refuses whatever comes ahead of it. Rows: - useStrict: a directive after a string holding U+2028 or U+2029 counts; a line comment that ends the source ends the directive's statement. - a Shared Worker's seeding leaves an ErrorEvent filename holding "null" as it is. - a route that calls fulfill, abort, fallback or continue while keploy still answers its continue gets Playwright's error once that answer ended, and the load is answered once; a load the browser dropped while keploy fetched its script is still counted answered, and the route's next call waits no more. - when the test takes its context's last route off, keploy answers a load from a page it has not seen yet, as Playwright sends it on. - a page that shows no list of its routes takes them off with its own unrouteAll, and keploy's route is added again. - settle (Chromium): a fetch at a Web Worker's URL is no worker's load, so the terminated worker's load is the one dropped. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/src/random.ts | 4 ++- playwright/test/random.test.mjs | 58 +++++++++++++++++++++++++++++---- playwright/test/unit.test.mjs | 17 ++++++++++ 3 files changed, 72 insertions(+), 7 deletions(-) diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 3970ac1..1b69fe1 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1854,9 +1854,11 @@ export function useStrict(source: string): boolean { const q = source[s.at]; if (q !== '"' && q !== '\'') return false; let j = s.at + 1; + // A string may hold U+2028 and U+2029; a line feed or carriage return + // in one, unescaped, makes the script a syntax error, which the browser + // refuses whatever comes ahead of it. while (j < n && source[j] !== q) { if (source[j] === '\\') j++; - else if (eol.test(source[j])) return false; j++; } if (j >= n) return false; diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index e69d194..889387a 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -2522,6 +2522,10 @@ test('workerAnswer: a Web Worker\'s is the app\'s script with its seeding\'s loa ['#!/usr/bin/env node\n\'use strict\';', true], ['"use strict" /* c\n */ a()', true], ['"use strict" + a', false], ['a(); "use strict";', false], ['"use\\x20strict";', false], ['"a" + b; "use strict";', false], ['', false], ['"use strict', false], ['"use\nstrict"', false], ['#!/x', false], ['`use strict`;', false], ['// "use strict"', false], ['"x\\"y"; "use strict"', true], + // U+2028 and U+2029 may stand in a string; a directive after one still counts. + ['"a
b"; "use strict"', true], ['\'a
b\'\n\'use strict\'', true], + // A line comment that ends the source ends the directive's statement. + ['"use strict" // c', true], ]) assert.equal(useStrict(src), strict, JSON.stringify(src)); assert.equal(withPrefix('a', 'P;'), 'P;a'); assert.equal(withPrefix('#!x\u2028a', 'P;'), '#!x\u2028P;a'); @@ -2538,7 +2542,7 @@ test('a Web Worker\'s seeding: an ErrorEvent\'s filename at the worker\'s own ma const read = (f) => w.run(`(() => { const e = new ErrorEvent(); e.f = ${JSON.stringify(f)}; return e.filename; })()`); assert.equal(read(href), kind === 's' ? href : 'http://h/w.js?a=1', kind); assert.equal(read(`${href}#h`), kind === 's' ? `${href}#h` : 'http://h/w.js?a=1#h', kind); - for (const f of ['http://h/o.js', markWorker('http://h/w.js?a=1', kind === 'w' ? 'm' : 'w', t), markWorker('http://h/w.js', kind, 'cd'.repeat(16)), 7]) assert.equal(read(f), f, `${kind} ${f}`); + for (const f of ['http://h/o.js', 'http://h/null.js', markWorker('http://h/w.js?a=1', kind === 'w' ? 'm' : 'w', t), markWorker('http://h/w.js', kind, 'cd'.repeat(16)), 7]) assert.equal(read(f), f, `${kind} ${f}`); assert.equal(w.run('(() => { try { return Object.getOwnPropertyDescriptor(ErrorEvent.prototype, \'filename\').get.call({}); } catch (e) { return e.message; } })()'), 'Illegal invocation', kind); } }); @@ -2684,6 +2688,15 @@ test('workerRoutes keeps its route on a page and on the context through the test await q.route('**/w.js', h); await q.unrouteAll(); assert.equal(q.routes.length, 1); + // A page that shows no list of its routes: its own unrouteAll takes them + // off, whatever else it shows, and keploy's is added again. + const bare = r.page(); + bare._unrouteInternal = async () => { throw new Error('not this way'); }; + await turn(); + await bare.route('**/w.js', h); + await bare.unrouteAll(); + assert.equal(bare.routes.length, 1); + assert.equal(bare.routes[0].match(new URL(markWorker('http://h/w.js', 'w', 'f1'.repeat(16)))), true); assert.equal(q.routes[0].match(new URL(markWorker('http://h/w.js', 'w', 'f1'.repeat(16)))), true); const n = c2.routes.length; await c2.unrouteAll(); @@ -2701,7 +2714,8 @@ test('workerRoutes keeps its route on a page and on the context through the test // error; with inner, Playwright's own answers past its API (_innerFulfill, // _innerContinue: sent on) answer it and leave it to the route that holds // it, whose continue, fulfill or abort then fails at the browser's end -// (route.<call>: ...). What the request saw is in did: keploy's fetch, its seeded answer, +// (route.<call>: ...). After drop (the browser dropped the request: a +// worker terminated as its script loads) every answer throws. What the request saw is in did: keploy's fetch, its seeded answer, // another answer, an error a handler threw ('threw', or 'dropped' after // 'ignoreErrors'), a handler that returned with no answer ('unanswered'), // or the URL the browser sent it to when no route answered it ('sent'). @@ -2744,9 +2758,11 @@ function played(inner = true) { let handling = false; let here = false; let answered = false; + let dropped = false; const request = { url: () => at, resourceType: () => 'script', headers: () => ({}), method: () => 'GET', postData: () => null, frame: () => { if (p === null) throw new Error('no frame'); return { page: () => p }; } }; const kind = (o) => (String(o.body).startsWith('importScripts(') ? 'seeded' : 'fulfill'); const serve = (call, what) => { + if (dropped) throw new Error(`route.${call}: Request context disposed.`); if (answered) throw new Error(`route.${call}: Route is already handled!`); answered = true; did.push(what); @@ -2801,7 +2817,7 @@ function played(inner = true) { } if (!answered) did.push(['sent', at]); })(); - return { did, end }; + return { did, end, drop: () => { dropped = true; } }; }; return { context, page, send, log }; } @@ -2898,12 +2914,42 @@ test('workerRoutes, when the test takes a route off that began to answer a worke assert.deepEqual(x.did, ['fetch', 'seeded']); }); -test('workerRoutes, when the test takes its context\'s last route off while it holds a worker script\'s load: keploy answers one from a page with no route of the test\'s, or from no page (Playwright sends those on), and leaves one from a page that has one (Playwright sent it through that page) to the route', async () => { - for (const [from, sent] of [['bare', true], ['routed', false], ['none', true]]) { +test('workerRoutes: a route of the test\'s that calls fulfill, abort, fallback or continue while keploy still answers its continue gets Playwright\'s error once that answer ended, and the load is answered once; one the browser dropped while keploy fetched its script (a worker terminated as it loads) is still counted answered, and the route\'s next call waits no more', async () => { + for (const [how, drop] of [['fulfill', false], ['abort', false], ['fallback', false], ['continue', false], ['abort', true]]) { + const label = `${how}${drop ? ' dropped' : ''}`; + const answered = []; const { context, page, send } = played(); - await workerRoutes(context); + await workerRoutes(context, { answered: (q) => answered.push(q.url()) }); const p = page(); await turn(); + let release = () => {}; + const held = new Promise((res) => { release = res; }); + let first = null; + let second = null; + await p.route(uwAt, async (route) => { + first = route.continue().then(() => 'ok', (e) => e.message); + second = (how === 'fulfill' ? route.fulfill({ body: 'own' }) : route[how]()).then(() => 'ok', (e) => e.message); + await turn(); + if (drop) x.drop(); + release(); + await first; + }); + const x = send(markWorker('http://h/uw.js', 'w', 'e4'.repeat(16)), p, held); + await x.end; + // Before the second call is awaited: it waits on the first's end. + assert.deepEqual(answered, ['http://h/uw.js'], label); + assert.deepEqual(x.did, drop ? ['fetch', 'unanswered'] : ['fetch', 'seeded'], label); + assert.equal(await first, drop ? 'route.fulfill: Request context disposed.' : 'ok', label); + assert.equal(await second, drop ? 'route.abort: Request context disposed.' : 'Route is already handled!', label); + } +}); + +test('workerRoutes, when the test takes its context\'s last route off while it holds a worker script\'s load: keploy answers one from a page with no route of the test\'s (one keploy has not seen yet too), or from no page (Playwright sends those on), and leaves one from a page that has one (Playwright sent it through that page) to the route', async () => { + for (const [from, sent] of [['bare', true], ['routed', false], ['none', true], ['unseen', true]]) { + const { context, page, send } = played(); + await workerRoutes(context); + const p = from === 'unseen' ? { _routes: [] } : page(); + await turn(); if (from === 'routed') await p.route(() => true, (route) => route.fallback()); let early = null; const c = async (route) => { diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 9e49eb9..273f9bf 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -504,6 +504,23 @@ test('settle (Chromium): the script load of a Web Worker its document terminated ctx2.emit('request', load(f, 'http://h/w.js')); off.webWorkerStopped(d2, 'http://h/w.js', 1); assert.equal(off.size, 1); + // A fetch at a Web Worker's URL is no worker's load: the second worker's + // load is the one dropped, and the fetch is still waited for. + const ctx3 = fakeContext(); + const on = new Inflight(ctx3, true); + const p3 = new EventEmitter(); + ctx3.emit('page', p3); + const h = { page: () => p3 }; + const d3 = on.webWorkerDocument(h); + const a1 = load(h, 'http://h/w.js'); + const viaFetch = load(h, 'http://h/w.js', 'fetch'); + const a2 = load(h, 'http://h/w.js'); + for (const r of [a1, viaFetch, a2]) ctx3.emit('request', r); + on.webWorkerStopped(d3, 'http://h/w.js', 2); + assert.equal(on.size, 2); + ctx3.emit('requestfinished', viaFetch); + ctx3.emit('requestfinished', a1); + assert.equal(on.size, 0); }); test('settle (Chromium): a commit with no answer (a route change inside the page) keeps the document, so its word still drops the load of a Web Worker it terminated; a load at a URL keploy marked for its own route is waited for though its place was terminated before it was reported (keploy\'s route ends it)', async () => { From a0c6060fd8023aa40022c49aaaddc8e354ed95e6 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 10:55:40 +0530 Subject: [PATCH 54/73] fix(playwright): take a seeded test's context offline before it closes Firefox and WebKit send a request that a page or a worker starts while its browser context closes to the network past every route, keploy's too. A Web Worker's script load that a test's last step started, or the load of the seeding keploy's answer makes when the test ends just after keploy answered the worker's script, then reached the app's server with keploy's __keploy_worker mark in its URL. keploy now takes a seeded test's context offline before it closes, whoever closes it: after keploy's end-of-test wait; as the test closes it itself; or as its browser closes (once a test's timeout cuts keploy's wait short, Playwright skips the context's own close, and the context closes with the browser as the worker ends). Such a request then fails in the browser instead. With seeding off it would reach the server, unmarked; no test step reads what it gets, and it is the same at record and at replay. A context that cannot go offline (its setOffline throws, or there is none) closes anyway, and the test prints one line naming it and saying that a request started as its context closes can reach the server with the mark. A context Playwright reuses for the next test (reuseContext, or PW_TEST_REUSE_CONTEXT, which VS Code's Show browser sets) is not closed, and Playwright's reset of it for the next test needs the network (Firefox clears its storage by loading its origin; in Chromium an offline error page cuts its navigation to about:blank), so keploy leaves it online. The run says once, at its first seeded test there, that a request its pages or workers start after a test ends can reach the server with the mark, and that running without reuse avoids it. Rows: - in every engine, seeded: a test whose last step starts a Web Worker (classic, module, from a timer) or a Shared Worker and returns, one whose page route takes itself off and then fulfills a Web Worker's script load, and one that closes its own context as a Web Worker starts, 20 runs of each with four runs at once: the server never sees the mark (before, of 32 runs of each, it did in up to 10 in Firefox and 8 in WebKit). - a test whose context's setOffline throws, or that has none, passes and says so once, seeded; with seeding off it says nothing. A test that puts back the context's own close, past keploy's, finds the context offline as Playwright closes it, seeded (keploy's call after its wait), and online with seeding off. - in every engine, in a reused context: keploy leaves it online, so three tests in it each reach the app's page online, as with seeding off, and the run says so once. - in every engine, a test whose timeout cuts keploy's wait short prints no line about going offline (the sandboxed-frame rows' exact lines): its context goes offline as its browser closes (before, it closed with the browser ahead of keploy's setOffline, which then failed in Firefox). - offlineBeforeClose: offline once, before the close, whichever comes first; a close during setOffline waits for it; before a browser closes, each seeded context it still has open goes offline, and not one that closed or that Playwright handed on for reuse. reusedContext: Playwright's own flag, and without one PW_TEST_REUSE_CONTEXT. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 4 +- playwright/src/index.ts | 40 ++++++-- playwright/src/settle.ts | 117 ++++++++++++++++++++++ playwright/test/fixtures/offline.spec.ts | 44 ++++++++ playwright/test/fixtures/teardown.spec.ts | 57 +++++++++++ playwright/test/rows.test.mjs | 115 ++++++++++++++++++++- playwright/test/unit.test.mjs | 110 +++++++++++++++++++- 7 files changed, 475 insertions(+), 12 deletions(-) create mode 100644 playwright/test/fixtures/offline.spec.ts create mode 100644 playwright/test/fixtures/teardown.spec.ts diff --git a/playwright/README.md b/playwright/README.md index 81cc3a9..66dfa0e 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither, as the test's browser context closes too (below). Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -100,6 +100,8 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). - **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. - **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. +- **... a request starts as a seeded test's browser context closes.** After a seeded test's end-of-test wait, keploy takes its browser context offline (`context.setOffline(true)`) before Playwright closes it, so a request the browser starts during close fails instead of reaching the app's server; with seeding off it would reach the server, unmarked. Firefox and WebKit send such a request (a Web Worker's script load a test's last step started, or the load of the seeding keploy's answer makes) to the network past every route, keploy's too, so it would carry keploy's `__keploy_worker` mark to the server. No test step reads what it gets, and it is the same at record and at replay. keploy takes it offline before it closes, whoever closes it: after keploy's wait; as the test closes it itself; or as its browser closes (once a test's timeout cuts keploy's wait short, Playwright skips the context's own close and the context closes with the browser as the worker ends). If the context cannot go offline (its `setOffline` throws, or there is none), it closes anyway, and the test prints one line naming it and saying that a request started as its context closes can reach the app's server with keploy's mark. Next step: have the test wait for what its last step starts (a message a worker posts, or what the page shows from it) before it ends. +- **... Playwright reuses a seeded test's browser context for the next test.** With `reuseContext: true`, or `PW_TEST_REUSE_CONTEXT` (which VS Code's Playwright extension sets for Show browser), Playwright does not close the context at a test's end, and its reset of it for the next test needs the network (Firefox clears the context's storage by loading its origin), so keploy does not take it offline. A request its pages or workers start after a test ends (a Web Worker's script load, or the load of its seeding) can then reach the app's server with keploy's `__keploy_worker` mark in its URL. The run prints one line, at its first seeded test in such a context, saying so. Next step: run keploy's tests without reuse (`reuseContext: false`, `PW_TEST_REUSE_CONTEXT` unset), or have each test wait for what its last step starts before it ends. - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: diff --git a/playwright/src/index.ts b/playwright/src/index.ts index ccbd879..5d07a35 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -30,6 +30,11 @@ // /agent/refusal), since keploy runs each test once in a run; // - before the test's browser context closes, the test waits until none of // its pages' calls is in flight (settle), so its calls end inside it; +// a seeded test's context then goes offline before it closes, whoever +// closes it (offlineBeforeClose), so a request the browser starts as it +// closes fails instead of reaching the app's server; a context +// Playwright reuses for the next test is not closed and stays online +// (reusedContext), and the worker says so once (reusedLine); // - the test ends (POST /agent/scope/end) after its last teardown, with // its outcome and what its context's pages drew their random numbers // from (random); when keploy saw a call with no recorded answer, or a @@ -67,7 +72,7 @@ import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; -import { Inflight, sharedWorkerLine, workersScript } from './settle'; +import { Inflight, offlineBeforeClose, reusedContext, reusedLine, sharedWorkerLine, workersScript } from './settle'; export { expect, KeployError }; export { seedFaker } from './random'; @@ -98,6 +103,9 @@ const saidFaker = new Set<string>(); // How many contexts this worker seeded: each one's report function has its // own name. let reports = 0; +// Whether this worker said that a seeded test's browser context Playwright +// reuses for the next test is not taken offline (reusedLine). +let saidReused = false; // How many contexts of a keploy test this worker opened: each one's Shared // Worker report function has its own name. let sharedReports = 0; @@ -288,19 +296,33 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ }); await context.addInitScript({ content: workersScript(sharedReport) }); const seeded = s.seed !== ''; + // Seeded: the context goes offline before it closes, whoever closes it, + // so nothing keploy marked reaches the app's server as it closes. + const offline = seeded ? offlineBeforeClose(context, name, (line) => { process.stdout.write(`${line}\n`); }) : null; const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; if (context.pages().length > 0) opened(); context.on('page', opened); try { - await use(context); + try { + await use(context); + } finally { + restore(); + for (const say of waiting) say(); + } + if (inflight.size > 0) { + const calls = inflight.describe(); + const shown = calls.slice(0, 5).join(', ') + (calls.length > 5 ? `, and ${calls.length - 5} more` : ''); + await base.step(`keploy: wait for the test's ${calls.length} call(s) in flight before its browser context closes: ${shown}`, () => inflight.settled()); + } } finally { - restore(); - for (const say of waiting) say(); - } - if (inflight.size > 0) { - const calls = inflight.describe(); - const shown = calls.slice(0, 5).join(', ') + (calls.length > 5 ? `, and ${calls.length - 5} more` : ''); - await base.step(`keploy: wait for the test's ${calls.length} call(s) in flight before its browser context closes: ${shown}`, () => inflight.settled()); + // A context Playwright reuses for the next test is not closed, and its + // reset for that test needs the network: it stays online. + if (offline !== null && !reusedContext(context, process.env)) { + await offline(); + } else if (offline !== null && !saidReused) { + saidReused = true; + process.stdout.write(`${reusedLine(name)}\n`); + } } }, diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index b8b2982..1b3d2ac 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -368,3 +368,120 @@ export function sharedWorkerLine(name: string, at: string): string { '(Playwright reports none of a Shared Worker\'s calls), so a call it still has in flight when the test ends is cut there; ' + 'have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends.'; } + +/** + * The line a seeded test shows when keploy could not take its browser + * context offline before Playwright closes it (closeOffline): why says what + * setOffline threw. + */ +export function offlineLine(name: string, why: string): string { + return `keploy: in ${name}, keploy could not take the test's browser context offline before it closes (${why}), ` + + 'so a request the browser starts as the context closes (a Web Worker\'s script load, or the load of its seeding) can reach the app\'s server ' + + 'with keploy\'s __keploy_worker mark in its URL; have the test wait for what its last step starts (a message a worker posts, or what the page shows from it) before it ends.'; +} + +/** + * Takes a seeded test's browser context offline once keploy's end-of-test + * wait is done, before the context closes (offlineBeforeClose). Firefox and WebKit + * send a request that a page or a worker starts while its context closes to + * the network past every route, keploy's too, so a worker's script load, or + * the load of its seeding, that keploy marked would reach the app's server + * with the mark in its URL; offline, the browser fails it instead. No test + * step reads what such a request gets, and it is the same at record and at + * replay. A context that cannot go offline (setOffline throws: the context + * or its browser is gone, or this Playwright has none) closes anyway, and + * the test says so through say. + */ +export async function closeOffline(context: { setOffline(offline: boolean): Promise<void> }, name: string, say: (line: string) => void): Promise<void> { + try { + await context.setOffline(true); + } catch (e) { + say(offlineLine(name, (e instanceof Error ? e.message : String(e)).split('\n')[0])); + } +} + +/** A browser as offlineBeforeClose needs it. */ +export interface ClosingBrowser { + close(options?: { reason?: string }): Promise<void>; +} + +/** A browser context as offlineBeforeClose needs it. */ +export interface Closing { + setOffline(offline: boolean): Promise<void>; + close(options?: { reason?: string }): Promise<void>; + browser(): ClosingBrowser | null; + on(event: 'close', listener: () => void): unknown; +} + +// For each browser, the seeded contexts it has open: each goes offline +// before the browser closes. +const openOf = new WeakMap<ClosingBrowser, Set<() => Promise<void>>>(); + +/** + * Has a seeded test's browser context go offline (closeOffline) before it + * closes, whoever closes it: keploy's fixture once its end-of-test wait is + * done (the function this returns), the test's own context.close(), or its + * browser's close. Playwright skips a context's own teardown once the test's + * time is spent, so when a test's timeout cut keploy's wait short the + * context closes with its browser, as the worker ends. The context goes + * offline once; a later call or close finds that done. A context that closed + * (or that Playwright handed on for reuse) is no longer its browser's to + * take offline. + */ +export function offlineBeforeClose(context: Closing, name: string, say: (line: string) => void): () => Promise<void> { + let done: Promise<void> | null = null; + const now = async (): Promise<void> => { + done ??= closeOffline(context, name, say); + await done; + }; + const close = context.close; + context.close = async (options?: { reason?: string }): Promise<void> => { + await now(); + return close.call(context, options); + }; + const browser = context.browser(); + if (browser !== null) { + let open = openOf.get(browser); + if (open === undefined) { + const all = new Set<() => Promise<void>>(); + open = all; + openOf.set(browser, all); + const closeBrowser = browser.close; + browser.close = async (options?: { reason?: string }): Promise<void> => { + await Promise.all([...all].map((f) => f())); + return closeBrowser.call(browser, options); + }; + } + const mine = open; + mine.add(now); + context.on('close', () => { mine.delete(now); }); + } + return now; +} + +/** + * Whether Playwright keeps this browser context open for the next test + * instead of closing it at the test's end (its reuseContext option, or + * PW_TEST_REUSE_CONTEXT, which VS Code's Playwright extension sets for Show + * browser). Playwright marks the context it hands out for reuse + * (_forReuse); a Playwright without that flag is taken as reusing it when + * PW_TEST_REUSE_CONTEXT is set. + */ +export function reusedContext(context: object, env: Record<string, string | undefined>): boolean { + const f = (context as { _forReuse?: unknown })._forReuse; + if (typeof f === 'boolean') return f; + return (env.PW_TEST_REUSE_CONTEXT ?? '') !== ''; +} + +/** + * The line a worker shows once, at its first seeded test in a browser + * context Playwright reuses: keploy does not take such a context offline, + * since Playwright's reset of it for the next test needs the network. + */ +export function reusedLine(name: string): string { + return `keploy: in ${name}, Playwright keeps the test's browser context open for the next test (its reuseContext option, or PW_TEST_REUSE_CONTEXT, which VS Code's Show browser sets), ` + + 'so keploy does not take it offline as the test ends (Playwright\'s reset of it for the next test needs the network), ' + + 'and a request its pages or workers start after a test ends (a Web Worker\'s script load, or the load of its seeding) can reach the app\'s server ' + + 'with keploy\'s __keploy_worker mark in its URL; run keploy\'s tests without reuse (reuseContext: false, PW_TEST_REUSE_CONTEXT unset), ' + + 'or have each test wait for what its last step starts (a message a worker posts, or what the page shows from it) before it ends.'; +} diff --git a/playwright/test/fixtures/offline.spec.ts b/playwright/test/fixtures/offline.spec.ts new file mode 100644 index 0000000..588a45b --- /dev/null +++ b/playwright/test/fixtures/offline.spec.ts @@ -0,0 +1,44 @@ +import type { BrowserContext, Page } from '@playwright/test'; +import fs from 'node:fs'; +import { test } from '../../dist'; + +// Run seeded and with seeding off. The context of a test whose +// setOffline throws, or that has none, cannot go offline before it closes. +// A test that puts back the context's own close, past keploy's, keeps +// whether its page reaches the app's server as Playwright closes it. +// The 'reused' tests run with PW_TEST_REUSE_CONTEXT=1 alone (-g reused): +// Playwright then keeps one browser context for the tests that ask the same +// options and does not close it between them. Each such test keeps whether +// its page loads the app's page (online or offline) and its context's id +// under its title. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); +const reach = (page: Page): Promise<string> => page.goto(`${base}/ov`).then(() => 'online', () => 'offline'); +const id = (context: BrowserContext): unknown => (context as unknown as { _guid?: string })._guid; + +test.use({ serviceWorkers: 'block' }); + +test('a context whose setOffline throws', async ({ context, page }) => { + await page.goto(`${base}/ov`); + (context as unknown as { setOffline: () => Promise<void> }).setOffline = async () => { throw new Error('no offline here\nand a second line'); }; +}); + +test('a context with no setOffline', async ({ context, page }) => { + await page.goto(`${base}/ov`); + (context as unknown as { setOffline?: unknown }).setOffline = undefined; +}); + +test('a context whose close keploy did not wrap', async ({ context, page }) => { + await page.goto(`${base}/ov`); + const close = (Object.getPrototypeOf(context) as { close: (o?: { reason?: string }) => Promise<void> }).close; + context.close = async function (this: BrowserContext, o?: { reason?: string }): Promise<void> { + keep('at close', await page.evaluate(() => fetch('/ov?at-close').then(() => 'online', () => 'offline'))); + return close.call(this, o); + }; +}); + +test.describe('reused', () => { + test('online one', async ({ context, page }) => { keep(test.info().title, [await reach(page), id(context)]); }); + test('online two', async ({ context, page }) => { keep(test.info().title, [await reach(page), id(context)]); }); + test('online three', async ({ context, page }) => { keep(test.info().title, [await reach(page), id(context)]); }); +}); diff --git a/playwright/test/fixtures/teardown.spec.ts b/playwright/test/fixtures/teardown.spec.ts new file mode 100644 index 0000000..9a9835a --- /dev/null +++ b/playwright/test/fixtures/teardown.spec.ts @@ -0,0 +1,57 @@ +import type { Route } from '@playwright/test'; +import { test } from '../../dist'; + +// Run seeded in Chromium, Firefox and WebKit (PW_BROWSER), each test +// TEARDOWN_COPIES times (default 1), with several runs at once: a test whose +// last step starts a worker and returns, and one whose page route takes +// itself off and then fulfills a Web Worker's script load, so the test ends +// and its browser context closes while the worker's script load, or the load +// of the seeding keploy's answer makes, is on its way; and one that closes +// its own browser context as a worker starts. Firefox and WebKit +// send a request started as the context closes past every route; the app's +// server must never see keploy's mark. +const base = process.env.DRAWS_BASE ?? ''; +const copies = Number(process.env.TEARDOWN_COPIES || 1); + +test.use({ serviceWorkers: 'block' }); + +for (let i = 0; i < copies; i++) { + test(`classic worker as the test ends ${i}`, async ({ page }) => { + await page.goto(`${base}/ov`); + await page.evaluate(() => { new Worker('/ov/pw.js'); }); + }); + + test(`module worker as the test ends ${i}`, async ({ page }) => { + await page.goto(`${base}/ov`); + await page.evaluate(() => { new Worker('/ov/pw.js', { type: 'module' }); }); + }); + + test(`classic worker started in a timer as the test ends ${i}`, async ({ page }) => { + await page.goto(`${base}/ov`); + await page.evaluate(() => { setTimeout(() => { new Worker('/ov/pw.js'); }, 0); }); + }); + + test(`classic worker as the test closes its context ${i}`, async ({ context, page }) => { + await page.goto(`${base}/ov`); + await page.evaluate(() => { new Worker('/ov/pw.js'); }); + await context.close(); + }); + + test(`shared worker as the test ends ${i}`, async ({ page }) => { + await page.goto(`${base}/ov`); + await page.evaluate(() => { new SharedWorker('/ov/ps.js'); }); + }); + + test(`unrouteAll default then fulfill ${i}`, async ({ page }) => { + await page.goto(`${base}/ov`); + await page.route('**/ov/pw.js', async (r: Route): Promise<void> => { + await page.unrouteAll({ behavior: 'default' }); + await r.fulfill({ contentType: 'text/javascript', body: 'postMessage("own")' }); + }); + await page.evaluate(() => new Promise((resolve) => { + const w = new Worker('/ov/pw.js'); + w.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + })); + }); +} diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index faf45f5..c87092c 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -11,7 +11,7 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, markWorker, workerMark, workerScript } from '../dist/random.js'; -import { sharedWorkerLine } from '../dist/settle.js'; +import { offlineLine, reusedLine, sharedWorkerLine } from '../dist/settle.js'; import { NO_NOTE_LINE, readNote, unreadNoteLine } from '../dist/note.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; @@ -2545,3 +2545,116 @@ for (const engine of ENGINES) { } finally { await rec.close(); } }); } + +// teardown.spec.ts, seeded, each test 5 times in each of four runs at once +// (the load the race needs: before keploy took the context offline, of 32 +// runs each, the server saw the mark in up to 10 in Firefox and 8 in WebKit): +// a test whose last step starts a Web Worker (classic, module, or from a +// timer) or a Shared Worker and returns, and one whose page route takes +// itself off and then fulfills a Web Worker's script load, end while what +// keploy marked is on its way; one closes its own context as a Web Worker +// starts. keploy takes the context offline once its end-of-test wait is +// done, or as the test closes it, before it closes, so the app's server +// never sees keploy's mark; each test ends as it does with seeding off. +for (const engine of ENGINES) { + test(`seeded, in ${engine}: a test whose last step starts a Web Worker (classic, module, from a timer) or a Shared Worker and returns, one whose page route takes itself off and then fulfills a Web Worker's script load, and one that closes its own browser context as a Web Worker starts, end while what keploy marked is on its way; keploy takes the browser context offline before it closes, so over 20 runs of each, four runs at once, the app's server never sees keploy's mark`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + try { + const runs = await Promise.all([0, 1, 2, 3].map(async () => { + const d = await drawsServer(); + try { + const r = await playwright(['teardown.spec.ts'], { ...keployEnv(rec.url), PW_BROWSER: engine, DRAWS_BASE: d.url, TEARDOWN_COPIES: '5' }); + return { ...r, seen: d.seen.slice() }; + } finally { await d.close(); } + })); + for (const { tests, out, seen } of runs) { + assert.equal(Object.keys(tests).length, 30, out); + for (const [title, t] of Object.entries(tests)) { + if (title.startsWith('unrouteAll default then fulfill ')) { + assert.equal(t.status, 'failed', `${title}\n${out}`); + assert.match(t.error, /route\.fulfill: Route is already handled!/, `${title}: ${t.error}`); + } else { + assert.equal(t.status, 'passed', `${title}: ${t.error}\n${out}`); + } + } + assert.deepEqual(seen.filter((u) => u.includes('__keploy_worker')), []); + assert.doesNotMatch(out, /could not take the test's browser context offline/); + assert.doesNotMatch(out, /Playwright keeps the test's browser context open/); + } + } finally { await rec.close(); } + }); +} + +// offline.spec.ts: a seeded test whose context's setOffline throws, or that +// has none, passes, its context closes, and it says once that a request +// started as its context closes can reach the app's server with keploy's +// mark (the first line of what setOffline threw); with seeding off keploy +// never takes a context offline, so neither says anything. A test that puts +// back the context's own close, past keploy's, finds its context offline as +// Playwright closes it (keploy took it offline after its wait), or online +// with seeding off. +test('seeded: a test whose browser context cannot go offline (its setOffline throws, or there is none) closes it anyway and says so once; with seeding off nothing is said; a context closed past keploy\'s close is offline as it closes, seeded, and online with seeding off', async () => { + const j = journal(); + const rec = await fakeAgent(j); + try { + for (const mode of ['seeded', 'off']) { + const d = await drawsServer(); + try { + const { tests, tmp, out } = await playwright(['offline.spec.ts', '-g', 'a context'], { ...keployEnv(rec.url), DRAWS_BASE: d.url, ...(mode === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }); + const why = { 'a context whose setOffline throws': 'no offline here', 'a context with no setOffline': 'context.setOffline is not a function' }; + const past = 'a context whose close keploy did not wrap'; + assert.deepEqual(Object.keys(tests).sort(), [...Object.keys(why), past].sort(), out); + assert.equal(tests[past].status, 'passed', `${mode} ${past}: ${tests[past].error}\n${out}`); + assert.equal(JSON.parse(fs.readFileSync(path.join(tmp, 'at close.json'), 'utf8')), mode === 'seeded' ? 'offline' : 'online', mode); + assert.doesNotMatch(tests[past].stdout, /could not take the test's browser context offline/, mode); + for (const [title, w] of Object.entries(why)) { + const t = tests[title]; + assert.equal(t.status, 'passed', `${mode} ${title}: ${t.error}\n${out}`); + const said = t.stdout.split('\n').filter((l) => l.includes('could not take the test\'s browser context offline')); + assert.deepEqual(said, mode === 'seeded' ? [offlineLine(title, w)] : [], `${mode} ${title}`); + } + assert.doesNotMatch(out, /Playwright keeps the test's browser context open/, mode); + } finally { await d.close(); } + } + } finally { await rec.close(); } +}); + +// offline.spec.ts's 'reused' tests under PW_TEST_REUSE_CONTEXT=1: Playwright +// keeps one context for the tests that ask the same options and does not +// close it between them, and resets it for the next test, which needs the +// network (Firefox clears its storage by loading its origin). keploy never +// takes such a context offline: three tests in one context each reach the +// app's page online, as with seeding off, and the run says once, at its +// first seeded test, that the context stays online and what a request after +// a test's end can then carry (one worker runs them all). +for (const engine of ENGINES) { + test(`seeded, in ${engine}: in a browser context Playwright reuses, keploy leaves the context online, so a test after a seeded one is online, as with seeding off, and the run says once that the reused context stays online`, async () => { + const j = journal(); + const rec = await fakeAgent(j); + try { + for (const mode of ['seeded', 'off']) { + const d = await drawsServer(); + try { + const { tests, tmp, out } = await playwright(['offline.spec.ts', '-g', 'reused'], { ...keployEnv(rec.url), PW_BROWSER: engine, PW_TEST_REUSE_CONTEXT: '1', DRAWS_BASE: d.url, ...(mode === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }); + const want = { 'online one': 'online', 'online two': 'online', 'online three': 'online' }; + const got = {}; + for (const [title, w] of Object.entries(want)) { + const key = Object.keys(tests).find((k) => k.endsWith(` › ${title}`)); + assert.ok(key !== undefined, `${mode} ${title}\n${out}`); + assert.equal(tests[key].status, 'passed', `${mode} ${title}: ${tests[key].error}\n${out}`); + got[title] = JSON.parse(fs.readFileSync(path.join(tmp, `${title}.json`), 'utf8')); + assert.equal(got[title][0], w, `${mode} ${title}`); + } + // The premise: they ran in one context. + assert.equal(got['online two'][1], got['online one'][1], mode); + assert.equal(got['online three'][1], got['online one'][1], mode); + assert.doesNotMatch(out, /could not take the test's browser context offline/); + const first = Object.keys(tests).find((k) => k.endsWith(' › online one')); + const said = Object.entries(tests).flatMap(([k, t]) => t.stdout.split('\n').filter((l) => l.includes('Playwright keeps the test\'s browser context open')).map((l) => [k, l])); + assert.deepEqual(said, mode === 'seeded' ? [[first, reusedLine(first)]] : [], `${mode}\n${out}`); + } finally { await d.close(); } + } + } finally { await rec.close(); } + }); +} diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 273f9bf..4a8f95f 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -8,7 +8,7 @@ import os from 'node:os'; import path from 'node:path'; import { AgentClient, KeployError } from '../dist/agent.js'; import { decide } from '../dist/join.js'; -import { Inflight, workersScript } from '../dist/settle.js'; +import { Inflight, closeOffline, offlineBeforeClose, offlineLine, reusedContext, reusedLine, workersScript } from '../dist/settle.js'; import { markWorker } from '../dist/random.js'; import { EventEmitter } from 'node:events'; import vm from 'node:vm'; @@ -728,3 +728,111 @@ test('keployText: keploy\'s reason once prefixed, or the fallback when it gave n // Only the whole word with its separator counts as keploy's prefix. assert.equal(keployText('keploy:x', 'fb'), 'keploy: keploy:x'); }); + +test('closeOffline takes the context offline and says nothing; when setOffline throws, or there is none, it says once why, the first line of what was thrown', async () => { + const said = []; + const say = (l) => { said.push(l); }; + const asked = []; + await closeOffline({ setOffline: async (o) => { asked.push(o); } }, 't', say); + assert.deepEqual([asked, said], [[true], []]); + await closeOffline({ setOffline: async () => { throw new Error('gone\ncall log'); } }, 't', say); + await closeOffline({ setOffline: async () => { throw 'a string'; } }, 'u', say); + await closeOffline({}, 'v', say); + assert.deepEqual(said, [offlineLine('t', 'gone'), offlineLine('u', 'a string'), offlineLine('v', 'context.setOffline is not a function')]); + assert.match(said[0], /^keploy: in t, keploy could not take the test's browser context offline before it closes \(gone\), so a request .* __keploy_worker mark/); +}); + +test('offlineBeforeClose: the context goes offline once, before it closes, whether keploy\'s wait ends first, Playwright closes it first (a test timeout), or the test closes it; close gets its options; a setOffline that throws says so once and the context closes anyway', async () => { + const fake = (setOffline) => { + const log = []; + const c = { setOffline: async (o) => { log.push(`offline ${o}`); await setOffline(); }, close: async function (opts) { log.push(['close', this === c, opts]); }, browser: () => null, on: () => c }; + return { c, log }; + }; + const said = []; + const say = (l) => { said.push(l); }; + // keploy's wait ends first, then Playwright closes it. + let { c, log } = fake(async () => {}); + let now = offlineBeforeClose(c, 't', say); + assert.deepEqual(log, []); + await now(); + await c.close({ reason: 'Test ended.' }); + await now(); + assert.deepEqual(log, ['offline true', ['close', true, { reason: 'Test ended.' }]]); + // Playwright closes it first (keploy's wait cut short), or the test does, + // twice; keploy's call after finds it done. + ({ c, log } = fake(async () => {})); + now = offlineBeforeClose(c, 't', say); + await c.close({ reason: 'Test timeout of 30000ms exceeded.' }); + await c.close(); + await now(); + assert.deepEqual(log, ['offline true', ['close', true, { reason: 'Test timeout of 30000ms exceeded.' }], ['close', true, undefined]]); + // A close while setOffline is under way waits for it. + let open; + ({ c, log } = fake(() => new Promise((f) => { open = f; }))); + now = offlineBeforeClose(c, 't', say); + const waited = now(); + const closing = c.close(); + await new Promise((f) => setImmediate(f)); + assert.deepEqual(log, ['offline true']); + open(); + await Promise.all([waited, closing]); + assert.deepEqual(log, ['offline true', ['close', true, undefined]]); + assert.deepEqual(said, []); + // setOffline throws: one line, and the context still closes. + ({ c, log } = fake(async () => { throw new Error('closed\ncall log'); })); + now = offlineBeforeClose(c, 'u', say); + await now(); + await c.close(); + assert.deepEqual([log, said], [['offline true', ['close', true, undefined]], [offlineLine('u', 'closed')]]); +}); + +test('reusedContext: Playwright\'s own _forReuse flag when the context has one; without it, whether PW_TEST_REUSE_CONTEXT is set; reusedLine names the test, why the context stays online and the next step', () => { + assert.equal(reusedContext({ _forReuse: true }, {}), true); + assert.equal(reusedContext({ _forReuse: false }, { PW_TEST_REUSE_CONTEXT: '1' }), false); + assert.equal(reusedContext({}, { PW_TEST_REUSE_CONTEXT: '1' }), true); + assert.equal(reusedContext({}, { PW_TEST_REUSE_CONTEXT: '' }), false); + assert.equal(reusedContext({}, {}), false); + assert.equal(reusedContext({ _forReuse: 'yes' }, { PW_TEST_REUSE_CONTEXT: '1' }), true); + assert.match(reusedLine('t'), /^keploy: in t, Playwright keeps the test's browser context open for the next test .*does not take it offline .* __keploy_worker mark in its URL; run keploy's tests without reuse \(reuseContext: false, PW_TEST_REUSE_CONTEXT unset\)/); +}); + +test('offlineBeforeClose: before a browser closes, each seeded context it has open goes offline (a test timeout: Playwright then closes the browser, not the context), and the browser closes once they have; a context that closed, or that Playwright handed on for reuse, is not asked; the browser\'s close is wrapped once', async () => { + const said = []; + const say = (l) => { said.push(l); }; + const log = []; + const b = { close: async function (opts) { log.push(['browser close', this === b, opts]); } }; + const fake = (n, setOffline = async () => {}) => { + const on = new Map(); + const c = { + setOffline: async (o) => { log.push(`${n} offline ${o}`); await setOffline(); }, + close: async () => { log.push(`${n} close`); on.get('close')?.(); }, + browser: () => b, + on: (e, f) => { on.set(e, f); return c; }, + emit: (e) => on.get(e)?.(), + }; + return c; + }; + let open; + const c1 = fake('c1', () => new Promise((f) => { open = f; })); + const c2 = fake('c2'); + const c3 = fake('c3'); + const c4 = fake('c4'); + offlineBeforeClose(c1, 't1', say); + const wrapped = b.close; + offlineBeforeClose(c2, 't2', say); + offlineBeforeClose(c3, 't3', say); + offlineBeforeClose(c4, 't4', say); + assert.equal(b.close, wrapped); + // c3 closed through keploy (offline first); c4 was handed on for reuse + // (Playwright's disconnect says close, with no close call). + await c3.close(); + c4.emit('close'); + log.length = 0; + const closing = b.close({ reason: 'Test ended.' }); + await new Promise((f) => setImmediate(f)); + assert.deepEqual(log, ['c1 offline true', 'c2 offline true']); + open(); + await closing; + assert.deepEqual(log, ['c1 offline true', 'c2 offline true', ['browser close', true, { reason: 'Test ended.' }]]); + assert.deepEqual(said, []); +}); From 6b8844570fbe66e0cace2faba2cb69bdc20ac309 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 13:39:56 +0530 Subject: [PATCH 55/73] revert(playwright): close a seeded test's context as Playwright does This reverts a0c6060 (take a seeded test's context offline before it closes). context.setOffline(true) is answered only once each page's renderer has handled it. A page with a dialog left open, or a main thread busy as the test ends, never does in Chromium and WebKit: the test then times out in teardown, and keploy's wait in its browser.close kept the whole run from ending. It also told the app it was offline before Playwright took the failure snapshot, so a failed test's error context showed the app's offline state, in every engine. Both are worse than what it closed. The close is again Playwright's own. The README's "What the run says when" says what that leaves: a request the browser starts while Playwright closes a seeded test's context, after the test has ended, can reach the app's server with keploy's mark in its URL in Firefox and WebKit, with the rates measured on a test whose last step starts a worker and returns. No step of the test reads its answer. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 5 +- playwright/src/index.ts | 40 ++------ playwright/src/settle.ts | 117 ---------------------- playwright/test/fixtures/offline.spec.ts | 44 -------- playwright/test/fixtures/teardown.spec.ts | 57 ----------- playwright/test/rows.test.mjs | 115 +-------------------- playwright/test/unit.test.mjs | 110 +------------------- 7 files changed, 13 insertions(+), 475 deletions(-) delete mode 100644 playwright/test/fixtures/offline.spec.ts delete mode 100644 playwright/test/fixtures/teardown.spec.ts diff --git a/playwright/README.md b/playwright/README.md index 66dfa0e..05557df 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither, as the test's browser context closes too (below). Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -100,10 +100,9 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). - **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. - **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. -- **... a request starts as a seeded test's browser context closes.** After a seeded test's end-of-test wait, keploy takes its browser context offline (`context.setOffline(true)`) before Playwright closes it, so a request the browser starts during close fails instead of reaching the app's server; with seeding off it would reach the server, unmarked. Firefox and WebKit send such a request (a Web Worker's script load a test's last step started, or the load of the seeding keploy's answer makes) to the network past every route, keploy's too, so it would carry keploy's `__keploy_worker` mark to the server. No test step reads what it gets, and it is the same at record and at replay. keploy takes it offline before it closes, whoever closes it: after keploy's wait; as the test closes it itself; or as its browser closes (once a test's timeout cuts keploy's wait short, Playwright skips the context's own close and the context closes with the browser as the worker ends). If the context cannot go offline (its `setOffline` throws, or there is none), it closes anyway, and the test prints one line naming it and saying that a request started as its context closes can reach the app's server with keploy's mark. Next step: have the test wait for what its last step starts (a message a worker posts, or what the page shows from it) before it ends. -- **... Playwright reuses a seeded test's browser context for the next test.** With `reuseContext: true`, or `PW_TEST_REUSE_CONTEXT` (which VS Code's Playwright extension sets for Show browser), Playwright does not close the context at a test's end, and its reset of it for the next test needs the network (Firefox clears the context's storage by loading its origin), so keploy does not take it offline. A request its pages or workers start after a test ends (a Web Worker's script load, or the load of its seeding) can then reach the app's server with keploy's `__keploy_worker` mark in its URL. The run prints one line, at its first seeded test in such a context, saying so. Next step: run keploy's tests without reuse (`reuseContext: false`, `PW_TEST_REUSE_CONTEXT` unset), or have each test wait for what its last step starts before it ends. - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. +- **... the browser starts a request while Playwright closes a seeded test's context.** Once the test has ended (its steps, settle and its teardown), Playwright closes its context, and keploy's route goes with it. A request the browser starts in that moment, after the test has ended, can reach the app's server with keploy's `__keploy_worker` mark in its URL, in Firefox and WebKit: a worker the test's last step started, whose script the browser asks for as the context closes. No step of the test reads what such a request gets, and nothing is said. Measured on a test whose last step starts a worker and returns, 32 runs of each with four at once: in Firefox, 9 or 10 of 32 for a Web Worker (classic, module, or one a timer starts) and none for a Shared Worker; in WebKit, 1 to 8 of 32; in Chromium, none; for a test that waits for its worker's message, none in any engine. Next step: have the test wait for what the worker it starts last does before it ends. - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: - a popup opened with noopener, as a link with `target="_blank"` opens one, or opened other than by `window.open` (a form, or a link with `rel="opener"`, that targets a new window); - a popup at another origin than its opener's page, or whose opener's page has loaded another document by the time the popup's begins, or whose opener the seeding could not place, and a frame of another origin inside one of those popups; diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 5d07a35..ccbd879 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -30,11 +30,6 @@ // /agent/refusal), since keploy runs each test once in a run; // - before the test's browser context closes, the test waits until none of // its pages' calls is in flight (settle), so its calls end inside it; -// a seeded test's context then goes offline before it closes, whoever -// closes it (offlineBeforeClose), so a request the browser starts as it -// closes fails instead of reaching the app's server; a context -// Playwright reuses for the next test is not closed and stays online -// (reusedContext), and the worker says so once (reusedLine); // - the test ends (POST /agent/scope/end) after its last teardown, with // its outcome and what its context's pages drew their random numbers // from (random); when keploy saw a call with no recorded answer, or a @@ -72,7 +67,7 @@ import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; -import { Inflight, offlineBeforeClose, reusedContext, reusedLine, sharedWorkerLine, workersScript } from './settle'; +import { Inflight, sharedWorkerLine, workersScript } from './settle'; export { expect, KeployError }; export { seedFaker } from './random'; @@ -103,9 +98,6 @@ const saidFaker = new Set<string>(); // How many contexts this worker seeded: each one's report function has its // own name. let reports = 0; -// Whether this worker said that a seeded test's browser context Playwright -// reuses for the next test is not taken offline (reusedLine). -let saidReused = false; // How many contexts of a keploy test this worker opened: each one's Shared // Worker report function has its own name. let sharedReports = 0; @@ -296,33 +288,19 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ }); await context.addInitScript({ content: workersScript(sharedReport) }); const seeded = s.seed !== ''; - // Seeded: the context goes offline before it closes, whoever closes it, - // so nothing keploy marked reaches the app's server as it closes. - const offline = seeded ? offlineBeforeClose(context, name, (line) => { process.stdout.write(`${line}\n`); }) : null; const opened = (): void => { if (seeded) s.seededPage = true; else s.ownPage = true; }; if (context.pages().length > 0) opened(); context.on('page', opened); try { - try { - await use(context); - } finally { - restore(); - for (const say of waiting) say(); - } - if (inflight.size > 0) { - const calls = inflight.describe(); - const shown = calls.slice(0, 5).join(', ') + (calls.length > 5 ? `, and ${calls.length - 5} more` : ''); - await base.step(`keploy: wait for the test's ${calls.length} call(s) in flight before its browser context closes: ${shown}`, () => inflight.settled()); - } + await use(context); } finally { - // A context Playwright reuses for the next test is not closed, and its - // reset for that test needs the network: it stays online. - if (offline !== null && !reusedContext(context, process.env)) { - await offline(); - } else if (offline !== null && !saidReused) { - saidReused = true; - process.stdout.write(`${reusedLine(name)}\n`); - } + restore(); + for (const say of waiting) say(); + } + if (inflight.size > 0) { + const calls = inflight.describe(); + const shown = calls.slice(0, 5).join(', ') + (calls.length > 5 ? `, and ${calls.length - 5} more` : ''); + await base.step(`keploy: wait for the test's ${calls.length} call(s) in flight before its browser context closes: ${shown}`, () => inflight.settled()); } }, diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 1b3d2ac..b8b2982 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -368,120 +368,3 @@ export function sharedWorkerLine(name: string, at: string): string { '(Playwright reports none of a Shared Worker\'s calls), so a call it still has in flight when the test ends is cut there; ' + 'have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends.'; } - -/** - * The line a seeded test shows when keploy could not take its browser - * context offline before Playwright closes it (closeOffline): why says what - * setOffline threw. - */ -export function offlineLine(name: string, why: string): string { - return `keploy: in ${name}, keploy could not take the test's browser context offline before it closes (${why}), ` + - 'so a request the browser starts as the context closes (a Web Worker\'s script load, or the load of its seeding) can reach the app\'s server ' + - 'with keploy\'s __keploy_worker mark in its URL; have the test wait for what its last step starts (a message a worker posts, or what the page shows from it) before it ends.'; -} - -/** - * Takes a seeded test's browser context offline once keploy's end-of-test - * wait is done, before the context closes (offlineBeforeClose). Firefox and WebKit - * send a request that a page or a worker starts while its context closes to - * the network past every route, keploy's too, so a worker's script load, or - * the load of its seeding, that keploy marked would reach the app's server - * with the mark in its URL; offline, the browser fails it instead. No test - * step reads what such a request gets, and it is the same at record and at - * replay. A context that cannot go offline (setOffline throws: the context - * or its browser is gone, or this Playwright has none) closes anyway, and - * the test says so through say. - */ -export async function closeOffline(context: { setOffline(offline: boolean): Promise<void> }, name: string, say: (line: string) => void): Promise<void> { - try { - await context.setOffline(true); - } catch (e) { - say(offlineLine(name, (e instanceof Error ? e.message : String(e)).split('\n')[0])); - } -} - -/** A browser as offlineBeforeClose needs it. */ -export interface ClosingBrowser { - close(options?: { reason?: string }): Promise<void>; -} - -/** A browser context as offlineBeforeClose needs it. */ -export interface Closing { - setOffline(offline: boolean): Promise<void>; - close(options?: { reason?: string }): Promise<void>; - browser(): ClosingBrowser | null; - on(event: 'close', listener: () => void): unknown; -} - -// For each browser, the seeded contexts it has open: each goes offline -// before the browser closes. -const openOf = new WeakMap<ClosingBrowser, Set<() => Promise<void>>>(); - -/** - * Has a seeded test's browser context go offline (closeOffline) before it - * closes, whoever closes it: keploy's fixture once its end-of-test wait is - * done (the function this returns), the test's own context.close(), or its - * browser's close. Playwright skips a context's own teardown once the test's - * time is spent, so when a test's timeout cut keploy's wait short the - * context closes with its browser, as the worker ends. The context goes - * offline once; a later call or close finds that done. A context that closed - * (or that Playwright handed on for reuse) is no longer its browser's to - * take offline. - */ -export function offlineBeforeClose(context: Closing, name: string, say: (line: string) => void): () => Promise<void> { - let done: Promise<void> | null = null; - const now = async (): Promise<void> => { - done ??= closeOffline(context, name, say); - await done; - }; - const close = context.close; - context.close = async (options?: { reason?: string }): Promise<void> => { - await now(); - return close.call(context, options); - }; - const browser = context.browser(); - if (browser !== null) { - let open = openOf.get(browser); - if (open === undefined) { - const all = new Set<() => Promise<void>>(); - open = all; - openOf.set(browser, all); - const closeBrowser = browser.close; - browser.close = async (options?: { reason?: string }): Promise<void> => { - await Promise.all([...all].map((f) => f())); - return closeBrowser.call(browser, options); - }; - } - const mine = open; - mine.add(now); - context.on('close', () => { mine.delete(now); }); - } - return now; -} - -/** - * Whether Playwright keeps this browser context open for the next test - * instead of closing it at the test's end (its reuseContext option, or - * PW_TEST_REUSE_CONTEXT, which VS Code's Playwright extension sets for Show - * browser). Playwright marks the context it hands out for reuse - * (_forReuse); a Playwright without that flag is taken as reusing it when - * PW_TEST_REUSE_CONTEXT is set. - */ -export function reusedContext(context: object, env: Record<string, string | undefined>): boolean { - const f = (context as { _forReuse?: unknown })._forReuse; - if (typeof f === 'boolean') return f; - return (env.PW_TEST_REUSE_CONTEXT ?? '') !== ''; -} - -/** - * The line a worker shows once, at its first seeded test in a browser - * context Playwright reuses: keploy does not take such a context offline, - * since Playwright's reset of it for the next test needs the network. - */ -export function reusedLine(name: string): string { - return `keploy: in ${name}, Playwright keeps the test's browser context open for the next test (its reuseContext option, or PW_TEST_REUSE_CONTEXT, which VS Code's Show browser sets), ` + - 'so keploy does not take it offline as the test ends (Playwright\'s reset of it for the next test needs the network), ' + - 'and a request its pages or workers start after a test ends (a Web Worker\'s script load, or the load of its seeding) can reach the app\'s server ' + - 'with keploy\'s __keploy_worker mark in its URL; run keploy\'s tests without reuse (reuseContext: false, PW_TEST_REUSE_CONTEXT unset), ' + - 'or have each test wait for what its last step starts (a message a worker posts, or what the page shows from it) before it ends.'; -} diff --git a/playwright/test/fixtures/offline.spec.ts b/playwright/test/fixtures/offline.spec.ts deleted file mode 100644 index 588a45b..0000000 --- a/playwright/test/fixtures/offline.spec.ts +++ /dev/null @@ -1,44 +0,0 @@ -import type { BrowserContext, Page } from '@playwright/test'; -import fs from 'node:fs'; -import { test } from '../../dist'; - -// Run seeded and with seeding off. The context of a test whose -// setOffline throws, or that has none, cannot go offline before it closes. -// A test that puts back the context's own close, past keploy's, keeps -// whether its page reaches the app's server as Playwright closes it. -// The 'reused' tests run with PW_TEST_REUSE_CONTEXT=1 alone (-g reused): -// Playwright then keeps one browser context for the tests that ask the same -// options and does not close it between them. Each such test keeps whether -// its page loads the app's page (online or offline) and its context's id -// under its title. -const base = process.env.DRAWS_BASE ?? ''; -const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); -const reach = (page: Page): Promise<string> => page.goto(`${base}/ov`).then(() => 'online', () => 'offline'); -const id = (context: BrowserContext): unknown => (context as unknown as { _guid?: string })._guid; - -test.use({ serviceWorkers: 'block' }); - -test('a context whose setOffline throws', async ({ context, page }) => { - await page.goto(`${base}/ov`); - (context as unknown as { setOffline: () => Promise<void> }).setOffline = async () => { throw new Error('no offline here\nand a second line'); }; -}); - -test('a context with no setOffline', async ({ context, page }) => { - await page.goto(`${base}/ov`); - (context as unknown as { setOffline?: unknown }).setOffline = undefined; -}); - -test('a context whose close keploy did not wrap', async ({ context, page }) => { - await page.goto(`${base}/ov`); - const close = (Object.getPrototypeOf(context) as { close: (o?: { reason?: string }) => Promise<void> }).close; - context.close = async function (this: BrowserContext, o?: { reason?: string }): Promise<void> { - keep('at close', await page.evaluate(() => fetch('/ov?at-close').then(() => 'online', () => 'offline'))); - return close.call(this, o); - }; -}); - -test.describe('reused', () => { - test('online one', async ({ context, page }) => { keep(test.info().title, [await reach(page), id(context)]); }); - test('online two', async ({ context, page }) => { keep(test.info().title, [await reach(page), id(context)]); }); - test('online three', async ({ context, page }) => { keep(test.info().title, [await reach(page), id(context)]); }); -}); diff --git a/playwright/test/fixtures/teardown.spec.ts b/playwright/test/fixtures/teardown.spec.ts deleted file mode 100644 index 9a9835a..0000000 --- a/playwright/test/fixtures/teardown.spec.ts +++ /dev/null @@ -1,57 +0,0 @@ -import type { Route } from '@playwright/test'; -import { test } from '../../dist'; - -// Run seeded in Chromium, Firefox and WebKit (PW_BROWSER), each test -// TEARDOWN_COPIES times (default 1), with several runs at once: a test whose -// last step starts a worker and returns, and one whose page route takes -// itself off and then fulfills a Web Worker's script load, so the test ends -// and its browser context closes while the worker's script load, or the load -// of the seeding keploy's answer makes, is on its way; and one that closes -// its own browser context as a worker starts. Firefox and WebKit -// send a request started as the context closes past every route; the app's -// server must never see keploy's mark. -const base = process.env.DRAWS_BASE ?? ''; -const copies = Number(process.env.TEARDOWN_COPIES || 1); - -test.use({ serviceWorkers: 'block' }); - -for (let i = 0; i < copies; i++) { - test(`classic worker as the test ends ${i}`, async ({ page }) => { - await page.goto(`${base}/ov`); - await page.evaluate(() => { new Worker('/ov/pw.js'); }); - }); - - test(`module worker as the test ends ${i}`, async ({ page }) => { - await page.goto(`${base}/ov`); - await page.evaluate(() => { new Worker('/ov/pw.js', { type: 'module' }); }); - }); - - test(`classic worker started in a timer as the test ends ${i}`, async ({ page }) => { - await page.goto(`${base}/ov`); - await page.evaluate(() => { setTimeout(() => { new Worker('/ov/pw.js'); }, 0); }); - }); - - test(`classic worker as the test closes its context ${i}`, async ({ context, page }) => { - await page.goto(`${base}/ov`); - await page.evaluate(() => { new Worker('/ov/pw.js'); }); - await context.close(); - }); - - test(`shared worker as the test ends ${i}`, async ({ page }) => { - await page.goto(`${base}/ov`); - await page.evaluate(() => { new SharedWorker('/ov/ps.js'); }); - }); - - test(`unrouteAll default then fulfill ${i}`, async ({ page }) => { - await page.goto(`${base}/ov`); - await page.route('**/ov/pw.js', async (r: Route): Promise<void> => { - await page.unrouteAll({ behavior: 'default' }); - await r.fulfill({ contentType: 'text/javascript', body: 'postMessage("own")' }); - }); - await page.evaluate(() => new Promise((resolve) => { - const w = new Worker('/ov/pw.js'); - w.onmessage = (e) => resolve(e.data); - w.onerror = () => resolve('error'); - })); - }); -} diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index c87092c..faf45f5 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -11,7 +11,7 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, markWorker, workerMark, workerScript } from '../dist/random.js'; -import { offlineLine, reusedLine, sharedWorkerLine } from '../dist/settle.js'; +import { sharedWorkerLine } from '../dist/settle.js'; import { NO_NOTE_LINE, readNote, unreadNoteLine } from '../dist/note.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; @@ -2545,116 +2545,3 @@ for (const engine of ENGINES) { } finally { await rec.close(); } }); } - -// teardown.spec.ts, seeded, each test 5 times in each of four runs at once -// (the load the race needs: before keploy took the context offline, of 32 -// runs each, the server saw the mark in up to 10 in Firefox and 8 in WebKit): -// a test whose last step starts a Web Worker (classic, module, or from a -// timer) or a Shared Worker and returns, and one whose page route takes -// itself off and then fulfills a Web Worker's script load, end while what -// keploy marked is on its way; one closes its own context as a Web Worker -// starts. keploy takes the context offline once its end-of-test wait is -// done, or as the test closes it, before it closes, so the app's server -// never sees keploy's mark; each test ends as it does with seeding off. -for (const engine of ENGINES) { - test(`seeded, in ${engine}: a test whose last step starts a Web Worker (classic, module, from a timer) or a Shared Worker and returns, one whose page route takes itself off and then fulfills a Web Worker's script load, and one that closes its own browser context as a Web Worker starts, end while what keploy marked is on its way; keploy takes the browser context offline before it closes, so over 20 runs of each, four runs at once, the app's server never sees keploy's mark`, async () => { - const j = journal(); - const rec = await fakeAgent(j); - try { - const runs = await Promise.all([0, 1, 2, 3].map(async () => { - const d = await drawsServer(); - try { - const r = await playwright(['teardown.spec.ts'], { ...keployEnv(rec.url), PW_BROWSER: engine, DRAWS_BASE: d.url, TEARDOWN_COPIES: '5' }); - return { ...r, seen: d.seen.slice() }; - } finally { await d.close(); } - })); - for (const { tests, out, seen } of runs) { - assert.equal(Object.keys(tests).length, 30, out); - for (const [title, t] of Object.entries(tests)) { - if (title.startsWith('unrouteAll default then fulfill ')) { - assert.equal(t.status, 'failed', `${title}\n${out}`); - assert.match(t.error, /route\.fulfill: Route is already handled!/, `${title}: ${t.error}`); - } else { - assert.equal(t.status, 'passed', `${title}: ${t.error}\n${out}`); - } - } - assert.deepEqual(seen.filter((u) => u.includes('__keploy_worker')), []); - assert.doesNotMatch(out, /could not take the test's browser context offline/); - assert.doesNotMatch(out, /Playwright keeps the test's browser context open/); - } - } finally { await rec.close(); } - }); -} - -// offline.spec.ts: a seeded test whose context's setOffline throws, or that -// has none, passes, its context closes, and it says once that a request -// started as its context closes can reach the app's server with keploy's -// mark (the first line of what setOffline threw); with seeding off keploy -// never takes a context offline, so neither says anything. A test that puts -// back the context's own close, past keploy's, finds its context offline as -// Playwright closes it (keploy took it offline after its wait), or online -// with seeding off. -test('seeded: a test whose browser context cannot go offline (its setOffline throws, or there is none) closes it anyway and says so once; with seeding off nothing is said; a context closed past keploy\'s close is offline as it closes, seeded, and online with seeding off', async () => { - const j = journal(); - const rec = await fakeAgent(j); - try { - for (const mode of ['seeded', 'off']) { - const d = await drawsServer(); - try { - const { tests, tmp, out } = await playwright(['offline.spec.ts', '-g', 'a context'], { ...keployEnv(rec.url), DRAWS_BASE: d.url, ...(mode === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }); - const why = { 'a context whose setOffline throws': 'no offline here', 'a context with no setOffline': 'context.setOffline is not a function' }; - const past = 'a context whose close keploy did not wrap'; - assert.deepEqual(Object.keys(tests).sort(), [...Object.keys(why), past].sort(), out); - assert.equal(tests[past].status, 'passed', `${mode} ${past}: ${tests[past].error}\n${out}`); - assert.equal(JSON.parse(fs.readFileSync(path.join(tmp, 'at close.json'), 'utf8')), mode === 'seeded' ? 'offline' : 'online', mode); - assert.doesNotMatch(tests[past].stdout, /could not take the test's browser context offline/, mode); - for (const [title, w] of Object.entries(why)) { - const t = tests[title]; - assert.equal(t.status, 'passed', `${mode} ${title}: ${t.error}\n${out}`); - const said = t.stdout.split('\n').filter((l) => l.includes('could not take the test\'s browser context offline')); - assert.deepEqual(said, mode === 'seeded' ? [offlineLine(title, w)] : [], `${mode} ${title}`); - } - assert.doesNotMatch(out, /Playwright keeps the test's browser context open/, mode); - } finally { await d.close(); } - } - } finally { await rec.close(); } -}); - -// offline.spec.ts's 'reused' tests under PW_TEST_REUSE_CONTEXT=1: Playwright -// keeps one context for the tests that ask the same options and does not -// close it between them, and resets it for the next test, which needs the -// network (Firefox clears its storage by loading its origin). keploy never -// takes such a context offline: three tests in one context each reach the -// app's page online, as with seeding off, and the run says once, at its -// first seeded test, that the context stays online and what a request after -// a test's end can then carry (one worker runs them all). -for (const engine of ENGINES) { - test(`seeded, in ${engine}: in a browser context Playwright reuses, keploy leaves the context online, so a test after a seeded one is online, as with seeding off, and the run says once that the reused context stays online`, async () => { - const j = journal(); - const rec = await fakeAgent(j); - try { - for (const mode of ['seeded', 'off']) { - const d = await drawsServer(); - try { - const { tests, tmp, out } = await playwright(['offline.spec.ts', '-g', 'reused'], { ...keployEnv(rec.url), PW_BROWSER: engine, PW_TEST_REUSE_CONTEXT: '1', DRAWS_BASE: d.url, ...(mode === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }); - const want = { 'online one': 'online', 'online two': 'online', 'online three': 'online' }; - const got = {}; - for (const [title, w] of Object.entries(want)) { - const key = Object.keys(tests).find((k) => k.endsWith(` › ${title}`)); - assert.ok(key !== undefined, `${mode} ${title}\n${out}`); - assert.equal(tests[key].status, 'passed', `${mode} ${title}: ${tests[key].error}\n${out}`); - got[title] = JSON.parse(fs.readFileSync(path.join(tmp, `${title}.json`), 'utf8')); - assert.equal(got[title][0], w, `${mode} ${title}`); - } - // The premise: they ran in one context. - assert.equal(got['online two'][1], got['online one'][1], mode); - assert.equal(got['online three'][1], got['online one'][1], mode); - assert.doesNotMatch(out, /could not take the test's browser context offline/); - const first = Object.keys(tests).find((k) => k.endsWith(' › online one')); - const said = Object.entries(tests).flatMap(([k, t]) => t.stdout.split('\n').filter((l) => l.includes('Playwright keeps the test\'s browser context open')).map((l) => [k, l])); - assert.deepEqual(said, mode === 'seeded' ? [[first, reusedLine(first)]] : [], `${mode}\n${out}`); - } finally { await d.close(); } - } - } finally { await rec.close(); } - }); -} diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 4a8f95f..273f9bf 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -8,7 +8,7 @@ import os from 'node:os'; import path from 'node:path'; import { AgentClient, KeployError } from '../dist/agent.js'; import { decide } from '../dist/join.js'; -import { Inflight, closeOffline, offlineBeforeClose, offlineLine, reusedContext, reusedLine, workersScript } from '../dist/settle.js'; +import { Inflight, workersScript } from '../dist/settle.js'; import { markWorker } from '../dist/random.js'; import { EventEmitter } from 'node:events'; import vm from 'node:vm'; @@ -728,111 +728,3 @@ test('keployText: keploy\'s reason once prefixed, or the fallback when it gave n // Only the whole word with its separator counts as keploy's prefix. assert.equal(keployText('keploy:x', 'fb'), 'keploy: keploy:x'); }); - -test('closeOffline takes the context offline and says nothing; when setOffline throws, or there is none, it says once why, the first line of what was thrown', async () => { - const said = []; - const say = (l) => { said.push(l); }; - const asked = []; - await closeOffline({ setOffline: async (o) => { asked.push(o); } }, 't', say); - assert.deepEqual([asked, said], [[true], []]); - await closeOffline({ setOffline: async () => { throw new Error('gone\ncall log'); } }, 't', say); - await closeOffline({ setOffline: async () => { throw 'a string'; } }, 'u', say); - await closeOffline({}, 'v', say); - assert.deepEqual(said, [offlineLine('t', 'gone'), offlineLine('u', 'a string'), offlineLine('v', 'context.setOffline is not a function')]); - assert.match(said[0], /^keploy: in t, keploy could not take the test's browser context offline before it closes \(gone\), so a request .* __keploy_worker mark/); -}); - -test('offlineBeforeClose: the context goes offline once, before it closes, whether keploy\'s wait ends first, Playwright closes it first (a test timeout), or the test closes it; close gets its options; a setOffline that throws says so once and the context closes anyway', async () => { - const fake = (setOffline) => { - const log = []; - const c = { setOffline: async (o) => { log.push(`offline ${o}`); await setOffline(); }, close: async function (opts) { log.push(['close', this === c, opts]); }, browser: () => null, on: () => c }; - return { c, log }; - }; - const said = []; - const say = (l) => { said.push(l); }; - // keploy's wait ends first, then Playwright closes it. - let { c, log } = fake(async () => {}); - let now = offlineBeforeClose(c, 't', say); - assert.deepEqual(log, []); - await now(); - await c.close({ reason: 'Test ended.' }); - await now(); - assert.deepEqual(log, ['offline true', ['close', true, { reason: 'Test ended.' }]]); - // Playwright closes it first (keploy's wait cut short), or the test does, - // twice; keploy's call after finds it done. - ({ c, log } = fake(async () => {})); - now = offlineBeforeClose(c, 't', say); - await c.close({ reason: 'Test timeout of 30000ms exceeded.' }); - await c.close(); - await now(); - assert.deepEqual(log, ['offline true', ['close', true, { reason: 'Test timeout of 30000ms exceeded.' }], ['close', true, undefined]]); - // A close while setOffline is under way waits for it. - let open; - ({ c, log } = fake(() => new Promise((f) => { open = f; }))); - now = offlineBeforeClose(c, 't', say); - const waited = now(); - const closing = c.close(); - await new Promise((f) => setImmediate(f)); - assert.deepEqual(log, ['offline true']); - open(); - await Promise.all([waited, closing]); - assert.deepEqual(log, ['offline true', ['close', true, undefined]]); - assert.deepEqual(said, []); - // setOffline throws: one line, and the context still closes. - ({ c, log } = fake(async () => { throw new Error('closed\ncall log'); })); - now = offlineBeforeClose(c, 'u', say); - await now(); - await c.close(); - assert.deepEqual([log, said], [['offline true', ['close', true, undefined]], [offlineLine('u', 'closed')]]); -}); - -test('reusedContext: Playwright\'s own _forReuse flag when the context has one; without it, whether PW_TEST_REUSE_CONTEXT is set; reusedLine names the test, why the context stays online and the next step', () => { - assert.equal(reusedContext({ _forReuse: true }, {}), true); - assert.equal(reusedContext({ _forReuse: false }, { PW_TEST_REUSE_CONTEXT: '1' }), false); - assert.equal(reusedContext({}, { PW_TEST_REUSE_CONTEXT: '1' }), true); - assert.equal(reusedContext({}, { PW_TEST_REUSE_CONTEXT: '' }), false); - assert.equal(reusedContext({}, {}), false); - assert.equal(reusedContext({ _forReuse: 'yes' }, { PW_TEST_REUSE_CONTEXT: '1' }), true); - assert.match(reusedLine('t'), /^keploy: in t, Playwright keeps the test's browser context open for the next test .*does not take it offline .* __keploy_worker mark in its URL; run keploy's tests without reuse \(reuseContext: false, PW_TEST_REUSE_CONTEXT unset\)/); -}); - -test('offlineBeforeClose: before a browser closes, each seeded context it has open goes offline (a test timeout: Playwright then closes the browser, not the context), and the browser closes once they have; a context that closed, or that Playwright handed on for reuse, is not asked; the browser\'s close is wrapped once', async () => { - const said = []; - const say = (l) => { said.push(l); }; - const log = []; - const b = { close: async function (opts) { log.push(['browser close', this === b, opts]); } }; - const fake = (n, setOffline = async () => {}) => { - const on = new Map(); - const c = { - setOffline: async (o) => { log.push(`${n} offline ${o}`); await setOffline(); }, - close: async () => { log.push(`${n} close`); on.get('close')?.(); }, - browser: () => b, - on: (e, f) => { on.set(e, f); return c; }, - emit: (e) => on.get(e)?.(), - }; - return c; - }; - let open; - const c1 = fake('c1', () => new Promise((f) => { open = f; })); - const c2 = fake('c2'); - const c3 = fake('c3'); - const c4 = fake('c4'); - offlineBeforeClose(c1, 't1', say); - const wrapped = b.close; - offlineBeforeClose(c2, 't2', say); - offlineBeforeClose(c3, 't3', say); - offlineBeforeClose(c4, 't4', say); - assert.equal(b.close, wrapped); - // c3 closed through keploy (offline first); c4 was handed on for reuse - // (Playwright's disconnect says close, with no close call). - await c3.close(); - c4.emit('close'); - log.length = 0; - const closing = b.close({ reason: 'Test ended.' }); - await new Promise((f) => setImmediate(f)); - assert.deepEqual(log, ['c1 offline true', 'c2 offline true']); - open(); - await closing; - assert.deepEqual(log, ['c1 offline true', 'c2 offline true', ['browser close', true, { reason: 'Test ended.' }]]); - assert.deepEqual(said, []); -}); From 6d19ea9d75e2cc66f22c1cd0938b0ae982a3973a Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 13:46:24 +0530 Subject: [PATCH 56/73] fix(playwright): keep a worker's script bytes; seed inline under a CSP keploy's answer to a Web Worker's script decoded the app's script as UTF-8 and wrote it out again. Chromium reads a worker's script by its charset, so a script served as iso-8859-1 read differently seeded than with seeding off. The answer now holds the bytes the server sent, with keploy's ASCII statements ahead of them and after them (in UTF-16 of the script's byte order when its byte order mark names UTF-16). A byte order mark and a hashbang line stay first, and the app's lines keep their numbers. A Content-Security-Policy on the worker script's response could refuse keploy's load of the seeding (importScripts or import at a marked URL), so the worker died at keploy's first statement in every engine. When the response carries one, the seeding now goes on the script's first line instead, on one line, which no policy refuses. In a module that line runs after the modules the script imports, so the test says so once for each address and names the next step. A classic worker's answer starts with keploy's own 'use strict' when the app's directive prologue has one. Rows: the bytes of a Latin-1 script, a UTF-8 and a UTF-16 byte order mark, hashbang lines ended by CR LF, CR, LF, U+2028 and U+2029; the seeding on one line draws what it draws on its own lines; the inline answer under a policy, for classic and module workers, and its line; a directive after a string ended by U+2028 alone is strict. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 3 +- playwright/src/index.ts | 11 ++- playwright/src/random.ts | 142 ++++++++++++++++++++++++++++---- playwright/test/random.test.mjs | 86 +++++++++++++++++-- 4 files changed, 220 insertions(+), 22 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 05557df..8ab6fac 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (`importScripts` for a classic worker, after the app's own `'use strict'` when it has one, so the app's code stays strict; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy, which could refuse that load, the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a Web Worker whose script carries a Content-Security-Policy, where keploy's seeding is on the app's first line: there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -101,6 +101,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. - **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. +- **... a module Web Worker's script is served with a Content-Security-Policy.** keploy puts the worker's seeding on its script's first line, as the policy could refuse the load of keploy's seeding, and in a module that line runs after the modules the script imports: a random number one of those modules draws as it loads is the browser's own, so it may differ at replay. The test says so once for each address (`keploy: in <test>, the module Web Worker at <address> is served with a Content-Security-Policy, ...`), with the next step: if one of those modules draws as it loads, serve the worker's script with no Content-Security-Policy where the tests run, and keploy loads its seeding ahead of those modules. A classic Web Worker's seeding on its first line runs before any of the app's code. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. - **... the browser starts a request while Playwright closes a seeded test's context.** Once the test has ended (its steps, settle and its teardown), Playwright closes its context, and keploy's route goes with it. A request the browser starts in that moment, after the test has ended, can reach the app's server with keploy's `__keploy_worker` mark in its URL, in Firefox and WebKit: a worker the test's last step started, whose script the browser asks for as the context closes. No step of the test reads what such a request gets, and nothing is said. Measured on a test whose last step starts a worker and returns, 32 runs of each with four at once: in Firefox, 9 or 10 of 32 for a Web Worker (classic, module, or one a timer starts) and none for a Shared Worker; in WebKit, 1 to 8 of 32; in Chromium, none; for a test that waits for its worker's message, none in any engine. Next step: have the test wait for what the worker it starts last does before it ends. - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: diff --git a/playwright/src/index.ts b/playwright/src/index.ts index ccbd879..cd175a1 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -66,7 +66,7 @@ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; -import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; +import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, lateLine, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; import { Inflight, sharedWorkerLine, workersScript } from './settle'; export { expect, KeployError }; @@ -259,6 +259,15 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ told.add(k); process.stdout.write(`${unplacedLine(name, a, why)}\n`); }, + // A module Web Worker whose seeding runs after its imports (its + // script carries a Content-Security-Policy): once for each address. + late: (at) => { + const a = pageAddress(at); + const k = `late\n${a}`; + if (told.has(k)) return; + told.add(k); + process.stdout.write(`${lateLine(name, a)}\n`); + }, }, browserName === 'webkit'); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox', drawnReport) }); const placing = placePages(context, mark); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 1b69fe1..d98a30b 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1802,6 +1802,79 @@ export function cspAllowsData(headers: Record<string, string>): boolean { return true; } +/** + * The encoding a script's bytes name with a byte order mark (which every + * engine reads a worker's script by, whatever its charset says), and the + * mark's length: UTF-16 of either byte order, else UTF-8 (0 when it has + * none). + */ +function byteOrder(b: Uint8Array): { enc: 'utf-16le' | 'utf-16be' | 'utf-8'; bom: number } { + if (b[0] === 0xff && b[1] === 0xfe) return { enc: 'utf-16le', bom: 2 }; + if (b[0] === 0xfe && b[1] === 0xff) return { enc: 'utf-16be', bom: 2 }; + return { enc: 'utf-8', bom: b[0] === 0xef && b[1] === 0xbb && b[2] === 0xbf ? 3 : 0 }; +} + +/** + * A script's text, for keploy to read (useStrict, a sourceURL of its own): + * its bytes in the encoding its byte order mark names, else UTF-8, the + * mark taken off. keploy's answer never holds this text: it holds the + * bytes (withBytes). + */ +export function scriptText(body: Uint8Array): string { + return new TextDecoder(byteOrder(body).enc).decode(body); +} + +/** + * keploy's answer to a Web Worker's script: body, the app's script as its + * server sent it, with prefix ahead of it and suffix after it. The app's + * bytes are never decoded and encoded again, so an engine that reads a + * worker's script by its charset (Chromium) reads the app's text as with + * seeding off. prefix goes after a byte order mark and after a hashbang + * line (#!, which only a source's very start may hold), each kept where it + * is, so every line of the app's keeps its number. prefix and suffix are + * ASCII, which reads the same in every charset a browser reads a script in, + * or, for a script whose byte order mark names UTF-16, UTF-16 of its byte + * order. + */ +export function withBytes(body: Uint8Array, prefix: string, suffix: string): Buffer { + const b = Buffer.from(body.buffer, body.byteOffset, body.byteLength); + const { enc, bom } = byteOrder(b); + const w = enc === 'utf-8' ? 1 : 2; + const unit = (i: number): number => (i + w > b.length ? -1 : enc === 'utf-16le' ? b[i] | (b[i + 1] << 8) : enc === 'utf-16be' ? (b[i] << 8) | b[i + 1] : b[i]); + const encode = (s: string): Buffer => { + if (w === 1) return Buffer.from(s, 'latin1'); + const u = Buffer.from(s, 'utf16le'); + return enc === 'utf-16be' ? u.swap16() : u; + }; + let at = bom; + if (unit(at) === 0x23 && unit(at + w) === 0x21) { + // The hashbang line's end: a line feed, a carriage return (and the line + // feed after it), or U+2028 or U+2029 (three bytes in UTF-8). + let i = at + 2 * w; + let end = -1; + for (; i < b.length && end < 0; i += w) { + const c = unit(i); + if (c === 0x0a || c === 0x2028 || c === 0x2029) end = i + w; + else if (c === 0x0d) end = unit(i + w) === 0x0a ? i + 2 * w : i + w; + else if (w === 1 && c === 0xe2 && b[i + 1] === 0x80 && (b[i + 2] === 0xa8 || b[i + 2] === 0xa9)) end = i + 3; + } + if (end < 0) return Buffer.concat([b, encode(`\n${prefix}${suffix}`)]); + at = end; + } + return Buffer.concat([b.subarray(0, at), encode(prefix), b.subarray(at), encode(suffix)]); +} + +/** + * A worker's seeding (workerScript) on one line, for keploy to put on the + * app's first line: each line of it that is a comment alone goes, and the + * rest are joined by a space. Each of its statements ends with its + * semicolon or block, and none of its strings, templates or regular + * expressions spans a line, so it means the same. + */ +export function oneLine(seeding: string): string { + return seeding.split('\n').filter((l) => !/^\s*\/\//.test(l)).map((l) => l.trim()).filter((l) => l !== '').join(' '); +} + /** * Where keploy's first statement goes in a script's source: ahead of it, or * after its hashbang line (#!, which only a source's very start may hold), @@ -1874,11 +1947,16 @@ export function useStrict(source: string): boolean { /** * keploy's answer to a marked worker script ('w', 'm', 's' or 'x', for the * instance n of token). A Web Worker's ('w', 'm') is the app's own script, - * source, as keploy fetched it, whose first statement loads the worker's - * seeding from a URL keploy marked ('q' for a classic worker, by - * importScripts, after a 'use strict' of the app's script's own when it has - * one, so it stays strict; 'p' for a module one, by an import ahead of every - * import of the app's, so it runs before any of them), on the app's first + * source, the bytes keploy fetched (withBytes; given as text, the answer is + * text too, its bytes read as UTF-8), whose first statement loads the + * worker's seeding from a URL keploy marked ('q' for a classic worker, by + * importScripts; 'p' for a module one, by an import ahead of every import + * of the app's, so it runs before any of them), or, inline (the script's + * response carries a Content-Security-Policy, which could refuse that + * load), is the seeding itself on one line (oneLine), which in a module + * runs after the modules the app's script imports; a classic worker's + * starts with a 'use strict' of keploy's when the app's script has one in + * its directive prologue, so it stays strict. It goes on the app's first * line, so each line keeps its number; then, unless the app's script names * itself, a sourceURL of the URL the app asked for. The app's script is so * loaded once, and runs at the worker's marked URL, after its seeding: a @@ -1915,14 +1993,19 @@ export function useStrict(source: string): boolean { * order, each in a microtask of its own, to the handlers the app's script * sets. */ -export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true, source?: string): string { +export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk?: boolean, source?: string, inline?: boolean, reach?: boolean): string; +export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk: boolean, source: Uint8Array, inline?: boolean, reach?: boolean): Buffer; +export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true, source?: string | Uint8Array, inline = false, reach = false): string | Buffer { const url = JSON.stringify(own); if ((kind === 'w' || kind === 'm') && source !== undefined) { + const bytes = typeof source === 'string' ? Buffer.from(source) : source; + const text = scriptText(bytes); const first = kind === 'w' - ? `${useStrict(source) ? '"use strict";' : ''}importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});` - : `import ${JSON.stringify(markWorker(own, 'p', token, n))};`; - const named = /(?:\/\/|\/\*)[#@][ \t]*sourceURL[ \t]*=/.test(source); - return withPrefix(source, first) + (named ? '' : `\n//# sourceURL=${noFragment(own)}\n`); + ? `${useStrict(text) ? '"use strict";' : ''}${inline ? oneLine(workerScript(token, n, kind, reach)) : `importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});`}` + : inline ? oneLine(workerScript(token, n, kind, reach)) : `import ${JSON.stringify(markWorker(own, 'p', token, n))};`; + const named = /(?:\/\/|\/\*)[#@][ \t]*sourceURL[ \t]*=/.test(text); + const answer = withBytes(bytes, first, named ? '' : `\n//# sourceURL=${noFragment(own)}\n`); + return typeof source === 'string' ? answer.toString() : answer; } if (kind === 'w') return `importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; if (kind === 's') return `${workerScript(token, n, kind)}queueMicrotask(importScripts.bind(globalThis, ${url}));\n`; @@ -1975,8 +2058,26 @@ export type WorkerHooks = { * the route does not reach ('nested'), which so runs unseeded. */ unmarked?: (at: string, why: 'worker' | 'nested') => void; + /** + * keploy put the seeding of the module Web Worker at at (its origin and + * path) on its script's first line, as the script's response carries a + * Content-Security-Policy: it runs after the modules the script imports. + */ + late?: (at: string) => void; }; +/** + * The line a test shows, once for each address, for a module Web Worker + * whose seeding started after the modules its script imports (WorkerHooks + * late). + */ +export function lateLine(name: string, at: string): string { + return `keploy: in ${name}, the module Web Worker at ${at} is served with a Content-Security-Policy, which could refuse keploy's load of its seeding, ` + + 'so keploy put its seeding on the first line of the worker\'s script, and it runs after the modules the script imports: ' + + 'a random number one of those modules draws as it loads is the browser\'s own, so it may differ at replay; ' + + 'if one does, serve the worker\'s script with no Content-Security-Policy where the tests run, and keploy loads its seeding ahead of those modules.'; +} + /** What keploy's routes go on: the test's context, or one of its pages. */ type Routes = Pick<BrowserContext, 'route' | 'unroute' | 'unrouteAll'>; @@ -2235,7 +2336,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = const whole = (m.kind === 's' || m.kind === 'x') && h !== undefined && otherwise(h); let status = 200; let headers: Record<string, string> = { 'content-type': 'text/javascript' }; - let source: string | undefined; + let source: Uint8Array | undefined; try { const got = await route.fetch(o); if (whole || !got.ok()) { @@ -2248,13 +2349,24 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } status = got.status(); headers = headersOf(got.headers()); - // A worker's script is read as UTF-8 whatever its charset says, as - // the browser reads it; keploy's answer is UTF-8 too. - if (m.kind === 'w' || m.kind === 'm') source = new TextDecoder().decode(await got.body()); + // A Web Worker's answer holds the app's script's bytes as the server + // sent them, whatever their charset (withBytes). + if (m.kind === 'w' || m.kind === 'm') source = await got.body(); } catch { // The worker loads the app's script itself, and fails as it would. } - await fulfill({ status, headers, body: workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers), source) }); + // A Content-Security-Policy on the script's response is the worker's + // own: it could refuse the load of keploy's seeding at its own URL + // (importScripts or import), so the seeding goes on the app's first + // line instead, which no policy refuses. A module's static imports run + // before its first line, so such a module Web Worker's seeding starts + // after them, and the test says so. + const inline = source !== undefined && headers['content-security-policy'] !== undefined; + if (inline && m.kind === 'm') { + const u = new URL(m.url); + hooks.late?.(u.origin + u.pathname); + } + await fulfill({ status, headers, body: source === undefined ? workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers)) : workerAnswer(m.token, n, m.kind as 'w' | 'm', m.url, true, source, inline, reach) }); }; const first = async (route: Route): Promise<void> => { const r = route.request(); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 889387a..880daf2 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, withBytes, oneLine, lateLine, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -2082,7 +2082,7 @@ function routed() { if (answer instanceof Error) throw answer; return answer; }, - fulfill: async (o) => { ended = true; did.push(['fulfill', { ...o, body: String(o.body) }]); }, + fulfill: async (o) => { ended = true; r.raw = o.body; did.push(['fulfill', { ...o, body: String(o.body) }]); }, continue: async (o) => { ended = true; did.push(['continue', o?.url ?? at]); }, abort: async () => { ended = true; did.push(['abort']); }, fallback: async (o) => { @@ -2117,9 +2117,13 @@ test('workerRoutes answers a marked classic worker script with its seeding then, ['fulfill', { status: 201, headers, body: `importScripts(${JSON.stringify(markWorker('http://h/w.js?a=1#f', 'q', t, 0))});app()\n//# sourceURL=http://h/w.js?a=1\n` }]]); const second = await r.ask(markWorker('http://h/w.js?a=1', 'w', t), answerOf(200, 'app()')); assert.equal(second[1][1].body, workerAnswer(t, 1, 'w', 'http://h/w.js?a=1', true, 'app()')); - // A worker's script is read as UTF-8, a BOM taken off, whatever its charset says. - const latin = await r.ask(markWorker('http://h/w.js', 'w', t), { ...answerOf(200, ''), body: async () => Buffer.from([0xef, 0xbb, 0xbf, 0x61, 0xc3, 0xa9]) }); - assert.equal(latin[1][1].body, workerAnswer(t, 2, 'w', 'http://h/w.js', true, 'a\u00e9')); + // The app's bytes as the server sent them: a byte order mark stays first, and a Latin-1 byte stays that byte. + await r.ask(markWorker('http://h/w.js', 'w', t), { ...answerOf(200, ''), body: async () => Buffer.from([0xef, 0xbb, 0xbf, 0x61, 0xc3, 0xa9]) }); + const q2 = `importScripts(${JSON.stringify(markWorker('http://h/w.js', 'q', t, 2))});`; + assert.deepEqual(r.raw, Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from(q2), Buffer.from([0x61, 0xc3, 0xa9]), Buffer.from('\n//# sourceURL=http://h/w.js\n')])); + await r.ask(markWorker('http://h/w.js', 'w', t), { ...answerOf(200, '', { 'content-type': 'text/javascript; charset=iso-8859-1' }), body: async () => Buffer.from([0x68, 0xe9]) }); + const q3 = `importScripts(${JSON.stringify(markWorker('http://h/w.js', 'q', t, 3))});`; + assert.deepEqual(r.raw, Buffer.concat([Buffer.from(q3), Buffer.from([0x68, 0xe9]), Buffer.from('\n//# sourceURL=http://h/w.js\n')])); // With no source (keploy could not fetch it), its seeding, then the app's script loaded in a microtask. assert.equal(workerAnswer(t, 1, 'w', 'http://h/w.js?a=1'), `importScripts(${JSON.stringify(markWorker('http://h/w.js?a=1', 'q', t, 1))});queueMicrotask(importScripts.bind(globalThis, "http://h/w.js?a=1"));\n`); }); @@ -2526,6 +2530,8 @@ test('workerAnswer: a Web Worker\'s is the app\'s script with its seeding\'s loa ['"a
b"; "use strict"', true], ['\'a
b\'\n\'use strict\'', true], // A line comment that ends the source ends the directive's statement. ['"use strict" // c', true], + // U+2028 and U+2029 end a line as a line feed does, so the directive after one is a statement of its own. + ['"a"\u2028"use strict"', true], ['"a"\u2029"use strict"', true], ['"a" "use strict"', false], ]) assert.equal(useStrict(src), strict, JSON.stringify(src)); assert.equal(withPrefix('a', 'P;'), 'P;a'); assert.equal(withPrefix('#!x\u2028a', 'P;'), '#!x\u2028P;a'); @@ -2975,3 +2981,73 @@ test('workerRoutes: a request at a worker script\'s URL that keploy handed nothi // The app's own fetch at that URL, from another page with no route. assert.deepEqual(await r.ask('http://h/k.js', null, { type: 'fetch', headers: {} }, r.page(true)), [['fallback']]); }); + +test('workerAnswer keeps the app\'s script\'s bytes: another charset\'s bytes, a byte order mark (UTF-8 or UTF-16) and a hashbang line keep their place, and keploy\'s statements are in the script\'s own encoding', () => { + const t = 'ab'.repeat(16); + const own = 'http://h/w.js'; + const q = `importScripts(${JSON.stringify(markWorker(own, 'q', t, 0))});`; + const tail = `\n//# sourceURL=${own}\n`; + const at = (bytes) => workerAnswer(t, 0, 'w', own, true, Buffer.from(bytes)); + const a = (x) => Buffer.from(x, 'latin1'); + assert.deepEqual(at([0x68, 0xe9]), Buffer.concat([a(q), Buffer.from([0x68, 0xe9]), a(tail)])); + assert.deepEqual(at([0xef, 0xbb, 0xbf, 0x61]), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), a(q), a('a'), a(tail)])); + const le = at(Buffer.concat([Buffer.from([0xff, 0xfe]), Buffer.from('"use strict";a\u00e9()', 'utf16le')])); + assert.deepEqual([...le.subarray(0, 2)], [0xff, 0xfe]); + assert.equal(new TextDecoder('utf-16le').decode(le), `"use strict";${q}"use strict";a\u00e9()${tail}`); + const be = at(Buffer.concat([Buffer.from([0xfe, 0xff]), Buffer.from('#!/x\na()', 'utf16le').swap16()])); + assert.deepEqual([...be.subarray(0, 2)], [0xfe, 0xff]); + assert.equal(new TextDecoder('utf-16be').decode(be), `#!/x\n${q}a()${tail}`); + assert.equal(new TextDecoder('utf-16le').decode(at(Buffer.concat([Buffer.from([0xff, 0xfe]), Buffer.from('#!/x\r\na()', 'utf16le')]))), `#!/x\r\n${q}a()${tail}`); + for (const [src, head] of [['#!/x\r\na\u00e9', '#!/x\r\n'], ['#!/x\ra', '#!/x\r'], ['#!/x\na', '#!/x\n'], ['#!/x\u2028a', '#!/x\u2028'], ['#!/x\u2029a', '#!/x\u2029'], ['\ufeff#!/x\na', '\ufeff#!/x\n']]) { + assert.deepEqual(at(Buffer.from(src)), Buffer.concat([Buffer.from(head), a(q), Buffer.from(src.slice(head.length)), a(tail)]), JSON.stringify(src)); + } + assert.deepEqual(at(Buffer.from('#!/x')), Buffer.concat([a(`#!/x\n${q}`), a(tail)])); + assert.deepEqual(at(Buffer.from('#')), Buffer.concat([a(q), a('#'), a(tail)])); + // Given text, it answers text: the same bytes read as UTF-8. + assert.equal(workerAnswer(t, 0, 'w', own, true, '#!/x\u2028a\u00e9'), `#!/x\u2028${q}a\u00e9${tail}`); + assert.deepEqual(withBytes(Buffer.from('a'), 'P;', ''), Buffer.from('P;a')); +}); + +test('oneLine: a worker\'s seeding on one line, ASCII, draws what it draws on its own lines, and keeps the worker\'s location free of the mark', () => { + const t = 'ef'.repeat(16); + for (const kind of ['w', 'm']) { + for (const reach of [false, true]) { + const s = workerScript(t, 3, kind, reach); + const one = oneLine(s); + assert.equal(/[\n\r\u2028\u2029]/.test(one), false); + assert.equal(/[^\x00-\x7f]/.test(one), false); + const href = markWorker('http://h/w.js?a=1', kind, t); + assert.deepEqual(workerAt(href, one).draws(), workerAt(href, s).draws()); + assert.equal(workerAt(href, one).run('location.href'), 'http://h/w.js?a=1'); + assert.equal(workerAt(href, `"use strict";${one}`).run('location.href'), 'http://h/w.js?a=1'); + } + } + assert.equal(oneLine('a;\n // c\n\n b;'), 'a; b;'); +}); + +test('workerRoutes answers a Web Worker whose script carries a Content-Security-Policy with its seeding on the script\'s first line, loading nothing of keploy\'s, and names a module one, whose seeding runs after its imports, once for each address', async () => { + const { r, context } = routed(); + const late = []; + await workerRoutes(context, { late: (at) => late.push(at) }); + const t = 'cd'.repeat(16); + const policy = { 'content-type': 'text/javascript', 'content-security-policy': "default-src 'none'" }; + const w = await r.ask(markWorker('http://h/w.js', 'w', t), answerOf(200, '"use strict";app()', policy)); + assert.equal(w[1][1].body, `"use strict";${oneLine(workerScript(t, 0, 'w'))}"use strict";app()\n//# sourceURL=http://h/w.js\n`); + assert.deepEqual(w[1][1].headers, policy); + const s = await r.ask(markWorker('http://h/w.js', 'w', t), answerOf(200, 'app()', policy)); + assert.equal(s[1][1].body, `${oneLine(workerScript(t, 1, 'w'))}app()\n//# sourceURL=http://h/w.js\n`); + assert.deepEqual(late, []); + const m = await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'import "./a.js";b()', policy)); + assert.equal(m[1][1].body, `${oneLine(workerScript(t, 2, 'm'))}import "./a.js";b()\n//# sourceURL=http://h/m.js\n`); + assert.deepEqual(late, ['http://h/m.js']); + // With no policy, keploy loads the seeding ahead of the imports, and says nothing. + const m2 = await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'b()')); + assert.equal(m2[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 3))};b()\n//# sourceURL=http://h/m.js\n`); + // A report-only policy refuses nothing. + await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'b()', { 'content-type': 'text/javascript', 'content-security-policy-report-only': "default-src 'none'" })); + assert.deepEqual(late, ['http://h/m.js']); + // A script keploy could not fetch: its seeding's load, as before, and no line. + assert.equal((await r.ask(markWorker('http://h/m.js', 'm', t), new Error('net')))[1][1].body, workerAnswer(t, 5, 'm', 'http://h/m.js')); + assert.deepEqual(late, ['http://h/m.js']); + assert.match(lateLine('a test', 'http://h/m.js'), /^keploy: in a test, the module Web Worker at http:\/\/h\/m\.js is served with a Content-Security-Policy, .* runs after the modules the script imports: .*; if one does, serve the worker's script with no Content-Security-Policy where the tests run/); +}); From b6b7c5af5bb0f3518522723333791fbee34fe7ec Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 13:59:43 +0530 Subject: [PATCH 57/73] fix(playwright): run a redirected worker at the URL its redirect gives keploy's answer to a worker's script followed the server's redirect and served the final script at the worker's first URL, so a classic worker's importScripts and a module worker's imports resolved against the wrong directory, and its location named the first URL. keploy now fetches a worker's script without following a redirect (maxRedirects 0). On a 301, 302, 303, 307 or 308 with a Location, the worker runs where the redirect takes it, as with seeding off, unseeded: Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script there. In Firefox keploy answers with the server's redirect, its Location resolved against the URL keploy asked for, so the mark never reaches the server. Chromium and WebKit follow no redirect a route answers with (WebKit refuses one; Chromium never ends a worker's load of one), so keploy sends the request on as the test's routes asked for it, and the server is asked for the script's first URL twice. A Shared Worker's request sent on is marked answered, as Chromium reports no end for it. The test prints one line per address naming the worker, why it runs unseeded and the next step. Rows: unit rows for each redirect status and kind, a 300 and a 302 with no Location, a fragment-only Location, a route's own headers on the request sent on, and a dropped request; a browser spec in each engine, seeded and with seeding off, for a Latin-1 worker script, four Content-Security-Policies on a worker's script (the module one named as seeded late), and classic, module and Shared Workers redirected to v2/, with the server's counts and no mark at the server. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 3 +- playwright/src/index.ts | 13 +++- playwright/src/random.ts | 88 +++++++++++++++++++++---- playwright/test/fixtures/served.spec.ts | 38 +++++++++++ playwright/test/random.test.mjs | 63 +++++++++++++++++- playwright/test/rows.test.mjs | 79 ++++++++++++++++++++++ 6 files changed, 267 insertions(+), 17 deletions(-) create mode 100644 playwright/test/fixtures/served.spec.ts diff --git a/playwright/README.md b/playwright/README.md index 8ab6fac..cbc54a9 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy, which could refuse that load, the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a Web Worker whose script carries a Content-Security-Policy, where keploy's seeding is on the app's first line: there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy, which could refuse that load, the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a Web Worker whose script carries a Content-Security-Policy, where keploy's seeding is on the app's first line: there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it (a script the server redirects aside, below); the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -123,6 +123,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with keploy's mark (a Trusted Types policy). Next step: start it from its script's http(s) URL. - A worker whose script a route of the test's own answered itself (`route.fulfill`). Next step: have that route call `route.continue()` or `route.fallback()` for the worker's script. - A Shared Worker whose script a route of the test's own sent on with a URL, method, headers or post data of its own (`route.continue` or `route.fallback` with options), in Chromium and WebKit: the seeding's load of the app's script, which these engines do not route, could not carry them. Next step: have that route continue or fall back with no options for the Shared Worker's script. + - A worker whose script the server redirects (a 301, 302, 303, 307 or 308 answer with a `Location`), classic or module, Web or Shared: Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script at the URL the redirect gives. keploy follows no redirect itself: the worker runs at the URL the redirect gives, as with seeding off (its `location`, `import.meta.url` and the URLs its imports resolve against are the same), with no mark. In Firefox, keploy's answer is the server's redirect, with its `Location` resolved against the URL the app asked for. Chromium and WebKit follow no redirect a route answers with (WebKit refuses one, and Chromium never ends a worker's load of one), so keploy sends the request on to the server as the test's routes asked for it, and the server is asked for the script's first URL twice: once by keploy, to see the redirect, and once by the browser. The line says so. Next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect, to seed it. - In WebKit, a worker that a Shared Worker starts draws the browser's own numbers too, as WebKit's route reaches nothing a Shared Worker asks for after its script, but no line names it yet (in Firefox the Shared Worker's own line names it). - **... a replay draws otherwise than its recording.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. The replay's end prints one line for each stream, document and frame path whose count differs from the recording's (above), naming the test, the two counts and the next step. These move a stream: - in the test process, what a worker does once, in whichever test first needs it, when another test needs it first at replay (one test replayed alone, or after keploy set an earlier test aside): a worker fixture registered through `@playwright/test`'s own `test`, or through a test type that `mergeTests` returned, whose `extend` and `use` are Playwright's own (register worker fixtures with this package's `test.extend`, or with `extend` on a test type made from it), and a library that draws `Math.random` the first time it is used inside a test (a pool or a cache it builds on first use: use it once outside the tests). This package keeps two such cases out of the test's stream itself: source maps parsed while a stack trace is prepared, and the setup of worker fixtures registered with its own `test.extend` or `test.use` (above); diff --git a/playwright/src/index.ts b/playwright/src/index.ts index cd175a1..0db660e 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -66,7 +66,7 @@ import type { Outcome, Random, TestRef } from './agent'; import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; -import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, lateLine, othersNow, pageAddress, placePages, processRandom, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; +import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, lateLine, othersNow, pageAddress, placePages, processRandom, redirectLine, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; import { Inflight, sharedWorkerLine, workersScript } from './settle'; export { expect, KeployError }; @@ -268,7 +268,16 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ told.add(k); process.stdout.write(`${lateLine(name, a)}\n`); }, - }, browserName === 'webkit'); + // A worker whose script the server redirects, which so runs + // unseeded: once for each address. + redirected: (at, twice) => { + const a = pageAddress(at); + const k = `redirected\n${a}`; + if (told.has(k)) return; + told.add(k); + process.stdout.write(`${redirectLine(name, a, twice)}\n`); + }, + }, browserName === 'webkit', browserName === 'firefox'); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox', drawnReport) }); const placing = placePages(context, mark); restore = placing.restore; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index d98a30b..1d80cc7 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -645,6 +645,9 @@ export const DOC_KEY = '__keploy_random_doc'; // off before the request leaves the browser. export const WORKER_PARAM = '__keploy_worker'; +/** The statuses a browser follows to the URL their Location gives. */ +const REDIRECTS = new Set([301, 302, 303, 307, 308]); + // The BroadcastChannel, by the context's mark, on which the documents of a // context say the scopes of the Service Worker registrations they make and // find (initScript). @@ -2064,6 +2067,12 @@ export type WorkerHooks = { * Content-Security-Policy: it runs after the modules the script imports. */ late?: (at: string) => void; + /** + * The server redirected the script of the worker at at (its origin and + * path), which so runs unseeded at the URL the redirect gives; twice: the + * server was asked for the script's first URL twice (Chromium, WebKit). + */ + redirected?: (at: string, twice: boolean) => void; }; /** @@ -2078,6 +2087,18 @@ export function lateLine(name: string, at: string): string { 'if one does, serve the worker\'s script with no Content-Security-Policy where the tests run, and keploy loads its seeding ahead of those modules.'; } +/** + * The line a test shows, once for each address, for a worker whose script + * the server redirects (WorkerHooks redirected). + */ +export function redirectLine(name: string, at: string, twice: boolean): string { + return `keploy: in ${name}, the worker at ${at} draws the browser's own random numbers, ${KIDS}: the server redirects its script, ` + + 'and no route reaches the URL a redirect gives (Playwright routes only its first request), so keploy cannot put its seeding ahead of the script; ' + + 'the worker runs at the URL the redirect gives, as with seeding off' + + (twice ? ', and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect)' : '') + + '; serve the worker\'s script where the tests run at the URL the app starts the worker at, with no redirect, to seed it.'; +} + /** What keploy's routes go on: the test's context, or one of its pages. */ type Routes = Pick<BrowserContext, 'route' | 'unroute' | 'unrouteAll'>; @@ -2275,7 +2296,7 @@ function preludeLoad(r: Request, kind: 'p' | 'q'): boolean { * to what they asked for, whole, unseeded, and the test hears of it * (routed, 'continued'). */ -export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}, reach = false): Promise<void> { +export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}, reach = false, follows = false): Promise<void> { const counts = new Map<string, number>(); // The answer's headers, but its encoding and length: the body given is // keploy's, of another length. @@ -2310,7 +2331,9 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // A route's fetch with o is the request the test's routes asked for: o's // URL, method, headers and post data, or the request's (the URL the app // asked for, or one a fallback gave, and what fallbacks gave it). - const answer = async (route: Route, m: WorkerMark, own: Route['fulfill'], h?: Handed, o: Asked = {}): Promise<void> => { + // send sends the request on to the app's server as the test's routes + // asked for it, for the browser to load and follow (a redirect). + const answer = async (route: Route, m: WorkerMark, own: Route['fulfill'], h?: Handed, o: Asked = {}, send: () => Promise<unknown> = () => route.continue()): Promise<void> => { // Once keploy's fulfill has run, nothing of the request is in flight: // one that throws (the worker was terminated while keploy fetched its // script, and the browser dropped the request) ends it too. @@ -2322,6 +2345,20 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = h?.ended(); } }; + // A Web Worker's request sent on is the browser's, in flight until the + // browser says it ended, unless sending it on throws (the browser + // dropped it). A Shared Worker's is answered once sent on: Chromium + // reports no end for it, as for one keploy fulfills. + const sendOn = async (): Promise<void> => { + let sent = false; + try { + await send(); + sent = true; + } finally { + if (!sent || m.kind === 's' || m.kind === 'x') hooks.answered?.(route.request()); + h?.ended(); + } + }; if (h !== undefined) h.started = true; if (m.kind === 'p' || m.kind === 'q') { await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, m.kind === 'p' ? 'm' : 'w', reach) }); @@ -2337,24 +2374,49 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = let status = 200; let headers: Record<string, string> = { 'content-type': 'text/javascript' }; let source: Uint8Array | undefined; + let moved: Awaited<ReturnType<Route['fetch']>> | undefined; try { - const got = await route.fetch(o); - if (whole || !got.ok()) { + // keploy follows no redirect: the browser does (below). + const got = await route.fetch({ ...o, maxRedirects: 0 }); + if (REDIRECTS.has(got.status()) && got.headers().location !== undefined) { + moved = got; + } else if (whole || !got.ok()) { await fulfill({ status: got.status(), headers: headersOf(got.headers()), body: await got.body() }); if (whole) { const u = new URL(m.url); hooks.routed?.(u.origin + u.pathname, 'continued'); } return; + } else { + status = got.status(); + headers = headersOf(got.headers()); + // A Web Worker's answer holds the app's script's bytes as the server + // sent them, whatever their charset (withBytes). + if (m.kind === 'w' || m.kind === 'm') source = await got.body(); } - status = got.status(); - headers = headersOf(got.headers()); - // A Web Worker's answer holds the app's script's bytes as the server - // sent them, whatever their charset (withBytes). - if (m.kind === 'w' || m.kind === 'm') source = await got.body(); } catch { // The worker loads the app's script itself, and fails as it would. } + // A worker whose script the server redirects runs at the URL the + // redirect gives, as with seeding off, where no route reaches its + // script (Playwright routes no redirect's next request), so keploy + // cannot put its seeding there: it runs unseeded, and the test says so. + // Firefox follows a redirect keploy answers with: it gets the server's, + // its Location whole, at the URL keploy asked for, so it never holds + // keploy's mark. Chromium and WebKit do not (WebKit refuses one, and + // Chromium never ends a worker's load of one): the request goes on to + // the server, which so is asked for the script's first URL twice. + if (moved !== undefined) { + const u = new URL(m.url); + hooks.redirected?.(u.origin + u.pathname, !follows); + if (follows) { + const to = new URL(moved.headers().location ?? '', moved.url()).href; + await fulfill({ status: moved.status(), headers: { ...headersOf(moved.headers()), location: to }, body: await moved.body() }); + } else { + await sendOn(); + } + return; + } // A Content-Security-Policy on the script's response is the worker's // own: it could refuse the load of keploy's seeding at its own URL // (importScripts or import), so the seeding goes on the app's first @@ -2419,7 +2481,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = return own.continue(o); } ask(h, o); - await answer(route, m, own.fulfill, h, o); + await answer(route, m, own.fulfill, h, o, () => own.continue(o)); }; route.fallback = async (o) => { if (h.gone) return own.fallback(o).catch(() => {}); @@ -2479,7 +2541,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await h.own.fallback(); return; } - await answer(route, h.m, h.own.fulfill, h); + await answer(route, h.m, h.own.fulfill, h, {}, () => h.own.continue()); }; // What Playwright does with the requests a route of the test's still held // when the test took the last route of a page, or of its context, off: @@ -2511,7 +2573,9 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } else if (!h.started) { h.gone = true; const fulfill = q._innerFulfill; - ends.push(answer(route, h.m, typeof fulfill === 'function' ? (o) => fulfill.call(route, o) : h.own.fulfill, h).catch(() => {})); + const sendOn = q._innerContinue; + ends.push(answer(route, h.m, typeof fulfill === 'function' ? (o) => fulfill.call(route, o) : h.own.fulfill, h, {}, + typeof sendOn === 'function' ? () => sendOn.call(route, false) : () => h.own.continue()).catch(() => {})); } else { ends.push(h.end); } diff --git a/playwright/test/fixtures/served.spec.ts b/playwright/test/fixtures/served.spec.ts new file mode 100644 index 0000000..5c3185b --- /dev/null +++ b/playwright/test/fixtures/served.spec.ts @@ -0,0 +1,38 @@ +import fs from 'node:fs'; +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER), seeded and with seeding +// off. Each test starts, from the page at /sv, workers whose scripts the +// server serves in a way of its own (a charset, a Content-Security-Policy, +// a redirect), one at a time, and keeps what each sent (its kind, draws and +// location, and what it read), or 'error' or 'none', under its title. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +test.use({ serviceWorkers: 'block' }); + +type Start = { url: string; type?: 'classic' | 'module'; shared?: boolean }; +const start = async (page: Page, list: Start[]): Promise<void> => { + await page.goto(`${base}/sv`); + const got: unknown[] = []; + for (const s of list) { + got.push(await page.evaluate((x) => new Promise((resolve) => { + setTimeout(() => resolve('none'), 10000); + if (x.shared) { + const w = new SharedWorker(x.url, { type: x.type ?? 'classic' }); + w.port.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + return; + } + const w = new Worker(x.url, { type: x.type ?? 'classic' }); + w.onmessage = (e) => resolve(e.data); + w.onerror = () => resolve('error'); + }), s)); + } + keep(test.info().title, got); +}; + +test('latin1', async ({ page }) => start(page, [{ url: '/sv/l1.js' }, { url: '/sv/l1.js', type: 'module' }])); +test('csp', async ({ page }) => start(page, [{ url: '/sv/cn.js' }, { url: '/sv/csn.js' }, { url: '/sv/ctt.js' }, { url: '/sv/cn.js', type: 'module' }])); +test('redirect', async ({ page }) => start(page, [{ url: '/rd/w.js' }, { url: '/rdm/m.js', type: 'module' }, { url: '/rds/s.js', shared: true }])); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 880daf2..009bf35 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, withBytes, oneLine, lateLine, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, withBytes, oneLine, lateLine, redirectLine, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -2080,10 +2080,15 @@ function routed() { const h = o.headers ?? over?.headers; did.push(h === undefined ? ['fetch', o.url ?? at] : ['fetch', o.url ?? at, h]); if (answer instanceof Error) throw answer; + if (answer.followed !== undefined && o.maxRedirects !== 0) return answer.followed; return answer; }, fulfill: async (o) => { ended = true; r.raw = o.body; did.push(['fulfill', { ...o, body: String(o.body) }]); }, - continue: async (o) => { ended = true; did.push(['continue', o?.url ?? at]); }, + continue: async (o) => { + if (as.dropped) throw new Error('Target closed'); + ended = true; + did.push(o?.headers === undefined ? ['continue', o?.url ?? at] : ['continue', o?.url ?? at, o.headers]); + }, abort: async () => { ended = true; did.push(['abort']); }, fallback: async (o) => { if (o === undefined || Object.keys(o).length === 0) return; @@ -2364,6 +2369,60 @@ test('workerRoutes keeps keploy\'s route ahead of every route the test adds to i assert.equal(context.routes[2].match(new URL('http://h/v.js')), false); }); +// A redirect answer: status with its Location at, at URL u, and what +// a fetch that follows it gets (followed). +const movedOf = (status, at, u, followed = answerOf(200, 'app()')) => ({ ...answerOf(status, '', { location: at, 'content-length': '0', 'x-r': '1' }), url: () => u, followed }); + +test('workerRoutes follows no redirect of a worker\'s script: Firefox (follows) gets the server\'s redirect, its Location whole at the URL keploy asked for; Chromium and WebKit send the request on as the test\'s routes asked; each kind runs unseeded, and the test hears of it once for each load, with twice off Firefox; a 3xx that is no redirect, or one with no Location, goes to the worker as the server gave it', async () => { + const t = 'ab'.repeat(16); + for (const follows of [false, true]) { + const { r, context } = routed(); + const heard = []; + const answered = []; + await workerRoutes(context, { redirected: (at, twice) => heard.push([at, twice]), answered: (q) => answered.push(q.url()) }, false, follows); + for (const [kind, status] of [['w', 301], ['m', 302], ['s', 303], ['x', 307], ['w', 308]]) { + const before = answered.length; + const got = await r.ask(markWorker('http://h/rd/w.js?a=1', kind, t), movedOf(status, 'v2/w.js', 'http://h/rd/w.js?a=1')); + assert.deepEqual(got, follows + ? [['fetch', 'http://h/rd/w.js?a=1'], ['fulfill', { status, headers: { location: 'http://h/rd/v2/w.js', 'x-r': '1' }, body: '' }]] + : [['fetch', 'http://h/rd/w.js?a=1'], ['continue', 'http://h/rd/w.js?a=1']], `${kind} ${status} ${follows}`); + // Answered: each one keploy fulfilled; of those sent on, a Shared Worker's (s, x) only. + assert.equal(answered.length - before, follows || kind === 's' || kind === 'x' ? 1 : 0, `${kind} ${status} ${follows}`); + } + assert.deepEqual(heard, Array(5).fill(['http://h/rd/w.js', !follows])); + // A Location that keeps only the fragment: whole, at the URL keploy asked for, never the marked one. + const frag = await r.ask(markWorker('http://h/rd/w.js', 'w', t), movedOf(302, '#x', 'http://h/rd/w.js')); + if (follows) assert.equal(frag[1][1].headers.location, 'http://h/rd/w.js#x'); + // Not redirects: a 300 with a Location, and a 302 with none, go to the worker as the server gave them. + for (const [status, headers] of [[300, { location: 'v2/w.js' }], [302, {}]]) { + const got = await r.ask(markWorker('http://h/rd/w.js', 'w', t), { ...answerOf(status, 'x', headers), url: () => 'http://h/rd/w.js', followed: answerOf(200, 'app()') }); + assert.deepEqual(got, [['fetch', 'http://h/rd/w.js'], ['fulfill', { status, headers, body: 'x' }]], `${status}`); + } + assert.equal(heard.length, 6); + } + // A route of the test's that continues it with headers of its own: keploy's fetch and the request sent on both carry them. + const { r, context } = routed(); + await workerRoutes(context); + const page = r.page(); + await page.route(() => true, (route) => route.continue({ headers: { 'x-auth': 'k' } })); + assert.deepEqual(await r.ask(markWorker('http://h/rd/w.js', 'm', t), movedOf(302, '/v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {} }, page), + [['fetch', 'http://h/rd/w.js', { 'x-auth': 'k' }], ['continue', 'http://h/rd/w.js', { 'x-auth': 'k' }]]); + // A load that sending on fails for (the browser dropped it) ends: answered, and the error goes to the test's continue. + const ended = []; + const { r: r2, context: c2 } = routed(); + await workerRoutes(c2, { answered: () => ended.push('answered') }); + const p2 = r2.page(); + await p2.route(() => true, (route) => route.continue({ headers: { 'x-b': '1' } }).catch((e) => { ended.push(e.message); route.abort(); })); + await r2.ask(markWorker('http://h/rd/w.js', 'w', t), movedOf(302, 'v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {}, dropped: true }, p2); + assert.deepEqual(ended, ['answered', 'Target closed']); + assert.equal(redirectLine('t', 'http://h/rd/w.js', true), 'keploy: in t, the worker at http://h/rd/w.js draws the browser\'s own random numbers, and so does each worker it starts, so their draws may differ at replay: ' + + 'the server redirects its script, and no route reaches the URL a redirect gives (Playwright routes only its first request), so keploy cannot put its seeding ahead of the script; ' + + 'the worker runs at the URL the redirect gives, as with seeding off, and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect); ' + + 'serve the worker\'s script where the tests run at the URL the app starts the worker at, with no redirect, to seed it.'); + assert.equal(redirectLine('t', 'a', false), redirectLine('t', 'a', true).replace(', and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect)', '')); + assert.notEqual(redirectLine('t', 'a', false), redirectLine('t', 'a', true)); +}); + test('workerRoutes sends a worker\'s script on with what the test\'s continue and fallbacks asked (a URL or headers of their own), at a URL a fallback gave too; a Shared Worker\'s, classic or module, that they asked for otherwise than the browser sent it gets the answer to what they asked, whole, unseeded, and is heard; the browser\'s own headers in another case or with one given as undefined, an option left undefined, or none, seed it', async () => { const { r, context } = routed(); const heard = []; diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index faf45f5..0fa7411 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -688,6 +688,28 @@ async function drawsServer() { if (u.pathname === '/csp') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('csp')); return; } // /cspf: the same, with a frame of its own origin in it. if (u.pathname === '/cspf') { res.writeHead(200, { 'content-type': 'text/html', 'content-security-policy': 'sandbox allow-scripts' }); res.end(page('<iframe src="/frame"></iframe>')); return; } + // served.spec.ts's: /sv starts workers whose scripts the server serves + // in a way of its own: /sv/l1.js in Latin-1 (an e-acute); /sv/cn.js, + // /sv/csn.js and /sv/ctt.js under a Content-Security-Policy each; and + // /rd/w.js, /rdm/m.js (a module) and /rds/s.js (a Shared Worker, by a + // relative Location) redirect to v2/ beside them, where each loads h + // beside it, which says H = 'v2'. + if (u.pathname === '/sv') { res.writeHead(200, { 'content-type': 'text/html' }); res.end(page('sv')); return; } + const told = (k, more = '') => `postMessage({ k: '${k}'${more}, draws: [Math.random(), Math.random()], href: String(location.href) });`; + if (u.pathname === '/sv/l1.js') { res.writeHead(200, { 'content-type': 'text/javascript; charset=iso-8859-1' }); res.end(Buffer.from(told('l1', ", s: 'h\u00e9'"), 'latin1')); return; } + const csp = { '/sv/cn.js': "default-src 'none'", '/sv/csn.js': "script-src 'none'", '/sv/ctt.js': "require-trusted-types-for 'script'" }[u.pathname]; + if (csp !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', 'content-security-policy': csp }); res.end(told(u.pathname.slice(4, -3))); return; } + const moved = { '/rd/w.js': '/rd/v2/w.js', '/rdm/m.js': '/rdm/v2/m.js', '/rds/s.js': 'v2/s.js' }[u.pathname]; + if (moved !== undefined) { res.writeHead(302, { location: moved }); res.end(); return; } + const v2 = { + '/rd/v2/w.js': `importScripts('h.js'); ${told('rd', ', H: self.H')}`, + '/rd/v2/h.js': "self.H = 'v2';", + '/rdm/v2/m.js': `import { H } from './h.mjs'; ${told('rdm', ', H, meta: import.meta.url')}`, + '/rdm/v2/h.mjs': "export const H = 'v2';", + '/rds/v2/s.js': `importScripts('h.js'); onconnect = (e) => e.ports[0].${told('rds', ', H: self.H')}`, + '/rds/v2/h.js': "self.H = 'v2';", + }[u.pathname]; + if (v2 !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end(v2); return; } if (u.pathname === '/held.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); held.script(() => res.end('')); return; } // parse.spec.ts's removed: /rhold.js is answered at /rrelease. if (u.pathname === '/rhold.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); rheld.push(res); return; } @@ -1719,6 +1741,63 @@ for (const engine of ENGINES) { }); } +for (const engine of ENGINES) { + test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and a module one under default-src 'none') runs seeded, its draws the seed's, the module one named as seeded late; one the server redirects (classic, module, and a Shared Worker) runs at the URL the redirect gives, loading what sits beside it there, as off, unseeded and named, the server asked for its first URL ${engine === 'firefox' ? 'once' : 'twice'}; the server never sees keploy's mark`, async () => { + const j = journal(); + const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: 'cd'.repeat(16) } } : undefined)); + try { + const run = (off) => drawRun(j, { ...keployEnv(a.url), PW_BROWSER: engine, ...(off ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }, ['served.spec.ts'], ['latin1', 'csp', 'redirect']); + const s1 = await run(false); + const s2 = await run(false); + const o1 = await run(true); + const o2 = await run(true); + const at = (p) => new RegExp(`^http://127\\.0\\.0\\.1:\\d+${p.replace(/\./g, '\\.')}$`); + for (const [mode, r] of [['seeded', s1], ['seeded again', s2], ['off', o1], ['off again', o2]]) { + assert.deepEqual(r.drew.latin1.map((w) => w.k), ['l1', 'l1'], mode); + assert.deepEqual(r.drew.csp.map((w) => w.k), ['cn', 'csn', 'ctt', 'cn'], mode); + for (const [w, p] of [...r.drew.latin1.map((w) => [w, '/sv/l1.js']), ...r.drew.csp.map((w) => [w, `/sv/${w.k}.js`])]) assert.match(w.href, at(p), mode); + const [rd, rdm, rds] = r.drew.redirect; + assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rds.k, rds.H], ['rd', 'v2', 'rdm', 'v2', 'rds', 'v2'], mode); + assert.match(rd.href, at('/rd/v2/w.js'), mode); + assert.match(rdm.href, at('/rdm/v2/m.js'), mode); + assert.match(rdm.meta, at('/rdm/v2/m.js'), mode); + assert.match(rds.href, at('/rds/v2/s.js'), mode); + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); + // Each Web Worker's script loaded once (two start at each of /sv/l1.js and /sv/cn.js). + assert.deepEqual(['/sv/l1.js', '/sv/cn.js', '/sv/csn.js', '/sv/ctt.js'].map((p) => r.seen.filter((u) => u === p).length), [2, 2, 1, 1], mode); + // A redirect's next URL, once; its first: once off and in Firefox, twice seeded in Chromium and WebKit. + for (const p of ['/rd/w.js', '/rdm/m.js', '/rds/s.js']) { + assert.equal(r.seen.filter((u) => u === p.replace(/\/([^/]+)$/, '/v2/$1')).length, 1, `${mode} ${p}`); + assert.equal(r.seen.filter((u) => u === p).length, r === s1 || r === s2 ? (engine === 'firefox' ? 1 : 2) : 1, `${mode} ${p}`); + } + } + // The Latin-1 text reads as off: Chromium decodes a worker's script by its charset, Firefox and WebKit as UTF-8. + for (const r of [s1, s2, o2]) assert.deepEqual(r.drew.latin1.map((w) => w.s), o1.drew.latin1.map((w) => w.s)); + if (engine === 'chromium') assert.deepEqual(s1.drew.latin1.map((w) => w.s), ['h\u00e9', 'h\u00e9']); + // Seeded: the seed's draws, the same in each run; off, and a redirected worker seeded, the browser's own. + const draws = (r, t) => r.drew[t].map((w) => w.draws); + assert.deepEqual(draws(s1, 'latin1'), draws(s2, 'latin1')); + assert.deepEqual(draws(s1, 'csp'), draws(s2, 'csp')); + assert.notDeepEqual(draws(o1, 'latin1'), draws(o2, 'latin1')); + assert.notDeepEqual(draws(o1, 'csp'), draws(o2, 'csp')); + for (const i of [0, 1, 2]) assert.notDeepEqual(s1.drew.redirect[i].draws, s2.drew.redirect[i].draws, `redirect ${i}`); + // The lines: the module one under a policy, seeded late; each redirected one, once, saying the second ask off Firefox. + const said = (r, re) => r.lines.filter((l) => re.test(l)); + for (const r of [s1, s2]) { + assert.equal(said(r, /^keploy: in .*csp.*, the module Web Worker at http:\/\/127\.0\.0\.1:\d+\/sv\/cn\.js is served with a Content-Security-Policy/).length, 1); + assert.equal(said(r, /Content-Security-Policy/).length, 1); + for (const p of ['/rd/w.js', '/rdm/m.js', '/rds/s.js']) { + const l = said(r, new RegExp(`^keploy: in .*redirect.*, the worker at http://127\\.0\\.0\\.1:\\d+${p.replace(/\./g, '\\.')} draws the browser's own random numbers, .*: the server redirects its script, `)); + assert.equal(l.length, 1, p); + assert.equal(/once more than with seeding off/.test(l[0]), engine !== 'firefox', p); + } + assert.equal(said(r, /redirects its script/).length, 3); + } + for (const r of [o1, o2]) assert.deepEqual(said(r, /redirects its script|Content-Security-Policy/), []); + } finally { await a.close(); } + }); +} + // overrides.spec.ts's tests, and the scripts each one's page starts its // workers at (/ov/<k>w.js and /ov/<k>s.js). const OV = [['continue headers on page', 'a'], ['continue headers on context', 'a'], ['continue url on page', 'o'], ['continue url on context', 'o'], From 7e7b6d3ccfcb6914075b7da75a74524ad99a7ea1 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 14:21:47 +0530 Subject: [PATCH 58/73] test(playwright): expect Firefox's Shared Worker line on a redirect Firefox lets no route reach a Shared Worker's script, so keploy never sees the redirect of /rds/s.js there: the run names that worker with the line for a Shared Worker keploy cannot reach, not the redirect line. The served row now expects two redirect lines and that one line in Firefox, and three redirect lines and no such line in Chromium and WebKit. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/test/rows.test.mjs | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 0fa7411..58314ab 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1786,12 +1786,15 @@ for (const engine of ENGINES) { for (const r of [s1, s2]) { assert.equal(said(r, /^keploy: in .*csp.*, the module Web Worker at http:\/\/127\.0\.0\.1:\d+\/sv\/cn\.js is served with a Content-Security-Policy/).length, 1); assert.equal(said(r, /Content-Security-Policy/).length, 1); - for (const p of ['/rd/w.js', '/rdm/m.js', '/rds/s.js']) { + // Firefox routes no Shared Worker's script: its redirected one is named as one keploy cannot reach. + const moved = engine === 'firefox' ? ['/rd/w.js', '/rdm/m.js'] : ['/rd/w.js', '/rdm/m.js', '/rds/s.js']; + for (const p of moved) { const l = said(r, new RegExp(`^keploy: in .*redirect.*, the worker at http://127\\.0\\.0\\.1:\\d+${p.replace(/\./g, '\\.')} draws the browser's own random numbers, .*: the server redirects its script, `)); assert.equal(l.length, 1, p); assert.equal(/once more than with seeding off/.test(l[0]), engine !== 'firefox', p); } - assert.equal(said(r, /redirects its script/).length, 3); + assert.equal(said(r, /redirects its script/).length, moved.length); + assert.equal(said(r, /^keploy: in .*redirect.*, the Shared Worker at http:\/\/127\.0\.0\.1:\d+\/rds\/s\.js draws the browser's own random numbers, .*: it is a Shared Worker, and Firefox does not let keploy reach/).length, engine === 'firefox' ? 1 : 0); } for (const r of [o1, o2]) assert.deepEqual(said(r, /redirects its script|Content-Security-Policy/), []); } finally { await a.close(); } From 0de7a5758fff2ece45f4a0e43eb00cc2b18533c0 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 18:07:30 +0530 Subject: [PATCH 59/73] fix(playwright): stop a redirected worker script's load in Chromium In Chromium, a seeded worker whose script the server redirects never started: keploy sent the load on at the unmarked URL, and Playwright sends a request whose URL a route changed to that URL again on each of the redirect's hops, so the server was asked for the first URL until the browser gave up and the worker failed. A test's own route.continue({ url }) on a worker's script does the same with no keploy; with seeding off and no such route the worker runs. keploy cannot take its mark off without changing the URL, so in Chromium it now stops the load (net::ERR_BLOCKED_BY_CLIENT) once its own fetch has seen the redirect: the server is asked for the script once, and the worker fails to load. The test prints one line per address naming the worker, why it does not start and the next step (serve the script with no redirect, turn seeding off for that test, or run it in Firefox or WebKit). Firefox and WebKit keep their answers. workerRoutes takes the browser's name in place of two flags. A load held by a route the test takes off is sent on as Playwright sends on a request it held (a fallback's continue), or stopped past Playwright's API, so the route's later call gets Playwright's own error. Rows: unit rows per engine for each redirect status and kind, through a route of the test's that continues or falls back, a dropped load, and the release as the test takes its route off (each bounded, so an answer that never ends a load fails the row); the served browser row in Chromium: each redirected worker fails seeded and is named, the server asked for its first URL once and never for the next. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 3 +- playwright/src/index.ts | 8 +- playwright/src/random.ts | 86 +++++++++++++++++----- playwright/test/random.test.mjs | 125 ++++++++++++++++++++++---------- playwright/test/rows.test.mjs | 36 +++++---- 5 files changed, 184 insertions(+), 74 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index cbc54a9..273e63a 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -123,8 +123,9 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with keploy's mark (a Trusted Types policy). Next step: start it from its script's http(s) URL. - A worker whose script a route of the test's own answered itself (`route.fulfill`). Next step: have that route call `route.continue()` or `route.fallback()` for the worker's script. - A Shared Worker whose script a route of the test's own sent on with a URL, method, headers or post data of its own (`route.continue` or `route.fallback` with options), in Chromium and WebKit: the seeding's load of the app's script, which these engines do not route, could not carry them. Next step: have that route continue or fall back with no options for the Shared Worker's script. - - A worker whose script the server redirects (a 301, 302, 303, 307 or 308 answer with a `Location`), classic or module, Web or Shared: Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script at the URL the redirect gives. keploy follows no redirect itself: the worker runs at the URL the redirect gives, as with seeding off (its `location`, `import.meta.url` and the URLs its imports resolve against are the same), with no mark. In Firefox, keploy's answer is the server's redirect, with its `Location` resolved against the URL the app asked for. Chromium and WebKit follow no redirect a route answers with (WebKit refuses one, and Chromium never ends a worker's load of one), so keploy sends the request on to the server as the test's routes asked for it, and the server is asked for the script's first URL twice: once by keploy, to see the redirect, and once by the browser. The line says so. Next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect, to seed it. + - In Firefox and WebKit, a worker whose script the server redirects (a 301, 302, 303, 307 or 308 answer with a `Location`), classic or module, Web or Shared: Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script at the URL the redirect gives. keploy follows no redirect itself: the worker runs at the URL the redirect gives, as with seeding off (its `location`, `import.meta.url` and the URLs its imports resolve against are the same), with no mark. In Firefox, keploy's answer is the server's redirect, with its `Location` resolved against the URL the app asked for. WebKit follows no redirect a route answers with (it refuses one), so keploy sends the request on to the server as the test's routes asked for it, and the server is asked for the script's first URL twice: once by keploy, to see the redirect, and once by the browser. The line says so. Next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect, to seed it. In Chromium such a worker does not start (below). - In WebKit, a worker that a Shared Worker starts draws the browser's own numbers too, as WebKit's route reaches nothing a Shared Worker asks for after its script, but no line names it yet (in Firefox the Shared Worker's own line names it). +- **... in Chromium, the server redirects a seeded worker's script.** The worker does not start, where with seeding off it runs at the URL the redirect gives. Chromium never ends a worker's load of a redirect that a route answers with, and Playwright follows no redirect of a request whose URL a route changed: on each hop it asks for the first URL again, until the browser gives up (a test's own `route.continue({ url })` on a worker's script meets the same, with no keploy). keploy changes the URL of a seeded worker's script load, to take its mark off before the request leaves the browser, so in Chromium keploy stops that load (`net::ERR_BLOCKED_BY_CLIENT`) after asking the server for the script once, to see the redirect, and the worker fails to load (its `error` event). This holds for classic and module Web Workers and for Shared Workers, at record and at replay alike. The test prints `keploy: in <test>, the worker at <address> does not start: the server redirects its script, ...` once for each address, with why and the next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect (keploy then seeds it), turn seeding off for that test (`test.use({ keploySeedRandom: false })`), or run it in Firefox or WebKit. - **... a replay draws otherwise than its recording.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. The replay's end prints one line for each stream, document and frame path whose count differs from the recording's (above), naming the test, the two counts and the next step. These move a stream: - in the test process, what a worker does once, in whichever test first needs it, when another test needs it first at replay (one test replayed alone, or after keploy set an earlier test aside): a worker fixture registered through `@playwright/test`'s own `test`, or through a test type that `mergeTests` returned, whose `extend` and `use` are Playwright's own (register worker fixtures with this package's `test.extend`, or with `extend` on a test type made from it), and a library that draws `Math.random` the first time it is used inside a test (a pool or a cache it builds on first use: use it once outside the tests). This package keeps two such cases out of the test's stream itself: source maps parsed while a stack trace is prepared, and the setup of worker fixtures registered with its own `test.extend` or `test.use` (above); - in the test process, a draw made a varying number of times (a retry loop's jitter): one stream serves all of a test's `Math.random` draws, its own and those of the code it calls, Playwright's multipart form boundary among them; diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 0db660e..17536a8 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -269,15 +269,15 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ process.stdout.write(`${lateLine(name, a)}\n`); }, // A worker whose script the server redirects, which so runs - // unseeded: once for each address. - redirected: (at, twice) => { + // unseeded, or in Chromium does not start: once for each address. + redirected: (at, how) => { const a = pageAddress(at); const k = `redirected\n${a}`; if (told.has(k)) return; told.add(k); - process.stdout.write(`${redirectLine(name, a, twice)}\n`); + process.stdout.write(`${redirectLine(name, a, how)}\n`); }, - }, browserName === 'webkit', browserName === 'firefox'); + }, browserName); await context.addInitScript({ content: initScript(s.seed, report, mark, browserName !== 'firefox', drawnReport) }); const placing = placePages(context, mark); restore = placing.restore; diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 1d80cc7..87f03f8 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -648,6 +648,15 @@ export const WORKER_PARAM = '__keploy_worker'; /** The statuses a browser follows to the URL their Location gives. */ const REDIRECTS = new Set([301, 302, 303, 307, 308]); +/** The browser a test runs in, as Playwright names it. */ +export type Engine = 'chromium' | 'firefox' | 'webkit'; + +/** + * The error keploy stops a redirected worker script's load with in Chromium + * (net::ERR_BLOCKED_BY_CLIENT: keploy, a client of the browser, stopped it). + */ +export const STOPPED = 'blockedbyclient'; + // The BroadcastChannel, by the context's mark, on which the documents of a // context say the scopes of the Service Worker registrations they make and // find (initScript). @@ -2069,12 +2078,17 @@ export type WorkerHooks = { late?: (at: string) => void; /** * The server redirected the script of the worker at at (its origin and - * path), which so runs unseeded at the URL the redirect gives; twice: the - * server was asked for the script's first URL twice (Chromium, WebKit). + * path). 'followed' (Firefox): it runs unseeded at the URL the redirect + * gives; 'twice' (WebKit): so too, and the server was asked for the + * script's first URL twice; 'stopped' (Chromium): keploy stopped its load, + * so the worker does not start (redirectLine says why). */ - redirected?: (at: string, twice: boolean) => void; + redirected?: (at: string, how: Redirected) => void; }; +/** What became of a worker whose script the server redirects (WorkerHooks redirected). */ +export type Redirected = 'followed' | 'twice' | 'stopped'; + /** * The line a test shows, once for each address, for a module Web Worker * whose seeding started after the modules its script imports (WorkerHooks @@ -2091,11 +2105,19 @@ export function lateLine(name: string, at: string): string { * The line a test shows, once for each address, for a worker whose script * the server redirects (WorkerHooks redirected). */ -export function redirectLine(name: string, at: string, twice: boolean): string { +export function redirectLine(name: string, at: string, how: Redirected): string { + if (how === 'stopped') { + return `keploy: in ${name}, the worker at ${at} does not start: the server redirects its script, and in Chromium Playwright follows no redirect ` + + 'of a request whose URL a route changed (it asks for the first URL again until the browser gives up, as for a test\'s own route.continue({ url })), ' + + 'while keploy changes the URL of a seeded worker\'s script load to take its mark off before the request leaves the browser; ' + + 'so keploy stopped the load after asking the server for the script once, to see the redirect, and the worker fails to load (its error event), ' + + 'where with seeding off it runs at the URL the redirect gives; to run it, serve the worker\'s script where the tests run at the URL the app starts ' + + 'the worker at, with no redirect (keploy then seeds it), turn seeding off for this test (test.use({ keploySeedRandom: false })), or run it in Firefox or WebKit.'; + } return `keploy: in ${name}, the worker at ${at} draws the browser's own random numbers, ${KIDS}: the server redirects its script, ` + 'and no route reaches the URL a redirect gives (Playwright routes only its first request), so keploy cannot put its seeding ahead of the script; ' + 'the worker runs at the URL the redirect gives, as with seeding off' + - (twice ? ', and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect)' : '') + + (how === 'twice' ? ', and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect)' : '') + '; serve the worker\'s script where the tests run at the URL the app starts the worker at, with no redirect, to seed it.'; } @@ -2295,8 +2317,13 @@ function preludeLoad(r: Request, kind: 'p' | 'q'): boolean { * test's routes asked for otherwise than the browser sent it gets the answer * to what they asked for, whole, unseeded, and the test hears of it * (routed, 'continued'). + * + * engine is the browser the test runs in: WebKit lets the route reach the + * scripts of the workers a seeded Web Worker starts, and each browser does + * its own with a worker script the server redirects (answer). */ -export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}, reach = false, follows = false): Promise<void> { +export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = {}, engine: Engine = 'chromium'): Promise<void> { + const reach = engine === 'webkit'; const counts = new Map<string, number>(); // The answer's headers, but its encoding and length: the body given is // keploy's, of another length. @@ -2332,8 +2359,10 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // URL, method, headers and post data, or the request's (the URL the app // asked for, or one a fallback gave, and what fallbacks gave it). // send sends the request on to the app's server as the test's routes - // asked for it, for the browser to load and follow (a redirect). - const answer = async (route: Route, m: WorkerMark, own: Route['fulfill'], h?: Handed, o: Asked = {}, send: () => Promise<unknown> = () => route.continue()): Promise<void> => { + // asked for it, for the browser to load and follow (a redirect); stop + // fails the request, with STOPPED. + const answer = async (route: Route, m: WorkerMark, own: Route['fulfill'], h?: Handed, o: Asked = {}, + send: () => Promise<unknown> = () => route.continue(), stop: () => Promise<unknown> = () => route.abort(STOPPED)): Promise<void> => { // Once keploy's fulfill has run, nothing of the request is in flight: // one that throws (the worker was terminated while keploy fetched its // script, and the browser dropped the request) ends it too. @@ -2359,6 +2388,15 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = h?.ended(); } }; + // Once keploy stopped a request, nothing of it is in flight. + const stopped = async (): Promise<void> => { + try { + await stop(); + } finally { + hooks.answered?.(route.request()); + h?.ended(); + } + }; if (h !== undefined) h.started = true; if (m.kind === 'p' || m.kind === 'q') { await fulfill({ status: 200, contentType: 'text/javascript', body: workerScript(m.token, m.n, m.kind === 'p' ? 'm' : 'w', reach) }); @@ -2403,17 +2441,25 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // cannot put its seeding there: it runs unseeded, and the test says so. // Firefox follows a redirect keploy answers with: it gets the server's, // its Location whole, at the URL keploy asked for, so it never holds - // keploy's mark. Chromium and WebKit do not (WebKit refuses one, and - // Chromium never ends a worker's load of one): the request goes on to - // the server, which so is asked for the script's first URL twice. + // keploy's mark. WebKit refuses one: the request goes on to the server, + // which so is asked for the script's first URL twice. Chromium never + // ends a worker's load of one, and a request sent on loops: keploy's + // first route took the mark off the request's URL, and on each of the + // redirect's hops Playwright sends the request on to that URL again, + // until the browser gives up. keploy stops it instead, and the test + // says why. if (moved !== undefined) { const u = new URL(m.url); - hooks.redirected?.(u.origin + u.pathname, !follows); - if (follows) { + if (engine === 'firefox') { + hooks.redirected?.(u.origin + u.pathname, 'followed'); const to = new URL(moved.headers().location ?? '', moved.url()).href; await fulfill({ status: moved.status(), headers: { ...headersOf(moved.headers()), location: to }, body: await moved.body() }); - } else { + } else if (engine === 'webkit') { + hooks.redirected?.(u.origin + u.pathname, 'twice'); await sendOn(); + } else { + hooks.redirected?.(u.origin + u.pathname, 'stopped'); + await stopped(); } return; } @@ -2481,7 +2527,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = return own.continue(o); } ask(h, o); - await answer(route, m, own.fulfill, h, o, () => own.continue(o)); + await answer(route, m, own.fulfill, h, o, () => own.continue(o), () => own.abort(STOPPED)); }; route.fallback = async (o) => { if (h.gone) return own.fallback(o).catch(() => {}); @@ -2541,7 +2587,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = await h.own.fallback(); return; } - await answer(route, h.m, h.own.fulfill, h, {}, () => h.own.continue()); + await answer(route, h.m, h.own.fulfill, h, {}, () => h.own.continue(), () => h.own.abort(STOPPED)); }; // What Playwright does with the requests a route of the test's still held // when the test took the last route of a page, or of its context, off: @@ -2559,7 +2605,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = // holds it: that route's continue, fulfill or abort then fails at the // browser's end, with the error it gets with no keploy, and its fallback // goes on to the next route. - type Quiet = { _innerFulfill?: Route['fulfill']; _innerContinue?: (fallback: boolean) => Promise<unknown> }; + type Quiet = { _innerFulfill?: Route['fulfill']; _innerContinue?: (fallback: boolean) => Promise<unknown>; _channel?: { abort?: (o: { errorCode: string }) => Promise<unknown> } }; const release = async (removed: Held[], sent: (r: Request) => boolean = () => true): Promise<void> => { const ends: Promise<unknown>[] = []; for (const x of removed) { @@ -2574,8 +2620,12 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = h.gone = true; const fulfill = q._innerFulfill; const sendOn = q._innerContinue; + const abort = q._channel?.abort; + // Sent on as Playwright sends on a request it held (a fallback's + // continue: no 'continued' in its trace), or stopped. ends.push(answer(route, h.m, typeof fulfill === 'function' ? (o) => fulfill.call(route, o) : h.own.fulfill, h, {}, - typeof sendOn === 'function' ? () => sendOn.call(route, false) : () => h.own.continue()).catch(() => {})); + typeof sendOn === 'function' ? () => sendOn.call(route, true) : () => h.own.continue(), + typeof abort === 'function' ? () => abort({ errorCode: STOPPED }) : () => h.own.abort(STOPPED)).catch(() => {})); } else { ends.push(h.end); } diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 009bf35..f3f4168 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, withBytes, oneLine, lateLine, redirectLine, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, withBytes, oneLine, lateLine, redirectLine, STOPPED, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -2089,7 +2089,11 @@ function routed() { ended = true; did.push(o?.headers === undefined ? ['continue', o?.url ?? at] : ['continue', o?.url ?? at, o.headers]); }, - abort: async () => { ended = true; did.push(['abort']); }, + abort: async (e) => { + if (as.dropped) throw new Error('Target closed'); + ended = true; + did.push(e === undefined ? ['abort'] : ['abort', e]); + }, fallback: async (o) => { if (o === undefined || Object.keys(o).length === 0) return; over = { ...over, ...o }; @@ -2369,30 +2373,40 @@ test('workerRoutes keeps keploy\'s route ahead of every route the test adds to i assert.equal(context.routes[2].match(new URL('http://h/v.js')), false); }); +// x, or 'still waiting' if it has not settled within a bounded number of +// turns: a fake route answers at once, so an answer that never ends the load +// it began fails a row instead of hanging it. +const ends = async (x) => Promise.race([x, (async () => { for (let i = 0; i < 200; i++) await turn(); return 'still waiting'; })()]); + // A redirect answer: status with its Location at, at URL u, and what // a fetch that follows it gets (followed). const movedOf = (status, at, u, followed = answerOf(200, 'app()')) => ({ ...answerOf(status, '', { location: at, 'content-length': '0', 'x-r': '1' }), url: () => u, followed }); -test('workerRoutes follows no redirect of a worker\'s script: Firefox (follows) gets the server\'s redirect, its Location whole at the URL keploy asked for; Chromium and WebKit send the request on as the test\'s routes asked; each kind runs unseeded, and the test hears of it once for each load, with twice off Firefox; a 3xx that is no redirect, or one with no Location, goes to the worker as the server gave it', async () => { +test('workerRoutes follows no redirect of a worker\'s script: Firefox gets the server\'s redirect, its Location whole at the URL keploy asked for; WebKit sends the request on as the test\'s routes asked; Chromium stops it (blocked by client), as one sent on would loop; the test hears of each load, with how; a 3xx that is no redirect, or one with no Location, goes to the worker as the server gave it', async () => { const t = 'ab'.repeat(16); - for (const follows of [false, true]) { + assert.equal(STOPPED, 'blockedbyclient'); + for (const engine of ['chromium', 'firefox', 'webkit']) { const { r, context } = routed(); const heard = []; const answered = []; - await workerRoutes(context, { redirected: (at, twice) => heard.push([at, twice]), answered: (q) => answered.push(q.url()) }, false, follows); + await workerRoutes(context, { redirected: (at, how) => heard.push([at, how]), answered: (q) => answered.push(q.url()) }, engine); for (const [kind, status] of [['w', 301], ['m', 302], ['s', 303], ['x', 307], ['w', 308]]) { const before = answered.length; const got = await r.ask(markWorker('http://h/rd/w.js?a=1', kind, t), movedOf(status, 'v2/w.js', 'http://h/rd/w.js?a=1')); - assert.deepEqual(got, follows - ? [['fetch', 'http://h/rd/w.js?a=1'], ['fulfill', { status, headers: { location: 'http://h/rd/v2/w.js', 'x-r': '1' }, body: '' }]] - : [['fetch', 'http://h/rd/w.js?a=1'], ['continue', 'http://h/rd/w.js?a=1']], `${kind} ${status} ${follows}`); - // Answered: each one keploy fulfilled; of those sent on, a Shared Worker's (s, x) only. - assert.equal(answered.length - before, follows || kind === 's' || kind === 'x' ? 1 : 0, `${kind} ${status} ${follows}`); + const then = { + firefox: ['fulfill', { status, headers: { location: 'http://h/rd/v2/w.js', 'x-r': '1' }, body: '' }], + webkit: ['continue', 'http://h/rd/w.js?a=1'], + chromium: ['abort', 'blockedbyclient'], + }[engine]; + assert.deepEqual(got, [['fetch', 'http://h/rd/w.js?a=1'], then], `${kind} ${status} ${engine}`); + // Answered: each one keploy fulfilled or stopped; of those sent on, a Shared Worker's (s, x) only. + assert.equal(answered.length - before, engine !== 'webkit' || kind === 's' || kind === 'x' ? 1 : 0, `${kind} ${status} ${engine}`); } - assert.deepEqual(heard, Array(5).fill(['http://h/rd/w.js', !follows])); + const how = { firefox: 'followed', webkit: 'twice', chromium: 'stopped' }[engine]; + assert.deepEqual(heard, Array(5).fill(['http://h/rd/w.js', how])); // A Location that keeps only the fragment: whole, at the URL keploy asked for, never the marked one. const frag = await r.ask(markWorker('http://h/rd/w.js', 'w', t), movedOf(302, '#x', 'http://h/rd/w.js')); - if (follows) assert.equal(frag[1][1].headers.location, 'http://h/rd/w.js#x'); + if (engine === 'firefox') assert.equal(frag[1][1].headers.location, 'http://h/rd/w.js#x'); // Not redirects: a 300 with a Location, and a 302 with none, go to the worker as the server gave them. for (const [status, headers] of [[300, { location: 'v2/w.js' }], [302, {}]]) { const got = await r.ask(markWorker('http://h/rd/w.js', 'w', t), { ...answerOf(status, 'x', headers), url: () => 'http://h/rd/w.js', followed: answerOf(200, 'app()') }); @@ -2400,27 +2414,44 @@ test('workerRoutes follows no redirect of a worker\'s script: Firefox (follows) } assert.equal(heard.length, 6); } - // A route of the test's that continues it with headers of its own: keploy's fetch and the request sent on both carry them. - const { r, context } = routed(); - await workerRoutes(context); - const page = r.page(); - await page.route(() => true, (route) => route.continue({ headers: { 'x-auth': 'k' } })); - assert.deepEqual(await r.ask(markWorker('http://h/rd/w.js', 'm', t), movedOf(302, '/v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {} }, page), - [['fetch', 'http://h/rd/w.js', { 'x-auth': 'k' }], ['continue', 'http://h/rd/w.js', { 'x-auth': 'k' }]]); - // A load that sending on fails for (the browser dropped it) ends: answered, and the error goes to the test's continue. - const ended = []; - const { r: r2, context: c2 } = routed(); - await workerRoutes(c2, { answered: () => ended.push('answered') }); - const p2 = r2.page(); - await p2.route(() => true, (route) => route.continue({ headers: { 'x-b': '1' } }).catch((e) => { ended.push(e.message); route.abort(); })); - await r2.ask(markWorker('http://h/rd/w.js', 'w', t), movedOf(302, 'v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {}, dropped: true }, p2); - assert.deepEqual(ended, ['answered', 'Target closed']); - assert.equal(redirectLine('t', 'http://h/rd/w.js', true), 'keploy: in t, the worker at http://h/rd/w.js draws the browser\'s own random numbers, and so does each worker it starts, so their draws may differ at replay: ' + + // A route of the test's that continues it with headers of its own: keploy's fetch carries them, and in WebKit the request sent on too; + // in Chromium keploy stops it through that route. + for (const engine of ['webkit', 'chromium']) { + const { r, context } = routed(); + await workerRoutes(context, {}, engine); + const page = r.page(); + await page.route(() => true, (route) => route.continue({ headers: { 'x-auth': 'k' } })); + assert.deepEqual(await r.ask(markWorker('http://h/rd/w.js', 'm', t), movedOf(302, '/v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {} }, page), + [['fetch', 'http://h/rd/w.js', { 'x-auth': 'k' }], engine === 'webkit' ? ['continue', 'http://h/rd/w.js', { 'x-auth': 'k' }] : ['abort', 'blockedbyclient']], engine); + // A route of the test's that falls back: keploy's last route sends it on (WebKit) or stops it (Chromium). + const { r: rf, context: cf } = routed(); + await workerRoutes(cf, {}, engine); + const pf = rf.page(); + await pf.route(() => true, (route) => route.fallback()); + assert.deepEqual(await rf.ask(markWorker('http://h/rd/w.js', 'w', t), movedOf(302, 'v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {} }, pf), + [['fetch', 'http://h/rd/w.js'], engine === 'webkit' ? ['continue', 'http://h/rd/w.js'] : ['abort', 'blockedbyclient']], `${engine} fallback`); + // A load that sending on or stopping fails for (the browser dropped it) ends: answered, and the error goes to the test's continue. + const ended = []; + const { r: r2, context: c2 } = routed(); + await workerRoutes(c2, { answered: () => ended.push('answered') }, engine); + const p2 = r2.page(); + await p2.route(() => true, (route) => route.continue({ headers: { 'x-b': '1' } }).catch((e) => { ended.push(e.message); })); + const asked = await ends(r2.ask(markWorker('http://h/rd/w.js', 'w', t), movedOf(302, 'v2/w.js', 'http://h/rd/w.js'), { type: 'script', headers: {}, dropped: true }, p2)); + assert.notEqual(asked, 'still waiting', engine); + assert.deepEqual(ended, ['answered', 'Target closed'], engine); + } + assert.equal(redirectLine('t', 'http://h/rd/w.js', 'twice'), 'keploy: in t, the worker at http://h/rd/w.js draws the browser\'s own random numbers, and so does each worker it starts, so their draws may differ at replay: ' + 'the server redirects its script, and no route reaches the URL a redirect gives (Playwright routes only its first request), so keploy cannot put its seeding ahead of the script; ' + 'the worker runs at the URL the redirect gives, as with seeding off, and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect); ' + 'serve the worker\'s script where the tests run at the URL the app starts the worker at, with no redirect, to seed it.'); - assert.equal(redirectLine('t', 'a', false), redirectLine('t', 'a', true).replace(', and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect)', '')); - assert.notEqual(redirectLine('t', 'a', false), redirectLine('t', 'a', true)); + assert.equal(redirectLine('t', 'a', 'followed'), redirectLine('t', 'a', 'twice').replace(', and the server was asked for the script\'s first URL once more than with seeding off (keploy asked it, to see the redirect)', '')); + assert.notEqual(redirectLine('t', 'a', 'followed'), redirectLine('t', 'a', 'twice')); + assert.equal(redirectLine('t', 'http://h/rd/w.js', 'stopped'), 'keploy: in t, the worker at http://h/rd/w.js does not start: the server redirects its script, and in Chromium Playwright follows no redirect ' + + 'of a request whose URL a route changed (it asks for the first URL again until the browser gives up, as for a test\'s own route.continue({ url })), ' + + 'while keploy changes the URL of a seeded worker\'s script load to take its mark off before the request leaves the browser; ' + + 'so keploy stopped the load after asking the server for the script once, to see the redirect, and the worker fails to load (its error event), ' + + 'where with seeding off it runs at the URL the redirect gives; to run it, serve the worker\'s script where the tests run at the URL the app starts ' + + 'the worker at, with no redirect (keploy then seeds it), turn seeding off for this test (test.use({ keploySeedRandom: false })), or run it in Firefox or WebKit.'); }); test('workerRoutes sends a worker\'s script on with what the test\'s continue and fallbacks asked (a URL or headers of their own), at a URL a fallback gave too; a Shared Worker\'s, classic or module, that they asked for otherwise than the browser sent it gets the answer to what they asked, whole, unseeded, and is heard; the browser\'s own headers in another case or with one given as undefined, an option left undefined, or none, seed it', async () => { @@ -2558,7 +2589,7 @@ test('workerRoutes answers a Web Worker\'s script with the app\'s script it fetc // Where the route reaches a Web Worker's own workers (reach: WebKit), // the seeding it serves at a p or q mark starts them seeded too. const three = routed(); - await workerRoutes(three.context, {}, true); + await workerRoutes(three.context, {}, 'webkit'); assert.notEqual(workerScript(tw, 0, 'w', true), workerScript(tw, 0, 'w')); assert.deepEqual(await three.r.ask(markWorker('http://h/w.js', 'q', tw, 0), undefined, { type: 'other', headers: {} }), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(tw, 0, 'w', true) }]]); assert.deepEqual(await three.r.ask(markWorker('http://h/m.js', 'p', tm, 0), undefined, { type: 'script', headers: {} }), [['fulfill', { status: 200, contentType: 'text/javascript', body: workerScript(tm, 0, 'm', true) }]]); @@ -2785,7 +2816,7 @@ test('workerRoutes keeps its route on a page and on the context through the test // 'ignoreErrors'), a handler that returned with no answer ('unanswered'), // or the URL the browser sent it to when no route answered it ('sent'). const uwAt = (u) => u.pathname === '/uw.js'; -function played(inner = true) { +function played(inner = true, moved = false) { const log = []; const listeners = []; const target = (name) => { @@ -2840,7 +2871,11 @@ function played(inner = true) { }; const route = { request: () => request, - fetch: async () => { did.push('fetch'); await held; return answerOf(200, 'app()'); }, + fetch: async () => { + did.push('fetch'); + await held; + return moved ? { ...answerOf(302, '', { location: 'v2/uw.js' }), url: () => at } : answerOf(200, 'app()'); + }, fulfill: async (o) => take('fulfill', kind(o)), continue: async () => take('continue', 'continue'), abort: async () => take('abort', 'abort'), @@ -2852,7 +2887,8 @@ function played(inner = true) { }; if (inner) { route._innerFulfill = async (o) => serve('fulfill', kind(o)); - route._innerContinue = async () => serve('continue', ['sent', at]); + route._innerContinue = async (fallback) => serve('continue', fallback === true ? ['sent', at] : ['sent', at, 'continued']); + route._channel = { abort: async (o) => serve('abort', ['stopped', o.errorCode]) }; } const end = (async () => { for (const t of p === null ? [context] : [p, context]) { @@ -2891,9 +2927,9 @@ function played(inner = true) { // on, then the test takes that route off (the page's last of the test's: // unroute, or unrouteAll with behavior), then answers the load (how), or // before (first). -async function takenOff(how, { first = false, off = (p, t) => p.unroute(uwAt, t), other = false, at = markWorker('http://h/uw.js', 'w', 'e1'.repeat(16)), inner = true } = {}) { - const { context, page, send } = played(inner); - await workerRoutes(context); +async function takenOff(how, { first = false, off = (p, t) => p.unroute(uwAt, t), other = false, at = markWorker('http://h/uw.js', 'w', 'e1'.repeat(16)), inner = true, moved = false, engine = 'chromium' } = {}) { + const { context, page, send } = played(inner, moved); + await workerRoutes(context, {}, engine); const p = page(); await turn(); if (other) await p.route(() => true, (route) => route.fallback()); @@ -2943,6 +2979,21 @@ test('workerRoutes, when the test takes off a page\'s last route of its own whil assert.deepEqual(b.did, ['fetch', 'seeded']); const n = await takenOff('continue', { inner: false, at: 'http://h/uw.js' }); assert.deepEqual(n.did, ['continue', ['threw', 'Route is already handled!']]); + // A worker's script the server redirects: WebKit sends it on as Playwright + // sends on a request it held (no 'continued'), Chromium stops it, Firefox + // gets the server's redirect, each past Playwright's API, so the route's + // later continue fails at the browser's end; with no such answers, through + // the route's own, so it fails at once. + // Each answer ends the load, so the route's later call goes on. + for (const [engine, then] of [['webkit', ['sent', 'http://h/uw.js']], ['chromium', ['stopped', 'blockedbyclient']], ['firefox', 'fulfill']]) { + const m = await ends(takenOff('continue', { moved: true, engine })); + assert.notEqual(m, 'still waiting', engine); + assert.deepEqual(m.early, ['fetch', then], engine); + assert.deepEqual(m.did, ['fetch', then, ['threw', 'route.continue: Route is already handled!']], engine); + const own = await ends(takenOff('continue', { moved: true, engine, inner: false })); + assert.notEqual(own, 'still waiting', `${engine} own`); + assert.deepEqual(own.did, ['fetch', { webkit: 'continue', chromium: 'abort', firefox: 'fulfill' }[engine], ['threw', 'Route is already handled!']], `${engine} own`); + } }); test('workerRoutes, when the test takes a route off that began to answer a worker script\'s load (continue, fulfill, abort), or while the page keeps a route of the test\'s, or with unrouteAll\'s \'wait\' from outside the route: that answer goes on as with no keploy, once, and the unroute waits for it', async () => { diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 58314ab..84959bd 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1742,7 +1742,7 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and a module one under default-src 'none') runs seeded, its draws the seed's, the module one named as seeded late; one the server redirects (classic, module, and a Shared Worker) runs at the URL the redirect gives, loading what sits beside it there, as off, unseeded and named, the server asked for its first URL ${engine === 'firefox' ? 'once' : 'twice'}; the server never sees keploy's mark`, async () => { + test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and a module one under default-src 'none') runs seeded, its draws the seed's, the module one named as seeded late; one the server redirects (classic, module, and a Shared Worker) ${engine === 'chromium' ? 'does not start seeded (keploy stops its load, which Chromium would send round the redirect again and again) and is named, the server asked for its first URL once and never for the next, and off it ' : ''}runs at the URL the redirect gives, loading what sits beside it there, as off${engine === 'chromium' ? '' : ', unseeded and named, the server asked for its first URL ' + (engine === 'webkit' ? 'twice' : 'once')}; the server never sees keploy's mark`, async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: 'cd'.repeat(16) } } : undefined)); try { @@ -1756,19 +1756,25 @@ for (const engine of ENGINES) { assert.deepEqual(r.drew.latin1.map((w) => w.k), ['l1', 'l1'], mode); assert.deepEqual(r.drew.csp.map((w) => w.k), ['cn', 'csn', 'ctt', 'cn'], mode); for (const [w, p] of [...r.drew.latin1.map((w) => [w, '/sv/l1.js']), ...r.drew.csp.map((w) => [w, `/sv/${w.k}.js`])]) assert.match(w.href, at(p), mode); - const [rd, rdm, rds] = r.drew.redirect; - assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rds.k, rds.H], ['rd', 'v2', 'rdm', 'v2', 'rds', 'v2'], mode); - assert.match(rd.href, at('/rd/v2/w.js'), mode); - assert.match(rdm.href, at('/rdm/v2/m.js'), mode); - assert.match(rdm.meta, at('/rdm/v2/m.js'), mode); - assert.match(rds.href, at('/rds/v2/s.js'), mode); + // Chromium, seeded: keploy stopped each redirected load, so none starts. + const stops = engine === 'chromium' && (r === s1 || r === s2); + if (stops) { + assert.deepEqual(r.drew.redirect, ['error', 'error', 'error'], mode); + } else { + const [rd, rdm, rds] = r.drew.redirect; + assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rds.k, rds.H], ['rd', 'v2', 'rdm', 'v2', 'rds', 'v2'], mode); + assert.match(rd.href, at('/rd/v2/w.js'), mode); + assert.match(rdm.href, at('/rdm/v2/m.js'), mode); + assert.match(rdm.meta, at('/rdm/v2/m.js'), mode); + assert.match(rds.href, at('/rds/v2/s.js'), mode); + } assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); // Each Web Worker's script loaded once (two start at each of /sv/l1.js and /sv/cn.js). assert.deepEqual(['/sv/l1.js', '/sv/cn.js', '/sv/csn.js', '/sv/ctt.js'].map((p) => r.seen.filter((u) => u === p).length), [2, 2, 1, 1], mode); - // A redirect's next URL, once; its first: once off and in Firefox, twice seeded in Chromium and WebKit. + // A redirect's next URL, once (never where keploy stopped the load); its first: once, but twice seeded in WebKit. for (const p of ['/rd/w.js', '/rdm/m.js', '/rds/s.js']) { - assert.equal(r.seen.filter((u) => u === p.replace(/\/([^/]+)$/, '/v2/$1')).length, 1, `${mode} ${p}`); - assert.equal(r.seen.filter((u) => u === p).length, r === s1 || r === s2 ? (engine === 'firefox' ? 1 : 2) : 1, `${mode} ${p}`); + assert.equal(r.seen.filter((u) => u === p.replace(/\/([^/]+)$/, '/v2/$1')).length, stops ? 0 : 1, `${mode} ${p}`); + assert.equal(r.seen.filter((u) => u === p).length, (r === s1 || r === s2) && engine === 'webkit' ? 2 : 1, `${mode} ${p}`); } } // The Latin-1 text reads as off: Chromium decodes a worker's script by its charset, Firefox and WebKit as UTF-8. @@ -1780,8 +1786,8 @@ for (const engine of ENGINES) { assert.deepEqual(draws(s1, 'csp'), draws(s2, 'csp')); assert.notDeepEqual(draws(o1, 'latin1'), draws(o2, 'latin1')); assert.notDeepEqual(draws(o1, 'csp'), draws(o2, 'csp')); - for (const i of [0, 1, 2]) assert.notDeepEqual(s1.drew.redirect[i].draws, s2.drew.redirect[i].draws, `redirect ${i}`); - // The lines: the module one under a policy, seeded late; each redirected one, once, saying the second ask off Firefox. + if (engine !== 'chromium') for (const i of [0, 1, 2]) assert.notDeepEqual(s1.drew.redirect[i].draws, s2.drew.redirect[i].draws, `redirect ${i}`); + // The lines: the module one under a policy, seeded late; each redirected one, once: in WebKit saying the second ask, in Chromium that it does not start. const said = (r, re) => r.lines.filter((l) => re.test(l)); for (const r of [s1, s2]) { assert.equal(said(r, /^keploy: in .*csp.*, the module Web Worker at http:\/\/127\.0\.0\.1:\d+\/sv\/cn\.js is served with a Content-Security-Policy/).length, 1); @@ -1789,9 +1795,11 @@ for (const engine of ENGINES) { // Firefox routes no Shared Worker's script: its redirected one is named as one keploy cannot reach. const moved = engine === 'firefox' ? ['/rd/w.js', '/rdm/m.js'] : ['/rd/w.js', '/rdm/m.js', '/rds/s.js']; for (const p of moved) { - const l = said(r, new RegExp(`^keploy: in .*redirect.*, the worker at http://127\\.0\\.0\\.1:\\d+${p.replace(/\./g, '\\.')} draws the browser's own random numbers, .*: the server redirects its script, `)); + const how = engine === 'chromium' ? 'does not start' : "draws the browser's own random numbers, .*"; + const l = said(r, new RegExp(`^keploy: in .*redirect.*, the worker at http://127\\.0\\.0\\.1:\\d+${p.replace(/\./g, '\\.')} ${how}: the server redirects its script, `)); assert.equal(l.length, 1, p); - assert.equal(/once more than with seeding off/.test(l[0]), engine !== 'firefox', p); + assert.equal(/once more than with seeding off/.test(l[0]), engine === 'webkit', p); + assert.equal(/stopped the load after asking the server for the script once/.test(l[0]), engine === 'chromium', p); } assert.equal(said(r, /redirects its script/).length, moved.length); assert.equal(said(r, /^keploy: in .*redirect.*, the Shared Worker at http:\/\/127\.0\.0\.1:\d+\/rds\/s\.js draws the browser's own random numbers, .*: it is a Shared Worker, and Firefox does not let keploy reach/).length, engine === 'firefox' ? 1 : 0); From ce58f5a8d6945ba06ecdeb904d1e5e2adc6e69c1 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 18:33:04 +0530 Subject: [PATCH 60/73] test(playwright): rows for a UTF-16 hashbang and lines said once A worker script in UTF-16 whose hashbang line ends with U+2028 or U+2029 (one code unit there) keeps that line first, with keploy's statement after it: a unit row for each byte order. The served browser row starts a second module worker under a Content-Security-Policy, and the redirected classic worker a second time, in each engine, seeded and with seeding off: the late line and the redirect line are each said once for their address, and the server's counts follow each start. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/test/fixtures/served.spec.ts | 4 ++-- playwright/test/random.test.mjs | 3 +++ playwright/test/rows.test.mjs | 25 +++++++++++++------------ 3 files changed, 18 insertions(+), 14 deletions(-) diff --git a/playwright/test/fixtures/served.spec.ts b/playwright/test/fixtures/served.spec.ts index 5c3185b..fadea4e 100644 --- a/playwright/test/fixtures/served.spec.ts +++ b/playwright/test/fixtures/served.spec.ts @@ -34,5 +34,5 @@ const start = async (page: Page, list: Start[]): Promise<void> => { }; test('latin1', async ({ page }) => start(page, [{ url: '/sv/l1.js' }, { url: '/sv/l1.js', type: 'module' }])); -test('csp', async ({ page }) => start(page, [{ url: '/sv/cn.js' }, { url: '/sv/csn.js' }, { url: '/sv/ctt.js' }, { url: '/sv/cn.js', type: 'module' }])); -test('redirect', async ({ page }) => start(page, [{ url: '/rd/w.js' }, { url: '/rdm/m.js', type: 'module' }, { url: '/rds/s.js', shared: true }])); +test('csp', async ({ page }) => start(page, [{ url: '/sv/cn.js' }, { url: '/sv/csn.js' }, { url: '/sv/ctt.js' }, { url: '/sv/cn.js', type: 'module' }, { url: '/sv/cn.js', type: 'module' }])); +test('redirect', async ({ page }) => start(page, [{ url: '/rd/w.js' }, { url: '/rdm/m.js', type: 'module' }, { url: '/rds/s.js', shared: true }, { url: '/rd/w.js' }])); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index f3f4168..286b37e 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3108,6 +3108,9 @@ test('workerAnswer keeps the app\'s script\'s bytes: another charset\'s bytes, a assert.deepEqual([...be.subarray(0, 2)], [0xfe, 0xff]); assert.equal(new TextDecoder('utf-16be').decode(be), `#!/x\n${q}a()${tail}`); assert.equal(new TextDecoder('utf-16le').decode(at(Buffer.concat([Buffer.from([0xff, 0xfe]), Buffer.from('#!/x\r\na()', 'utf16le')]))), `#!/x\r\n${q}a()${tail}`); + // In UTF-16, U+2028 and U+2029 are one code unit each, and end the hashbang line too. + assert.equal(new TextDecoder('utf-16le').decode(at(Buffer.concat([Buffer.from([0xff, 0xfe]), Buffer.from('#!/x\u2028a()', 'utf16le')]))), `#!/x\u2028${q}a()${tail}`); + assert.equal(new TextDecoder('utf-16be').decode(at(Buffer.concat([Buffer.from([0xfe, 0xff]), Buffer.from('#!/x\u2029a()', 'utf16le').swap16()]))), `#!/x\u2029${q}a()${tail}`); for (const [src, head] of [['#!/x\r\na\u00e9', '#!/x\r\n'], ['#!/x\ra', '#!/x\r'], ['#!/x\na', '#!/x\n'], ['#!/x\u2028a', '#!/x\u2028'], ['#!/x\u2029a', '#!/x\u2029'], ['\ufeff#!/x\na', '\ufeff#!/x\n']]) { assert.deepEqual(at(Buffer.from(src)), Buffer.concat([Buffer.from(head), a(q), Buffer.from(src.slice(head.length)), a(tail)]), JSON.stringify(src)); } diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 84959bd..05230b7 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1742,7 +1742,7 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and a module one under default-src 'none') runs seeded, its draws the seed's, the module one named as seeded late; one the server redirects (classic, module, and a Shared Worker) ${engine === 'chromium' ? 'does not start seeded (keploy stops its load, which Chromium would send round the redirect again and again) and is named, the server asked for its first URL once and never for the next, and off it ' : ''}runs at the URL the redirect gives, loading what sits beside it there, as off${engine === 'chromium' ? '' : ', unseeded and named, the server asked for its first URL ' + (engine === 'webkit' ? 'twice' : 'once')}; the server never sees keploy's mark`, async () => { + test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and two module ones under default-src 'none') runs seeded, its draws the seed's, the module ones named as seeded late once; one the server redirects (classic, module, a Shared Worker, and the classic one again) ${engine === 'chromium' ? 'does not start seeded (keploy stops its load, which Chromium would send round the redirect again and again) and is named once for each address, the server asked for its first URL once for each start and never for the next, and off it ' : ''}runs at the URL the redirect gives, loading what sits beside it there, as off${engine === 'chromium' ? '' : ', unseeded and named, the server asked for its first URL ' + (engine === 'webkit' ? 'twice' : 'once')}; the server never sees keploy's mark`, async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: 'cd'.repeat(16) } } : undefined)); try { @@ -1754,27 +1754,28 @@ for (const engine of ENGINES) { const at = (p) => new RegExp(`^http://127\\.0\\.0\\.1:\\d+${p.replace(/\./g, '\\.')}$`); for (const [mode, r] of [['seeded', s1], ['seeded again', s2], ['off', o1], ['off again', o2]]) { assert.deepEqual(r.drew.latin1.map((w) => w.k), ['l1', 'l1'], mode); - assert.deepEqual(r.drew.csp.map((w) => w.k), ['cn', 'csn', 'ctt', 'cn'], mode); + assert.deepEqual(r.drew.csp.map((w) => w.k), ['cn', 'csn', 'ctt', 'cn', 'cn'], mode); for (const [w, p] of [...r.drew.latin1.map((w) => [w, '/sv/l1.js']), ...r.drew.csp.map((w) => [w, `/sv/${w.k}.js`])]) assert.match(w.href, at(p), mode); // Chromium, seeded: keploy stopped each redirected load, so none starts. const stops = engine === 'chromium' && (r === s1 || r === s2); if (stops) { - assert.deepEqual(r.drew.redirect, ['error', 'error', 'error'], mode); + assert.deepEqual(r.drew.redirect, ['error', 'error', 'error', 'error'], mode); } else { - const [rd, rdm, rds] = r.drew.redirect; - assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rds.k, rds.H], ['rd', 'v2', 'rdm', 'v2', 'rds', 'v2'], mode); + const [rd, rdm, rds, rd2] = r.drew.redirect; + assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rds.k, rds.H, rd2.k, rd2.H], ['rd', 'v2', 'rdm', 'v2', 'rds', 'v2', 'rd', 'v2'], mode); assert.match(rd.href, at('/rd/v2/w.js'), mode); + assert.match(rd2.href, at('/rd/v2/w.js'), mode); assert.match(rdm.href, at('/rdm/v2/m.js'), mode); assert.match(rdm.meta, at('/rdm/v2/m.js'), mode); assert.match(rds.href, at('/rds/v2/s.js'), mode); } assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); - // Each Web Worker's script loaded once (two start at each of /sv/l1.js and /sv/cn.js). - assert.deepEqual(['/sv/l1.js', '/sv/cn.js', '/sv/csn.js', '/sv/ctt.js'].map((p) => r.seen.filter((u) => u === p).length), [2, 2, 1, 1], mode); - // A redirect's next URL, once (never where keploy stopped the load); its first: once, but twice seeded in WebKit. - for (const p of ['/rd/w.js', '/rdm/m.js', '/rds/s.js']) { - assert.equal(r.seen.filter((u) => u === p.replace(/\/([^/]+)$/, '/v2/$1')).length, stops ? 0 : 1, `${mode} ${p}`); - assert.equal(r.seen.filter((u) => u === p).length, (r === s1 || r === s2) && engine === 'webkit' ? 2 : 1, `${mode} ${p}`); + // Each Web Worker's script loaded once (two start at /sv/l1.js, three at /sv/cn.js). + assert.deepEqual(['/sv/l1.js', '/sv/cn.js', '/sv/csn.js', '/sv/ctt.js'].map((p) => r.seen.filter((u) => u === p).length), [2, 3, 1, 1], mode); + // For each start (/rd/w.js starts twice), a redirect's next URL once (never where keploy stopped the load), and its first once, but twice seeded in WebKit. + for (const [p, starts] of [['/rd/w.js', 2], ['/rdm/m.js', 1], ['/rds/s.js', 1]]) { + assert.equal(r.seen.filter((u) => u === p.replace(/\/([^/]+)$/, '/v2/$1')).length, stops ? 0 : starts, `${mode} ${p}`); + assert.equal(r.seen.filter((u) => u === p).length, starts * ((r === s1 || r === s2) && engine === 'webkit' ? 2 : 1), `${mode} ${p}`); } } // The Latin-1 text reads as off: Chromium decodes a worker's script by its charset, Firefox and WebKit as UTF-8. @@ -1786,7 +1787,7 @@ for (const engine of ENGINES) { assert.deepEqual(draws(s1, 'csp'), draws(s2, 'csp')); assert.notDeepEqual(draws(o1, 'latin1'), draws(o2, 'latin1')); assert.notDeepEqual(draws(o1, 'csp'), draws(o2, 'csp')); - if (engine !== 'chromium') for (const i of [0, 1, 2]) assert.notDeepEqual(s1.drew.redirect[i].draws, s2.drew.redirect[i].draws, `redirect ${i}`); + if (engine !== 'chromium') for (const i of [0, 1, 2, 3]) assert.notDeepEqual(s1.drew.redirect[i].draws, s2.drew.redirect[i].draws, `redirect ${i}`); // The lines: the module one under a policy, seeded late; each redirected one, once: in WebKit saying the second ask, in Chromium that it does not start. const said = (r, re) => r.lines.filter((l) => re.test(l)); for (const r of [s1, s2]) { From 813d20a579e237942337c3f28ac13684d1c0efc6 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 18:34:42 +0530 Subject: [PATCH 61/73] refactor(playwright): drop withPrefix, unused since scripts keep bytes Since a worker's answer keeps the app's script's bytes (withBytes), nothing in the package calls withPrefix, the string form of the same placement after a hashbang line. It and its two unit rows go; withBytes and its rows cover the placement. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/src/random.ts | 13 ------------- playwright/test/random.test.mjs | 4 +--- 2 files changed, 1 insertion(+), 16 deletions(-) diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 87f03f8..a887b76 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1887,19 +1887,6 @@ export function oneLine(seeding: string): string { return seeding.split('\n').filter((l) => !/^\s*\/\//.test(l)).map((l) => l.trim()).filter((l) => l !== '').join(' '); } -/** - * Where keploy's first statement goes in a script's source: ahead of it, or - * after its hashbang line (#!, which only a source's very start may hold), - * so every line of the app's keeps its number. - */ -export function withPrefix(source: string, prefix: string): string { - if (!source.startsWith('#!')) return prefix + source; - const m = /\r\n|[\n\r\u2028\u2029]/.exec(source); - if (m === null) return `${source}\n${prefix}`; - const i = m.index + m[0].length; - return source.slice(0, i) + prefix + source.slice(i); -} - /** * Whether a classic script's directive prologue (the string statements it * starts with, after a hashbang line, white space and comments, each ended diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 286b37e..d995257 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withPrefix, withBytes, oneLine, lateLine, redirectLine, STOPPED, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withBytes, oneLine, lateLine, redirectLine, STOPPED, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -2623,8 +2623,6 @@ test('workerAnswer: a Web Worker\'s is the app\'s script with its seeding\'s loa // U+2028 and U+2029 end a line as a line feed does, so the directive after one is a statement of its own. ['"a"\u2028"use strict"', true], ['"a"\u2029"use strict"', true], ['"a" "use strict"', false], ]) assert.equal(useStrict(src), strict, JSON.stringify(src)); - assert.equal(withPrefix('a', 'P;'), 'P;a'); - assert.equal(withPrefix('#!x\u2028a', 'P;'), '#!x\u2028P;a'); }); test('a Web Worker\'s seeding: an ErrorEvent\'s filename at the worker\'s own marked URL shows it with no mark; another, one that is no string and the getter\'s own error stay as they were; a Shared Worker\'s seeding leaves it as it is', () => { From 22bf477043e639b12c9a58925b5f1c3c74c25c2c Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 19:22:33 +0530 Subject: [PATCH 62/73] test(playwright): end a route-error test after its worker's message The row for a page route that takes itself off was flaky in WebKit: two of its tests fail by their route's error ('Route is already handled!') and ended at once, while the seeded worker they started still loaded its seeding. As Playwright closed the context, WebKit sent that load past every route, so the app's server got a URL with keploy's mark (2 of 76 such tests in WebKit, none of 76 in Firefox). That is the close-time case the README states, whose next step is to wait for what the worker does before the test ends. Each test now ends once its worker's message has come and its route's answer has ended; the two that fail keep their route's error and throw it then ('ignoreErrors' still drops its own), so they fail with the same error, and the row checks every test's worker: it runs at the app's URL and, seeded, draws its seeding's numbers. The README and the case's measurements name a test that fails while its worker loads, and the seeding load keploy's answer makes. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 2 +- playwright/test/fixtures/unroute.spec.ts | 48 ++++++++++++++++++++---- playwright/test/rows.test.mjs | 13 +++++-- 3 files changed, 50 insertions(+), 13 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 273e63a..bf93221 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -103,7 +103,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a module Web Worker's script is served with a Content-Security-Policy.** keploy puts the worker's seeding on its script's first line, as the policy could refuse the load of keploy's seeding, and in a module that line runs after the modules the script imports: a random number one of those modules draws as it loads is the browser's own, so it may differ at replay. The test says so once for each address (`keploy: in <test>, the module Web Worker at <address> is served with a Content-Security-Policy, ...`), with the next step: if one of those modules draws as it loads, serve the worker's script with no Content-Security-Policy where the tests run, and keploy loads its seeding ahead of those modules. A classic Web Worker's seeding on its first line runs before any of the app's code. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. -- **... the browser starts a request while Playwright closes a seeded test's context.** Once the test has ended (its steps, settle and its teardown), Playwright closes its context, and keploy's route goes with it. A request the browser starts in that moment, after the test has ended, can reach the app's server with keploy's `__keploy_worker` mark in its URL, in Firefox and WebKit: a worker the test's last step started, whose script the browser asks for as the context closes. No step of the test reads what such a request gets, and nothing is said. Measured on a test whose last step starts a worker and returns, 32 runs of each with four at once: in Firefox, 9 or 10 of 32 for a Web Worker (classic, module, or one a timer starts) and none for a Shared Worker; in WebKit, 1 to 8 of 32; in Chromium, none; for a test that waits for its worker's message, none in any engine. Next step: have the test wait for what the worker it starts last does before it ends. +- **... the browser starts a request while Playwright closes a seeded test's context.** Once the test has ended (its steps, settle and its teardown), Playwright closes its context, and keploy's route goes with it. A request the browser starts in that moment, after the test has ended, can reach the app's server with keploy's `__keploy_worker` mark in its URL, in Firefox and WebKit: a worker the test started just before it ended (its last step, or a step that then failed), whose script, or the seeding keploy's answer to that script loads, the browser asks for as the context closes. No step of the test reads what such a request gets, and nothing is said. Measured on a test whose last step starts a worker and returns, 32 runs of each with four at once: in Firefox, 9 or 10 of 32 for a Web Worker (classic, module, or one a timer starts) and none for a Shared Worker; in WebKit, 1 to 8 of 32; in Chromium, none; for a test that waits for its worker's message, none in any engine. For a test that fails by its route's error while the Web Worker it started loads: in WebKit, 2 of 76; in Firefox, none of 76. Next step: have the test wait for what the worker it starts last does before it ends (a test that fails by an error it can catch keeps it and throws it after that). - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: - a popup opened with noopener, as a link with `target="_blank"` opens one, or opened other than by `window.open` (a form, or a link with `rel="opener"`, that targets a new window); - a popup at another origin than its opener's page, or whose opener's page has loaded another document by the time the popup's begins, or whose opener the seeding could not place, and a frame of another origin inside one of those popups; diff --git a/playwright/test/fixtures/unroute.spec.ts b/playwright/test/fixtures/unroute.spec.ts index e876574..c2d882b 100644 --- a/playwright/test/fixtures/unroute.spec.ts +++ b/playwright/test/fixtures/unroute.spec.ts @@ -11,7 +11,13 @@ import { test } from '../../dist'; // and a fallback does nothing more. unrouteAll's 'wait' from the test, // while the route holds the load, waits for the route, whose continue // sends it. The worker's message (its kind, location and draws) is kept -// under the test's title. +// under the test's title. Each test ends only once its worker's message +// has come and its route's answer has ended, and one that fails by its +// route's error throws it then (the route keeps it; after 'ignoreErrors' +// Playwright drops it), so no test ends while its worker still loads: a +// load the browser starts as Playwright closes a seeded test's context can +// reach the server with keploy's mark (the README's "What the run says +// when ..."). const base = process.env.DRAWS_BASE ?? ''; const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); @@ -23,30 +29,56 @@ const work = (): Promise<unknown> => new Promise((resolve) => { w.onerror = () => resolve('error'); }); +// A route's answer: run(answer, keepIt) runs it and, with keepIt, keeps +// what it throws instead of throwing it; after() waits until the answer +// has ended, then throws what was kept. +const routeAnswer = (): { run: (answer: () => Promise<void>, keepIt: boolean) => Promise<void>; after: () => Promise<void> } => { + let err: { e: unknown } | null = null; + let ended = (): void => {}; + const end = new Promise<void>((resolve) => { ended = resolve; }); + return { + run: async (answer, keepIt) => { + try { + await answer(); + } catch (e) { + if (!keepIt) throw e; + err = { e }; + } finally { + ended(); + } + }, + after: async () => { + await end; + if (err !== null) throw err.e; + }, + }; +}; + for (const how of ['continue', 'fallback'] as const) { test(`unroute then ${how}`, async ({ page }) => { await page.goto(`${base}/ov`); + const answered = routeAnswer(); const h = async (r: Route): Promise<void> => { await page.unroute('**/ov/pw.js', h); - await (how === 'continue' ? r.continue() : r.fallback()); + await answered.run(() => (how === 'continue' ? r.continue() : r.fallback()), true); }; await page.route('**/ov/pw.js', h); - keep(test.info().title, await page.evaluate(() => new Promise((resolve) => { - const w = new Worker('/ov/pw.js'); - w.onmessage = (e) => resolve(e.data); - w.onerror = () => resolve('error'); - }))); + keep(test.info().title, await page.evaluate(work)); + await answered.after(); }); } for (const [behavior, how] of [['default', 'fulfill'], ['ignoreErrors', 'abort']] as const) { test(`unrouteAll ${behavior} then ${how}`, async ({ page }) => { await page.goto(`${base}/ov`); + const answered = routeAnswer(); await page.route('**/ov/pw.js', async (r: Route): Promise<void> => { await page.unrouteAll({ behavior }); - await (how === 'fulfill' ? r.fulfill({ contentType: 'text/javascript', body: 'postMessage("own")' }) : r.abort()); + // After 'ignoreErrors', Playwright drops the route's error itself. + await answered.run(() => (how === 'fulfill' ? r.fulfill({ contentType: 'text/javascript', body: 'postMessage("own")' }) : r.abort()), behavior === 'default'); }); keep(test.info().title, await page.evaluate(work)); + await answered.after(); }); } diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 05230b7..c5a22c0 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -2591,11 +2591,16 @@ test('seeded: after a test that throws, and after one that times out, the worker // (unroute.spec.ts): seeded, each test fails with Playwright's own error, // or passes, as with seeding off; the app's server is asked for each // worker's script once, never with keploy's mark; seeded, the worker of each -// test that passes draws its seeding's numbers. +// test draws its seeding's numbers. Each test ends once its worker's +// message has come and its route's answer has ended, and one that fails +// throws its route's error then, so it ends after its worker's loads, as +// the README's next step for a request the browser starts as Playwright +// closes a seeded test's context says. for (const engine of ENGINES) { test(`seeded, in ${engine}: a page route that takes itself off (unroute, unrouteAll with each behavior), then continues, fulfills or aborts a Web Worker's script load, fails the test with Playwright's own error, as with seeding off (after 'ignoreErrors' it passes), one that falls back passes, and unrouteAll's 'wait' waits for the route's continue; the app's server is asked for each script once, never with the mark, and each worker runs, seeded`, async () => { - // The tests that pass, whose worker's message is kept. + // The tests that pass; every test's worker's message is kept. const passing = ['unroute then fallback', 'unrouteAll ignoreErrors then abort', 'unrouteAll wait then continue']; + const all = ['unroute then continue', 'unrouteAll default then fulfill', ...passing]; const j = journal(); const rec = await fakeAgent(j); try { @@ -2620,7 +2625,7 @@ for (const engine of ENGINES) { for (const title of passing) { assert.equal(tests[title].status, 'passed', `${mode} ${title}: ${tests[title].error}\n${out}`); } - for (const title of passing) { + for (const title of all) { const got = JSON.parse(fs.readFileSync(path.join(tmp, `${title}.json`), 'utf8')); assert.equal(got.kind, 'pw', `${mode} ${title}`); assert.match(got.href, /^http:\/\/127\.0\.0\.1:\d+\/ov\/pw\.js$/, `${mode} ${title}`); @@ -2628,7 +2633,7 @@ for (const engine of ENGINES) { assert.deepEqual(seen.filter((u) => u.startsWith('/ov/')), Array(5).fill('/ov/pw.js'), mode); assert.deepEqual(seen.filter((u) => u.includes('__keploy_worker')), [], mode); } - for (const title of passing) { + for (const title of all) { const [mark] = onceMarks(runs.seeded.seeds[title], [['/ov/pw.js', 'w']]); const got = JSON.parse(fs.readFileSync(path.join(runs.seeded.tmp, `${title}.json`), 'utf8')); assert.deepEqual(got.draws, workerAt('http://h/ov/pw.js', workerScript(mark.token, 0, 'w')).draws(), title); From 3c171691eebd4b71a3182fb918234214879ca3c4 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 21:52:56 +0530 Subject: [PATCH 63/73] fix(playwright): seed in place only where a worker's CSP may refuse it A worker's script served with any Content-Security-Policy had keploy's seeding put on the script's first line, so a module Web Worker under a policy that allows a script from its own origin ('self', as helmet's default does) ran its imports before its seeding: a random number an import drew as it loaded was the browser's own. And a Shared Worker under a policy that refuses a script from its own origin (default-src 'none') did not start when seeded, as keploy's answer loaded the app's script from its URL, which the policy refused; with seeding off it runs. cspAllowsOwn reads the policy. keploy keeps its load at the worker's own URL where every policy allows it ('self', 'strict-dynamic', *, the scheme or the origin; no sandbox; no Trusted Types for a classic worker; no upgrade of http, but in Chromium on a loopback host), and puts the seeding on the first line only where one could refuse it. A Shared Worker whose policy could refuse its load of the app's script is answered so too: the app's bytes with the seeding on the first line, run at its marked URL; a module one is named as seeded late, as a module Web Worker is. Rows: the policy table; the route's answers for each kind; a browser row for each engine with module workers whose imports draw under default-src 'self' and helmet's default, 'strict-dynamic' workers, and a classic and a module Shared Worker under default-src 'none'. Also corrects two README sentences on redirects in Firefox and a comment on byte order marks. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 8 +- playwright/src/index.ts | 9 +- playwright/src/random.ts | 157 ++++++++++++++++------ playwright/test/fixtures/served.spec.ts | 5 + playwright/test/random.test.mjs | 169 +++++++++++++++++++++++- playwright/test/rows.test.mjs | 61 ++++++++- 6 files changed, 360 insertions(+), 49 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index bf93221..f6e565d 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy, which could refuse that load, the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark. Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a Web Worker whose script carries a Content-Security-Policy, where keploy's seeding is on the app's first line: there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it (a script the server redirects aside, below); the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker. Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy that could refuse that load (one keploy cannot tell allows a script from the worker's own origin: its source list for scripts (`script-src-elem`, else `script-src`, else `default-src`) lists `'self'`, `'strict-dynamic'`, `*`, the origin's scheme or the origin itself (its scheme, host and port, or a wildcard host that holds it, with a path that holds the script's, or none), and for a module worker so does its list for workers (`worker-src`, else `child-src`, else those); it has no `sandbox`; a classic worker's requires no Trusted Types for scripts; and it upgrades no http request (`upgrade-insecure-requests`), which Chromium never does for a loopback host (`localhost`, `127.0.0.1`), so helmet's default policy allows it there in Chromium, and in Firefox and WebKit on https), the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark; where the script's policy could refuse that load (as above), it is the app's script itself, as a Web Worker's is, with the seeding on its first line, run at the worker's marked URL, in a module Shared Worker after the modules the app's script imports, which the test says (below). Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark, but in a module one whose seeding is on its script's first line (above), where it is its marked URL. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a worker whose seeding is on the app's first line (a policy that could refuse keploy's load, above): there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it (a script the server redirects aside, below); the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker (once, by keploy, for one whose seeding is on its script's first line). Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); @@ -100,8 +100,8 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). - **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. - **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. -- **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. With seeding off, or for a classic Web Worker (its `location`) or a Shared Worker (its `import.meta.url` too), the worker sees the app's URL. -- **... a module Web Worker's script is served with a Content-Security-Policy.** keploy puts the worker's seeding on its script's first line, as the policy could refuse the load of keploy's seeding, and in a module that line runs after the modules the script imports: a random number one of those modules draws as it loads is the browser's own, so it may differ at replay. The test says so once for each address (`keploy: in <test>, the module Web Worker at <address> is served with a Content-Security-Policy, ...`), with the next step: if one of those modules draws as it loads, serve the worker's script with no Content-Security-Policy where the tests run, and keploy loads its seeding ahead of those modules. A classic Web Worker's seeding on its first line runs before any of the app's code. +- **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. A module Shared Worker whose seeding is on its script's first line (below) reads its marked URL so too. With seeding off, or for a classic Web Worker (its `location`) or another Shared Worker (its `import.meta.url` too), the worker sees the app's URL. +- **... a module worker's script is served with a Content-Security-Policy that could refuse keploy's load.** When keploy cannot tell that the policy allows a script from the worker's own origin (`'self'` in `script-src` or `default-src` does; the rest under Workers, above), keploy puts the worker's seeding on its script's first line, and in a module that line runs after the modules the script imports: a random number one of those modules draws as it loads is the browser's own, so it may differ at replay. This holds for a module Web Worker and, in Chromium and WebKit, a module Shared Worker. The test says so once for each address (`keploy: in <test>, the module Web Worker at <address> is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, ...`, or `module Shared Worker`), with the next step: if one of those modules draws as it loads, have the policy allow scripts from the worker's own origin (`'self'` in `script-src`) where the tests run, or serve the worker's script with no Content-Security-Policy there, and keploy loads its seeding ahead of those modules. A classic worker's seeding on its first line runs before any of the app's code. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. - **... the browser starts a request while Playwright closes a seeded test's context.** Once the test has ended (its steps, settle and its teardown), Playwright closes its context, and keploy's route goes with it. A request the browser starts in that moment, after the test has ended, can reach the app's server with keploy's `__keploy_worker` mark in its URL, in Firefox and WebKit: a worker the test started just before it ended (its last step, or a step that then failed), whose script, or the seeding keploy's answer to that script loads, the browser asks for as the context closes. No step of the test reads what such a request gets, and nothing is said. Measured on a test whose last step starts a worker and returns, 32 runs of each with four at once: in Firefox, 9 or 10 of 32 for a Web Worker (classic, module, or one a timer starts) and none for a Shared Worker; in WebKit, 1 to 8 of 32; in Chromium, none; for a test that waits for its worker's message, none in any engine. For a test that fails by its route's error while the Web Worker it started loads: in WebKit, 2 of 76; in Firefox, none of 76. Next step: have the test wait for what the worker it starts last does before it ends (a test that fails by an error it can catch keeps it and throws it after that). - **... a document cannot be placed.** It draws the browser's own numbers, never another page's, and the test prints `keploy: in <test>, the document at <address> drew the browser's own random numbers, so its draws may differ at replay: ...` with the reason, once for each address and reason, in its record and in each replay. Nothing tells these documents apart from the next ones at their place, so seeded, two of them could draw the same numbers: @@ -123,7 +123,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with keploy's mark (a Trusted Types policy). Next step: start it from its script's http(s) URL. - A worker whose script a route of the test's own answered itself (`route.fulfill`). Next step: have that route call `route.continue()` or `route.fallback()` for the worker's script. - A Shared Worker whose script a route of the test's own sent on with a URL, method, headers or post data of its own (`route.continue` or `route.fallback` with options), in Chromium and WebKit: the seeding's load of the app's script, which these engines do not route, could not carry them. Next step: have that route continue or fall back with no options for the Shared Worker's script. - - In Firefox and WebKit, a worker whose script the server redirects (a 301, 302, 303, 307 or 308 answer with a `Location`), classic or module, Web or Shared: Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script at the URL the redirect gives. keploy follows no redirect itself: the worker runs at the URL the redirect gives, as with seeding off (its `location`, `import.meta.url` and the URLs its imports resolve against are the same), with no mark. In Firefox, keploy's answer is the server's redirect, with its `Location` resolved against the URL the app asked for. WebKit follows no redirect a route answers with (it refuses one), so keploy sends the request on to the server as the test's routes asked for it, and the server is asked for the script's first URL twice: once by keploy, to see the redirect, and once by the browser. The line says so. Next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect, to seed it. In Chromium such a worker does not start (below). + - In Firefox and WebKit, a worker whose script the server redirects (a 301, 302, 303, 307 or 308 answer with a `Location`), classic or module, a Web Worker, and in WebKit a Shared Worker too (Firefox's route never reaches a Shared Worker's script, so a redirected one there gets the Shared Worker line above): Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script at the URL the redirect gives. keploy follows no redirect itself: the worker runs at the URL the redirect gives, as with seeding off (its `location`, `import.meta.url` and the URLs its imports resolve against are the same), with no mark. In Firefox, keploy's answer is the server's redirect, with its `Location` resolved against the URL keploy asked the server for (the app's, or one a route of the test's gave with `continue` or `fallback`). WebKit follows no redirect a route answers with (it refuses one), so keploy sends the request on to the server as the test's routes asked for it, and the server is asked for the script's first URL twice: once by keploy, to see the redirect, and once by the browser. The line says so. Next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect, to seed it. In Chromium such a worker does not start (below). - In WebKit, a worker that a Shared Worker starts draws the browser's own numbers too, as WebKit's route reaches nothing a Shared Worker asks for after its script, but no line names it yet (in Firefox the Shared Worker's own line names it). - **... in Chromium, the server redirects a seeded worker's script.** The worker does not start, where with seeding off it runs at the URL the redirect gives. Chromium never ends a worker's load of a redirect that a route answers with, and Playwright follows no redirect of a request whose URL a route changed: on each hop it asks for the first URL again, until the browser gives up (a test's own `route.continue({ url })` on a worker's script meets the same, with no keploy). keploy changes the URL of a seeded worker's script load, to take its mark off before the request leaves the browser, so in Chromium keploy stops that load (`net::ERR_BLOCKED_BY_CLIENT`) after asking the server for the script once, to see the redirect, and the worker fails to load (its `error` event). This holds for classic and module Web Workers and for Shared Workers, at record and at replay alike. The test prints `keploy: in <test>, the worker at <address> does not start: the server redirects its script, ...` once for each address, with why and the next step: serve the worker's script where the tests run at the URL the app starts the worker at, with no redirect (keploy then seeds it), turn seeding off for that test (`test.use({ keploySeedRandom: false })`), or run it in Firefox or WebKit. - **... a replay draws otherwise than its recording.** A seed gives a replay its recording's numbers only while each stream is drawn the same number of times, in the same order. The replay's end prints one line for each stream, document and frame path whose count differs from the recording's (above), naming the test, the two counts and the next step. These move a stream: diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 17536a8..2490e82 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -259,14 +259,15 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ told.add(k); process.stdout.write(`${unplacedLine(name, a, why)}\n`); }, - // A module Web Worker whose seeding runs after its imports (its - // script carries a Content-Security-Policy): once for each address. - late: (at) => { + // A module worker whose seeding runs after its imports (its + // script's Content-Security-Policy could refuse keploy's load): + // once for each address. + late: (at, shared) => { const a = pageAddress(at); const k = `late\n${a}`; if (told.has(k)) return; told.add(k); - process.stdout.write(`${lateLine(name, a)}\n`); + process.stdout.write(`${lateLine(name, a, shared)}\n`); }, // A worker whose script the server redirects, which so runs // unseeded, or in Chromium does not start: once for each address. diff --git a/playwright/src/random.ts b/playwright/src/random.ts index a887b76..519f248 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1607,9 +1607,12 @@ export function markWorker(url: string, kind: WorkerMark['kind'], token: string, * marked: keploy's own code. An error the browser's own call throws in a * seeded function or in the location's getters (a float array, a this that * is no Crypto or location) reaches its caller with none of the seeding's - * frames on the stack it keeps (V8's and WebKit's). + * frames on the stack it keeps (V8's and WebKit's). inPlace: the app's + * script runs in keploy's answer itself, after this, at the worker's + * marked URL, as a Web Worker's always does and a Shared Worker's does + * when its policy could refuse the load of the app's script (workerAnswer). */ -export function workerScript(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', reach = false): string { +export function workerScript(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', reach = false, inPlace = false): string { return `(() => { const g = globalThis; const k = ${JSON.stringify(`${token}\n${n}\n`)}; @@ -1622,12 +1625,12 @@ ${STREAM_JS} // and WebKit, a stack of its own, made as it was thrown, with the seeding's // frames: they are left out of it before it reaches the caller, and a // stack with none, or that is no string, stays as it was. A Web Worker's - // app's script runs at the worker's own marked URL, after its seeding - // (workerAnswer): a frame there is the app's, kept, with the mark taken + // app's script (and a Shared Worker's, in place) runs at the worker's + // own marked URL, after its seeding (workerAnswer): a frame there is the app's, kept, with the mark taken // off (each engine names the app's URL there already: the answer's // sourceURL). const marked = new RegExp(${JSON.stringify(`[?&;]${WORKER_PARAM}=`)}); - const main = ${kind === 'w' || kind === 'm' ? `new RegExp(${JSON.stringify(`[?&]${WORKER_PARAM}=${kind}\\.${token}(?![.0-9a-f])`)}, 'g')` : 'null'}; + const main = ${kind === 'w' || kind === 'm' || inPlace ? `new RegExp(${JSON.stringify(`[?&]${WORKER_PARAM}=${kind}\\.${token}(?![.0-9a-f])`)}, 'g')` : 'null'}; const unmark = (s) => s.split('\\n').map((l) => (main === null ? l : l.replace(main, ''))).filter((l) => !marked.test(l)).join('\\n'); const thrown = (e) => { try { @@ -1712,7 +1715,8 @@ ${DRAW_JS} } // An error the app's script lets out names the script it was thrown in // (an ErrorEvent's filename): Firefox and WebKit name a Web Worker's - // marked URL, shown here with no mark, as Chromium names it. + // marked URL (and a Shared Worker's, in place), shown here with no mark, + // as Chromium names it. if (main !== null) { try { const E = g.ErrorEvent; @@ -1815,8 +1819,75 @@ export function cspAllowsData(headers: Record<string, string>): boolean { } /** - * The encoding a script's bytes name with a byte order mark (which every - * engine reads a worker's script by, whatever its charset says), and the + * Whether the Content-Security-Policy of a worker's script (headers, its + * response's) lets the worker load a script from url, a URL of the + * worker's own origin (keploy's seeding, at the script's own URL marked, + * or the app's script there with no mark), so keploy may load it there: + * no policy holds a sandbox (which could give the worker an opaque origin, + * which 'self' does not match); each policy that names a source list for + * scripts (script-src-elem, else script-src, else default-src) lists + * 'self', 'strict-dynamic' (which allows a script a script loads, as + * keploy's are), *, url's scheme, or url's origin (a host source of its + * scheme, host and port, with a path that holds url's, or none), and so, + * for a module worker, does each that names one for its worker's loads + * (worker-src, else child-src, else script-src, else default-src: an + * engine may read a module's imports by it); for a + * classic worker, whose importScripts takes a string, no policy requires + * Trusted Types for scripts; and no policy upgrades an http url to https + * (upgrade-insecure-requests), but in Chromium for a loopback host, which + * Chromium never upgrades. A source keploy does not read so (a wildcard + * host aside: *.example.com), counts as one that refuses: keploy then + * puts the seeding in the script itself (workerAnswer inline), which no + * policy refuses. + */ +export function cspAllowsOwn(headers: Record<string, string>, url: string, classic: boolean, engine = ''): boolean { + const v = headers['content-security-policy']; + if (v === undefined) return true; + let u: URL; + try { u = new URL(url); } catch { return false; } + const scheme = u.protocol.slice(0, -1).toLowerCase(); + const host = u.hostname.toLowerCase(); + const loopback = host === 'localhost' || host.endsWith('.localhost') || /^127\.\d+\.\d+\.\d+$/.test(host) || host === '[::1]'; + const port = u.port !== '' ? u.port : scheme === 'https' ? '443' : scheme === 'http' ? '80' : ''; + const source = (x: string): boolean => { + const t = x.toLowerCase(); + if (t === "'self'" || t === "'strict-dynamic'") return true; + if (t === '*') return scheme === 'http' || scheme === 'https'; + if (/^[a-z][a-z0-9+.-]*:$/.test(t)) return t === `${scheme}:` || (t === 'http:' && scheme === 'https'); + const h = /^(?:([a-z][a-z0-9+.-]*):\/\/)?(\*|(?:\*\.)?[a-z0-9-]+(?:\.[a-z0-9-]+)*)(?::(\d+|\*))?(\/[^?#]*)?$/.exec(t); + if (h === null) return false; + if (h[1] !== undefined && h[1] !== scheme && !(h[1] === 'http' && scheme === 'https')) return false; + if (h[1] === undefined && scheme !== 'http' && scheme !== 'https') return false; + if (h[2] !== '*' && !(h[2].startsWith('*.') ? host.endsWith(h[2].slice(1)) : host === h[2])) return false; + if (h[3] === undefined ? u.port !== '' : h[3] !== '*' && h[3] !== port) return false; + // A path: what x says, with its letters' case, read as url's path is + // written (one with an escape keploy does not read). + const p = h[4] === undefined ? '' : /^(?:[a-z][a-z0-9+.-]*:\/\/)?[^/]*(\/[^?#]*)$/i.exec(x)?.[1] ?? ''; + if (p === '' || p === '/') return true; + if (p.includes('%') || u.pathname.includes('%')) return false; + return p.endsWith('/') ? u.pathname.startsWith(p) : u.pathname === p; + }; + for (const policy of v.split(/[,\n]/)) { + const ds = new Map<string, string[]>(); + for (const d of policy.split(';')) { + const t = d.trim().split(/\s+/).filter((x) => x !== ''); + if (t.length > 0 && !ds.has(t[0].toLowerCase())) ds.set(t[0].toLowerCase(), t.slice(1)); + } + if (ds.has('sandbox')) return false; + if (classic && (ds.get('require-trusted-types-for') ?? []).some((x) => x.toLowerCase() === "'script'")) return false; + if (ds.has('upgrade-insecure-requests') && scheme === 'http' && !(loopback && engine === 'chromium')) return false; + const first = (...names: string[]): string[] | undefined => names.map((x) => ds.get(x)).find((x) => x !== undefined); + for (const list of [first('script-src-elem', 'script-src', 'default-src'), classic ? undefined : first('worker-src', 'child-src', 'script-src', 'default-src')]) { + if (list !== undefined && !list.some(source)) return false; + } + } + return true; +} + +/** + * The encoding a script's bytes name with a byte order mark (which + * Chromium reads a worker's script by, whatever its charset says; Firefox + * and WebKit refuse a UTF-16 one, seeded or not), and the * mark's length: UTF-16 of either byte order, else UTF-8 (0 when it has * none). */ @@ -1951,8 +2022,8 @@ export function useStrict(source: string): boolean { * worker's seeding from a URL keploy marked ('q' for a classic worker, by * importScripts; 'p' for a module one, by an import ahead of every import * of the app's, so it runs before any of them), or, inline (the script's - * response carries a Content-Security-Policy, which could refuse that - * load), is the seeding itself on one line (oneLine), which in a module + * response carries a Content-Security-Policy that could refuse that + * load: cspAllowsOwn), is the seeding itself on one line (oneLine), which in a module * runs after the modules the app's script imports; a classic worker's * starts with a 'use strict' of keploy's when the app's script has one in * its directive prologue, so it stays strict. It goes on the app's first @@ -1967,7 +2038,11 @@ export function useStrict(source: string): boolean { * app's script (source undefined), the worker gets its seeding, then loads * the app's script itself, from own, as below. * - * A Shared Worker's ('s', 'x') is the worker's seeding, then the app's own + * A Shared Worker's ('s', 'x'), inline (its script's policy could refuse + * its load of the app's script, below: cspAllowsOwn), is so too: the app's + * script, source, with the seeding on its first line, run at the worker's + * marked URL; in a module one after the modules the app's script imports. + * Else it is the worker's seeding, then the app's own * script, loaded from own, the URL the app asked for, with no mark. The app's * script so runs at its own URL: its import.meta.url, the URLs its imports * resolve against and a stack trace through it show no mark, and the @@ -1996,12 +2071,12 @@ export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | ' export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk: boolean, source: Uint8Array, inline?: boolean, reach?: boolean): Buffer; export function workerAnswer(token: string, n: number, kind: 'w' | 'm' | 's' | 'x', own: string, dataOk = true, source?: string | Uint8Array, inline = false, reach = false): string | Buffer { const url = JSON.stringify(own); - if ((kind === 'w' || kind === 'm') && source !== undefined) { + if ((kind === 'w' || kind === 'm' || inline) && source !== undefined) { const bytes = typeof source === 'string' ? Buffer.from(source) : source; const text = scriptText(bytes); - const first = kind === 'w' - ? `${useStrict(text) ? '"use strict";' : ''}${inline ? oneLine(workerScript(token, n, kind, reach)) : `importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});`}` - : inline ? oneLine(workerScript(token, n, kind, reach)) : `import ${JSON.stringify(markWorker(own, 'p', token, n))};`; + const first = kind === 'w' || kind === 's' + ? `${useStrict(text) ? '"use strict";' : ''}${inline ? oneLine(workerScript(token, n, kind, reach, true)) : `importScripts(${JSON.stringify(markWorker(own, 'q', token, n))});`}` + : inline ? oneLine(workerScript(token, n, kind, reach, true)) : `import ${JSON.stringify(markWorker(own, 'p', token, n))};`; const named = /(?:\/\/|\/\*)[#@][ \t]*sourceURL[ \t]*=/.test(text); const answer = withBytes(bytes, first, named ? '' : `\n//# sourceURL=${noFragment(own)}\n`); return typeof source === 'string' ? answer.toString() : answer; @@ -2058,11 +2133,12 @@ export type WorkerHooks = { */ unmarked?: (at: string, why: 'worker' | 'nested') => void; /** - * keploy put the seeding of the module Web Worker at at (its origin and - * path) on its script's first line, as the script's response carries a - * Content-Security-Policy: it runs after the modules the script imports. + * keploy put the seeding of the module worker at at (its origin and + * path; shared: a Shared Worker) on its script's first line, as the + * script's Content-Security-Policy could refuse its load (cspAllowsOwn): + * it runs after the modules the script imports. */ - late?: (at: string) => void; + late?: (at: string, shared: boolean) => void; /** * The server redirected the script of the worker at at (its origin and * path). 'followed' (Firefox): it runs unseeded at the URL the redirect @@ -2077,15 +2153,15 @@ export type WorkerHooks = { export type Redirected = 'followed' | 'twice' | 'stopped'; /** - * The line a test shows, once for each address, for a module Web Worker - * whose seeding started after the modules its script imports (WorkerHooks - * late). + * The line a test shows, once for each address, for a module worker + * (shared: a Shared Worker) whose seeding started after the modules its + * script imports (WorkerHooks late). */ -export function lateLine(name: string, at: string): string { - return `keploy: in ${name}, the module Web Worker at ${at} is served with a Content-Security-Policy, which could refuse keploy's load of its seeding, ` + +export function lateLine(name: string, at: string, shared = false): string { + return `keploy: in ${name}, the module ${shared ? 'Shared' : 'Web'} Worker at ${at} is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, ` + 'so keploy put its seeding on the first line of the worker\'s script, and it runs after the modules the script imports: ' + 'a random number one of those modules draws as it loads is the browser\'s own, so it may differ at replay; ' + - 'if one does, serve the worker\'s script with no Content-Security-Policy where the tests run, and keploy loads its seeding ahead of those modules.'; + 'if one does, have the policy allow scripts from the worker\'s own origin (\'self\' in script-src) where the tests run, or serve the worker\'s script with no Content-Security-Policy there, and keploy loads its seeding ahead of those modules.'; } /** @@ -2399,6 +2475,7 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = let status = 200; let headers: Record<string, string> = { 'content-type': 'text/javascript' }; let source: Uint8Array | undefined; + let inline = false; let moved: Awaited<ReturnType<Route['fetch']>> | undefined; try { // keploy follows no redirect: the browser does (below). @@ -2415,9 +2492,18 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } else { status = got.status(); headers = headersOf(got.headers()); - // A Web Worker's answer holds the app's script's bytes as the server - // sent them, whatever their charset (withBytes). - if (m.kind === 'w' || m.kind === 'm') source = await got.body(); + // The script's Content-Security-Policy is the worker's own: one + // keploy cannot tell allows a script from the worker's origin + // (cspAllowsOwn) could refuse the load of keploy's seeding at the + // script's own URL (a Web Worker's importScripts or import), or the + // seeding's load of the app's script there (a Shared Worker's), so + // the seeding goes on the app's first line instead (inline), which + // no policy refuses. A Web Worker's answer holds the app's script's + // bytes as the server sent them, whatever their charset (withBytes), + // and so does an inline Shared Worker's. + const refuses = !cspAllowsOwn(headers, m.url, m.kind === 'w' || m.kind === 's', engine); + if (m.kind === 'w' || m.kind === 'm' || refuses) source = await got.body(); + inline = refuses; } } catch { // The worker loads the app's script itself, and fails as it would. @@ -2450,18 +2536,13 @@ export async function workerRoutes(context: BrowserContext, hooks: WorkerHooks = } return; } - // A Content-Security-Policy on the script's response is the worker's - // own: it could refuse the load of keploy's seeding at its own URL - // (importScripts or import), so the seeding goes on the app's first - // line instead, which no policy refuses. A module's static imports run - // before its first line, so such a module Web Worker's seeding starts - // after them, and the test says so. - const inline = source !== undefined && headers['content-security-policy'] !== undefined; - if (inline && m.kind === 'm') { + // A module's static imports run before its first line, so an inline + // module worker's seeding starts after them, and the test says so. + if (inline && (m.kind === 'm' || m.kind === 'x')) { const u = new URL(m.url); - hooks.late?.(u.origin + u.pathname); + hooks.late?.(u.origin + u.pathname, m.kind === 'x'); } - await fulfill({ status, headers, body: source === undefined ? workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers)) : workerAnswer(m.token, n, m.kind as 'w' | 'm', m.url, true, source, inline, reach) }); + await fulfill({ status, headers, body: source === undefined ? workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, cspAllowsData(headers)) : workerAnswer(m.token, n, m.kind as 'w' | 'm' | 's' | 'x', m.url, true, source, inline, reach) }); }; const first = async (route: Route): Promise<void> => { const r = route.request(); diff --git a/playwright/test/fixtures/served.spec.ts b/playwright/test/fixtures/served.spec.ts index fadea4e..2d120a5 100644 --- a/playwright/test/fixtures/served.spec.ts +++ b/playwright/test/fixtures/served.spec.ts @@ -36,3 +36,8 @@ const start = async (page: Page, list: Start[]): Promise<void> => { test('latin1', async ({ page }) => start(page, [{ url: '/sv/l1.js' }, { url: '/sv/l1.js', type: 'module' }])); test('csp', async ({ page }) => start(page, [{ url: '/sv/cn.js' }, { url: '/sv/csn.js' }, { url: '/sv/ctt.js' }, { url: '/sv/cn.js', type: 'module' }, { url: '/sv/cn.js', type: 'module' }])); test('redirect', async ({ page }) => start(page, [{ url: '/rd/w.js' }, { url: '/rdm/m.js', type: 'module' }, { url: '/rds/s.js', shared: true }, { url: '/rd/w.js' }])); +// Under a Content-Security-Policy of the script's own: module Web Workers +// that import a module drawing as it loads (default-src 'self', helmet's +// default), a module and a classic one under 'strict-dynamic', and a +// classic and a module Shared Worker under default-src 'none'. +test('own', async ({ page }) => start(page, [{ url: '/mc/s.js', type: 'module' }, { url: '/mc/h.js', type: 'module' }, { url: '/mc/sd.js', type: 'module' }, { url: '/mc/sdc.js' }, { url: '/sv/sn.js', shared: true }, { url: '/sv/snm.js', type: 'module', shared: true }])); diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index d995257..81bd491 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3,7 +3,7 @@ // a page. No browser. import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withBytes, oneLine, lateLine, redirectLine, STOPPED, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData } from '../dist/random.js'; +import { DOC_KEY, ENV_SEED_RANDOM, FAKER, LOADS_KEY, OFF_LINE, PAGE_KEY, SEEDED_LINE, esmFile, exportOf, fakerLine, fakersIn, fakersOf, firstScript, freshSeed, initScript, besideOf, othersNow, pageAddress, pageScript, placePages, processRandom, seatFaker, seatRandom, seedFaker, seedingFromEnv, setAside, unplacedLine, unseededLine, WORKER_PARAM, SW_CHANNEL, SW_LOCK, markWorker, useStrict, withBytes, oneLine, lateLine, redirectLine, STOPPED, workerMark, workerScript, workerRoutes, workerAnswer, cspAllowsData, cspAllowsOwn } from '../dist/random.js'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -3160,5 +3160,170 @@ test('workerRoutes answers a Web Worker whose script carries a Content-Security- // A script keploy could not fetch: its seeding's load, as before, and no line. assert.equal((await r.ask(markWorker('http://h/m.js', 'm', t), new Error('net')))[1][1].body, workerAnswer(t, 5, 'm', 'http://h/m.js')); assert.deepEqual(late, ['http://h/m.js']); - assert.match(lateLine('a test', 'http://h/m.js'), /^keploy: in a test, the module Web Worker at http:\/\/h\/m\.js is served with a Content-Security-Policy, .* runs after the modules the script imports: .*; if one does, serve the worker's script with no Content-Security-Policy where the tests run/); + assert.match(lateLine('a test', 'http://h/m.js'), /^keploy: in a test, the module Web Worker at http:\/\/h\/m\.js is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, .* runs after the modules the script imports: .*; if one does, have the policy allow scripts from the worker's own origin \('self' in script-src\) where the tests run, or serve the worker's script with no Content-Security-Policy there/); +}); + +test('cspAllowsOwn: a policy that allows a script from the worker\'s own origin lets keploy load there; one it cannot tell does (none, another host or port, a nonce, a sandbox, Trusted Types for a classic worker, an upgrade of http but in Chromium on a loopback host) refuses', () => { + const csp = (v) => ({ 'content-type': 'text/javascript', 'content-security-policy': v }); + const helmet = "default-src 'self';base-uri 'self';font-src 'self' https: data:;form-action 'self';frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests"; + assert.equal(cspAllowsOwn({}, 'http://h/w.js', true), true); + assert.equal(cspAllowsOwn({ 'content-security-policy-report-only': "default-src 'none'" }, 'http://h/w.js', true), true); + // [policy, url, classic, engine, allows] + const rows = [ + ["default-src 'self'", 'http://h/w.js', false, 'webkit', true], + ["default-src 'SELF'", 'http://h/w.js', true, 'firefox', true], + ["img-src 'none'", 'http://h/w.js', false, 'firefox', true], + ["default-src 'none'", 'http://h/w.js', true, 'chromium', false], + ["default-src 'none'; script-src 'self'", 'http://h/w.js', false, 'chromium', true], + ["script-src-elem 'none'; script-src 'self'", 'http://h/w.js', true, 'chromium', false], + ["script-src 'none' 'self'", 'http://h/w.js', true, 'chromium', true], + ["script-src 'nonce-k'", 'http://h/w.js', true, 'chromium', false], + ["script-src 'nonce-k' 'strict-dynamic'", 'http://h/w.js', false, 'webkit', true], + ["script-src 'sha256-AAAA' 'unsafe-inline' 'unsafe-eval' data: blob:", 'http://h/w.js', true, 'chromium', false], + ['script-src *', 'http://h/w.js', true, 'chromium', true], + ['script-src *', 'ftp://h/w.js', true, 'chromium', false], + ['script-src http:', 'http://h/w.js', true, 'chromium', true], + ['script-src http:', 'https://h/w.js', true, 'chromium', true], + ['script-src https:', 'http://h/w.js', true, 'chromium', false], + ['script-src http://h', 'http://h/w.js', true, 'chromium', true], + ['script-src HTTP://H/', 'http://h/w.js', true, 'chromium', true], + ['script-src http://h', 'https://h/w.js', true, 'chromium', true], + ['script-src https://h', 'http://h/w.js', true, 'chromium', false], + ['script-src h', 'http://h/w.js', true, 'chromium', true], + ['script-src h', 'ftp://h/w.js', true, 'chromium', false], + ['script-src g', 'http://h/w.js', true, 'chromium', false], + ['script-src http://*', 'http://h/w.js', true, 'chromium', true], + ['script-src *.h', 'http://a.h/w.js', true, 'chromium', true], + ['script-src *.h', 'http://h/w.js', true, 'chromium', false], + ['script-src *.h', 'http://ah/w.js', true, 'chromium', false], + ['script-src http://h:8080', 'http://h:8080/w.js', true, 'chromium', true], + ['script-src http://h:8080', 'http://h/w.js', true, 'chromium', false], + ['script-src http://h', 'http://h:8080/w.js', true, 'chromium', false], + ['script-src http://h:80', 'http://h/w.js', true, 'chromium', true], + ['script-src http://h:*', 'http://h:9/w.js', true, 'chromium', true], + ['script-src http://h/js/', 'http://h/js/w.js', true, 'chromium', true], + ['script-src http://h/js/', 'http://h/w.js', true, 'chromium', false], + ['script-src http://h/JS/', 'http://h/js/w.js', true, 'chromium', false], + ['script-src http://h/js/w.js', 'http://h/js/w.js', true, 'chromium', true], + ['script-src http://h/js/w.js', 'http://h/js/w.jsx', true, 'chromium', false], + ['script-src http://h/js%20a/', 'http://h/js%20a/w.js', true, 'chromium', false], + ['script-src http://h/j/', 'http://h/j/%61.js', true, 'chromium', false], + ["script-src 'self'; worker-src 'none'", 'http://h/w.js', true, 'chromium', true], + ["script-src 'self'; worker-src 'none'", 'http://h/w.js', false, 'chromium', false], + ["script-src 'self'; child-src 'none'", 'http://h/w.js', false, 'chromium', false], + ["script-src 'self'; worker-src 'self'; child-src 'none'", 'http://h/w.js', false, 'chromium', true], + ["default-src 'self'; sandbox allow-scripts", 'http://h/w.js', false, 'chromium', false], + ["default-src 'self'; require-trusted-types-for 'script'", 'http://h/w.js', true, 'chromium', false], + ["default-src 'self'; require-trusted-types-for 'script'", 'http://h/w.js', false, 'chromium', true], + ["default-src 'self'; trusted-types a", 'http://h/w.js', true, 'chromium', true], + ["script-src 'self', default-src 'none'", 'http://h/w.js', false, 'chromium', false], + ["script-src 'self'\ndefault-src 'self'", 'http://h/w.js', false, 'chromium', true], + ["default-src 'self', img-src 'none'", 'http://h/w.js', true, 'chromium', true], + ["img-src 'none'\ndefault-src 'none'", 'http://h/w.js', true, 'chromium', false], + ["script-src 'none'; script-src 'self'", 'http://h/w.js', true, 'chromium', false], + ["Default-Src 'none'", 'http://h/w.js', true, 'chromium', false], + ["require-trusted-types-for 'SCRIPT'", 'http://h/w.js', true, 'chromium', false], + ['script-src https://h:443', 'https://h/w.js', true, 'chromium', true], + ['script-src *', 'https://h/w.js', true, 'chromium', true], + ['script-src http://h/', 'http://h/a%20b.js', true, 'chromium', true], + [helmet, 'http://127.0.0.1:3000/w.js', false, 'chromium', true], + [helmet, 'http://localhost:3000/w.js', false, 'chromium', true], + [helmet, 'http://a.localhost:3000/w.js', false, 'chromium', true], + [helmet, 'http://127.9.0.1/w.js', true, 'chromium', true], + [helmet, 'http://[::1]:3000/w.js', false, 'chromium', true], + [helmet, 'http://127.0.0.1:3000/w.js', false, 'firefox', false], + [helmet, 'http://127.0.0.1:3000/w.js', false, 'webkit', false], + [helmet, 'http://127.0.0.1:3000/w.js', false, '', false], + [helmet, 'http://app.test:3000/w.js', false, 'chromium', false], + [helmet, 'http://128.0.0.1/w.js', false, 'chromium', false], + [helmet, 'https://app.test/w.js', false, 'webkit', true], + ]; + for (const [v, url, classic, engine, allows] of rows) assert.equal(cspAllowsOwn(csp(v), url, classic, engine), allows, `${v} | ${url} | ${classic ? 'classic' : 'module'} | ${engine}`); + assert.equal(cspAllowsOwn(csp("default-src 'self'"), 'not a url', true), false); +}); + +test('workerRoutes loads a Web Worker\'s seeding at its script\'s URL under a policy that allows a script from the worker\'s own origin, and says nothing; under one that could refuse it, puts the seeding on the script\'s first line, and names a module one', async () => { + const { r, context } = routed(); + const late = []; + await workerRoutes(context, { late: (...a) => late.push(a) }); + const t = 'ce'.repeat(16); + const self = { 'content-type': 'text/javascript', 'content-security-policy': "default-src 'self'" }; + const m = await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'import "./a.js";b()', self)); + assert.equal(m[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 0))};import "./a.js";b()\n//# sourceURL=http://h/m.js\n`); + const w = await r.ask(markWorker('http://h/w.js', 'w', t), answerOf(200, 'b()', self)); + assert.equal(w[1][1].body, `importScripts(${JSON.stringify(markWorker('http://h/w.js', 'q', t, 1))});b()\n//# sourceURL=http://h/w.js\n`); + assert.deepEqual(late, []); + // Trusted Types refuse a classic worker's importScripts of a string, not a module's import. + const tt = { 'content-type': 'text/javascript', 'content-security-policy': "default-src 'self'; require-trusted-types-for 'script'" }; + assert.equal((await r.ask(markWorker('http://h/w.js', 'w', t), answerOf(200, 'b()', tt)))[1][1].body, `${oneLine(workerScript(t, 2, 'w'))}b()\n//# sourceURL=http://h/w.js\n`); + assert.equal((await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'b()', tt)))[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 3))};b()\n//# sourceURL=http://h/m.js\n`); + assert.deepEqual(late, []); + // helmet's policy upgrades http: in Chromium (the routes' engine here) not on a loopback host. + const helmet = { 'content-type': 'text/javascript', 'content-security-policy': "default-src 'self';script-src 'self';upgrade-insecure-requests" }; + assert.equal((await r.ask(markWorker('http://127.0.0.1:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `import ${JSON.stringify(markWorker('http://127.0.0.1:9/m.js', 'p', t, 4))};b()\n//# sourceURL=http://127.0.0.1:9/m.js\n`); + assert.equal((await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `${oneLine(workerScript(t, 5, 'm'))}b()\n//# sourceURL=http://h/m.js\n`); + assert.deepEqual(late, [['http://h/m.js', false]]); + // In Firefox and WebKit, a loopback host too. + for (const engine of ['firefox', 'webkit']) { + const e = routed(); + const said = []; + await workerRoutes(e.context, { late: (...a) => said.push(a) }, engine); + assert.equal((await e.r.ask(markWorker('http://127.0.0.1:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `${oneLine(workerScript(t, 0, 'm', engine === 'webkit'))}b()\n//# sourceURL=http://127.0.0.1:9/m.js\n`, engine); + assert.deepEqual(said, [['http://127.0.0.1:9/m.js', false]], engine); + } +}); + +test('workerRoutes answers a Shared Worker whose script\'s policy could refuse the load of the app\'s script with the app\'s script itself, its seeding on the first line (kept strict), and names a module one; one whose policy allows it loads the app\'s script as before', async () => { + const { r, context } = routed(); + const late = []; + const answered = []; + await workerRoutes(context, { late: (...a) => late.push(a), answered: (q) => answered.push(q.url()) }); + const t = 'cf'.repeat(16); + const none = { 'content-type': 'text/javascript', 'content-security-policy': "default-src 'none'" }; + const s = await r.ask(markWorker('http://h/s.js', 's', t), answerOf(200, '"use strict";app()', none)); + assert.deepEqual(s, [['fetch', 'http://h/s.js'], ['fulfill', { status: 200, headers: none, body: `"use strict";${oneLine(workerScript(t, 0, 's', false, true))}"use strict";app()\n//# sourceURL=http://h/s.js\n` }]]); + assert.equal((await r.ask(markWorker('http://h/s.js', 's', t), answerOf(200, 'app()', none)))[1][1].body, `${oneLine(workerScript(t, 1, 's', false, true))}app()\n//# sourceURL=http://h/s.js\n`); + assert.deepEqual(late, []); + const x = await r.ask(markWorker('http://h/x.js', 'x', t), answerOf(200, 'import "./a.js";b()', none)); + assert.equal(x[1][1].body, `${oneLine(workerScript(t, 2, 'x', false, true))}import "./a.js";b()\n//# sourceURL=http://h/x.js\n`); + assert.deepEqual(late, [['http://h/x.js', true]]); + // Its bytes as the server sent them. + const latin1 = Buffer.from('app("hé")', 'latin1'); + const l = await r.ask(markWorker('http://h/l.js', 's', t), { status: () => 200, ok: () => true, headers: () => none, body: async () => latin1 }); + assert.deepEqual(r.raw, withBytes(latin1, oneLine(workerScript(t, 3, 's', false, true)), '\n//# sourceURL=http://h/l.js\n')); + assert.equal(l[1][0], 'fulfill'); + // Trusted Types refuse a classic one's importScripts of a string; a module one's import() is no sink. + const tt = { 'content-type': 'text/javascript', 'content-security-policy': "require-trusted-types-for 'script'" }; + assert.equal((await r.ask(markWorker('http://h/t.js', 's', t), answerOf(200, 'app()', tt)))[1][1].body, `${oneLine(workerScript(t, 4, 's', false, true))}app()\n//# sourceURL=http://h/t.js\n`); + assert.equal((await r.ask(markWorker('http://h/u.js', 'x', t), answerOf(200, 'app()', tt)))[1][1].body, workerAnswer(t, 5, 'x', 'http://h/u.js', true)); + // A policy that allows a script from the worker's origin: the app's script is loaded from its own URL, as before. + const self = { 'content-type': 'text/javascript', 'content-security-policy': "script-src 'self'" }; + assert.equal((await r.ask(markWorker('http://h/v.js', 's', t), answerOf(200, 'app()', self)))[1][1].body, workerAnswer(t, 6, 's', 'http://h/v.js')); + assert.equal((await r.ask(markWorker('http://h/y.js', 'x', t), answerOf(200, 'app()', self)))[1][1].body, workerAnswer(t, 7, 'x', 'http://h/y.js', false)); + assert.deepEqual(late, [['http://h/x.js', true]]); + // A body keploy cannot read: the seeding's load of the app's script, as before, and no line. + const gone = await r.ask(markWorker('http://h/g.js', 'x', t), { status: () => 200, ok: () => true, headers: () => none, body: async () => { throw new Error('gone'); } }); + assert.equal(gone[1][1].body, workerAnswer(t, 8, 'x', 'http://h/g.js', false)); + assert.deepEqual(late, [['http://h/x.js', true]]); + assert.equal(answered.length, 9); +}); + +test('an inline Shared Worker\'s answer runs its seeding, then the app\'s script, at the worker\'s marked URL: it draws the seed\'s numbers, and its location and an ErrorEvent\'s filename there show no mark', () => { + const t = 'd0'.repeat(16); + for (const kind of ['s', 'x']) { + const href = markWorker('http://h/s.js?q=1', kind, t); + const w = workerAt(href, workerAnswer(t, 2, kind, 'http://h/s.js?q=1', true, 'globalThis.ran = location.href;', true)); + assert.equal(w.run('ran'), 'http://h/s.js?q=1', kind); + assert.deepEqual(w.draws(), workerAt(href, workerScript(t, 2, kind)).draws(), kind); + // In place, the app's script runs at the marked URL: an ErrorEvent's filename there shows it with no mark. + class ErrorEvent {} + const e = workerAt(href, '', { ErrorEvent }); + e.run('Object.defineProperty(ErrorEvent.prototype, \'filename\', { get() { return this.f; }, enumerable: true, configurable: true });'); + e.run(workerScript(t, 2, kind, false, true)); + assert.equal(e.run(`(() => { const v = new ErrorEvent(); v.f = ${JSON.stringify(href)}; return v.filename; })()`), 'http://h/s.js?q=1', kind); + } + assert.match(workerScript(t, 0, 's', false, true), /const main = new RegExp\(/); + assert.match(workerScript(t, 0, 's'), /const main = null;/); + assert.match(lateLine('a test', 'http://h/x.js', true), /^keploy: in a test, the module Shared Worker at http:\/\/h\/x\.js is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, .* runs after the modules the script imports: .*; if one does, have the policy allow scripts from the worker's own origin \('self' in script-src\) where the tests run, or serve the worker's script with no Content-Security-Policy there/); + assert.match(lateLine('a test', 'http://h/m.js'), /^keploy: in a test, the module Web Worker at /); }); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index c5a22c0..eddcfa0 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -699,6 +699,22 @@ async function drawsServer() { if (u.pathname === '/sv/l1.js') { res.writeHead(200, { 'content-type': 'text/javascript; charset=iso-8859-1' }); res.end(Buffer.from(told('l1', ", s: 'h\u00e9'"), 'latin1')); return; } const csp = { '/sv/cn.js': "default-src 'none'", '/sv/csn.js': "script-src 'none'", '/sv/ctt.js': "require-trusted-types-for 'script'" }[u.pathname]; if (csp !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', 'content-security-policy': csp }); res.end(told(u.pathname.slice(4, -3))); return; } + // /mc/s.js and /mc/h.js, module Web Workers that import /mc/dep.mjs, + // which draws as it loads, under default-src 'self' and under helmet's + // default policy; /mc/sd.js (a module) and /mc/sdc.js (a classic one) + // under a nonce and 'strict-dynamic'; /sv/sn.js and /sv/snm.js, a + // classic and a module Shared Worker, under default-src 'none'. + const helmetCsp = "default-src 'self';base-uri 'self';font-src 'self' https: data:;form-action 'self';frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests"; + const ownCsp = { + '/mc/s.js': ["default-src 'self'", `import { D } from './dep.mjs'; ${told('s', ', dep: D')}`], + '/mc/h.js': [helmetCsp, `import { D } from './dep.mjs'; ${told('h', ', dep: D')}`], + '/mc/sd.js': ["script-src 'nonce-k' 'strict-dynamic'", told('sd')], + '/mc/sdc.js': ["script-src 'nonce-k' 'strict-dynamic'", told('sdc')], + '/sv/sn.js': ["default-src 'none'", `onconnect = (e) => e.ports[0].${told('sn')}`], + '/sv/snm.js': ["default-src 'none'", `onconnect = (e) => e.ports[0].${told('snm')}`], + }[u.pathname]; + if (ownCsp !== undefined) { res.writeHead(200, { 'content-type': 'text/javascript', 'content-security-policy': ownCsp[0] }); res.end(ownCsp[1]); return; } + if (u.pathname === '/mc/dep.mjs') { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end('export const D = [Math.random()];'); return; } const moved = { '/rd/w.js': '/rd/v2/w.js', '/rdm/m.js': '/rdm/v2/m.js', '/rds/s.js': 'v2/s.js' }[u.pathname]; if (moved !== undefined) { res.writeHead(302, { location: moved }); res.end(); return; } const v2 = { @@ -1792,7 +1808,9 @@ for (const engine of ENGINES) { const said = (r, re) => r.lines.filter((l) => re.test(l)); for (const r of [s1, s2]) { assert.equal(said(r, /^keploy: in .*csp.*, the module Web Worker at http:\/\/127\.0\.0\.1:\d+\/sv\/cn\.js is served with a Content-Security-Policy/).length, 1); - assert.equal(said(r, /Content-Security-Policy/).length, 1); + // served.spec.ts's 'own' test runs too: its lines name /mc/ and /sv/sn*.js (its own row). + assert.equal(said(r, /\/sv\/c[a-z]+\.js is served with a Content-Security-Policy/).length, 1); + assert.equal(said(r, /Content-Security-Policy/).filter((l) => !/\/mc\/|\/sv\/snm?\.js/.test(l)).length, 1); // Firefox routes no Shared Worker's script: its redirected one is named as one keploy cannot reach. const moved = engine === 'firefox' ? ['/rd/w.js', '/rdm/m.js'] : ['/rd/w.js', '/rdm/m.js', '/rds/s.js']; for (const p of moved) { @@ -1810,6 +1828,47 @@ for (const engine of ENGINES) { }); } +for (const engine of ENGINES) { + test(`served, in ${engine}: a worker's script whose Content-Security-Policy allows a script from the worker's own origin keeps its seeding ahead of the modules it imports (default-src 'self'; a nonce and 'strict-dynamic', module and classic; helmet's default${engine === 'chromium' ? '' : ', which upgrades http, as one keploy cannot tell allows it here: seeded late and named, as it runs off'}), draws the seed's numbers there, its imports' too, and is named as late by no line; a Shared Worker under default-src 'none', classic and module, starts, as off, ${engine === 'firefox' ? 'unseeded (Firefox routes no Shared Worker\'s script)' : 'seeded, the module one named as seeded late once'}; the server never sees keploy's mark`, async () => { + const j = journal(); + const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: 'ce'.repeat(16) } } : undefined)); + try { + const run = (off) => drawRun(j, { ...keployEnv(a.url), PW_BROWSER: engine, ...(off ? { KEPLOY_SEED_RANDOM: 'off' } : {}) }, ['served.spec.ts'], ['own']); + const s1 = await run(false); + const s2 = await run(false); + const o1 = await run(true); + const o2 = await run(true); + const names = ['s', 'h', 'sd', 'sdc', 'sn', 'snm']; + const kinds = (r) => r.drew.own.map((w) => (typeof w === 'string' ? w : w.k)); + for (const [mode, r] of [['seeded', s1], ['seeded again', s2], ['off', o1], ['off again', o2]]) { + // Each starts as it does off; all but helmet's in every engine. + assert.deepEqual(kinds(r), kinds(o1), mode); + assert.deepEqual(kinds(r).filter((k, i) => i !== 1), names.filter((k, i) => i !== 1), mode); + assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); + // Each script loaded once by the browser, as off (a Shared Worker's twice seeded in Chromium and WebKit: keploy asks it too). + for (const p of ['/mc/s.js', '/mc/sd.js', '/mc/sdc.js']) assert.equal(r.seen.filter((u) => u === p).length, 1, `${mode} ${p}`); + } + if (engine === 'chromium') assert.equal(kinds(s1)[1], 'h'); + const at = (r, i) => r.drew.own[i]; + // Seeded: the seed's draws, the same in each run, an import's too; off, the browser's own. + const seededAt = engine === 'chromium' ? [0, 1, 2, 3, 4, 5] : engine === 'webkit' ? [0, 2, 3, 4, 5] : [0, 2, 3]; + for (const i of seededAt) { + assert.deepEqual(at(s1, i).draws, at(s2, i).draws, names[i]); + if (at(s1, i).dep !== undefined) assert.deepEqual(at(s1, i).dep, at(s2, i).dep, `${names[i]} dep`); + } + assert.notDeepEqual(at(o1, 0).draws, at(o2, 0).draws); + assert.notDeepEqual(at(o1, 0).dep, at(o2, 0).dep); + if (engine === 'firefox') for (const i of [4, 5]) assert.notDeepEqual(at(s1, i).draws, at(s2, i).draws, names[i]); + // The lines: a module worker seeded after its imports, once each. + // served.spec.ts's other tests run too: only this test's addresses (/mc/, /sv/sn.js, /sv/snm.js). + const late = (r) => r.lines.filter((l) => /is served with a Content-Security-Policy/.test(l) && /\/mc\/|\/sv\/snm?\.js/.test(l)).map((l) => /the (module (?:Web|Shared) Worker) at http:\/\/127\.0\.0\.1:\d+(\S+) is served/.exec(l)?.slice(1).join(' ')).sort(); + const want = { chromium: ['module Shared Worker /sv/snm.js'], firefox: ['module Web Worker /mc/h.js'], webkit: ['module Shared Worker /sv/snm.js', 'module Web Worker /mc/h.js'] }[engine]; + for (const r of [s1, s2]) assert.deepEqual(late(r), want); + for (const r of [o1, o2]) assert.deepEqual(late(r), []); + } finally { await a.close(); } + }); +} + // overrides.spec.ts's tests, and the scripts each one's page starts its // workers at (/ov/<k>w.js and /ov/<k>s.js). const OV = [['continue headers on page', 'a'], ['continue headers on context', 'a'], ['continue url on page', 'o'], ['continue url on context', 'o'], From c10b4f1d065c09f03edc700fec6d6c34464d1450 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 22:46:33 +0530 Subject: [PATCH 64/73] test(playwright): match a CSP host source with no wildcard only itself A row of the worker policy table where a host source with no wildcard meets its own host (allowed), a subdomain of it, and a longer name that ends in it (both refused). Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/test/random.test.mjs | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 81bd491..89d9881 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3196,6 +3196,11 @@ test('cspAllowsOwn: a policy that allows a script from the worker\'s own origin ['script-src *.h', 'http://a.h/w.js', true, 'chromium', true], ['script-src *.h', 'http://h/w.js', true, 'chromium', false], ['script-src *.h', 'http://ah/w.js', true, 'chromium', false], + // A host with no wildcard matches only itself: not a subdomain, nor a + // longer name that ends in it. + ['script-src h', 'http://h/w.js', true, 'chromium', true], + ['script-src h', 'http://a.h/w.js', true, 'chromium', false], + ['script-src h', 'http://ah/w.js', true, 'chromium', false], ['script-src http://h:8080', 'http://h:8080/w.js', true, 'chromium', true], ['script-src http://h:8080', 'http://h/w.js', true, 'chromium', false], ['script-src http://h', 'http://h:8080/w.js', true, 'chromium', false], From fe83dcb93cd486413f78c3660b227443b69ec204 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 23:27:41 +0530 Subject: [PATCH 65/73] fix(playwright): end settle's wait on a request a route answered A test whose own route answered a request with a redirect waited out its timeout at the end: settle waits for the browser to report each call's end, and none comes. Chromium never ends a Web Worker's script load a route fulfills with a 302 or 307 (its worker never starts, with no keploy too), Firefox never ends the redirect's first request, and WebKit refuses such a fulfill, whose error fails the test, and leaves the request with the route. Seeded and with seeding off alike. settle now watches each route Playwright hands the test's pages and context. Once a route's fulfill returns, its request has the whole answer the route gave and nothing of it is on the network, so settle stops waiting on it. Once a route's handler throws while the route still holds its request (no route answered it or sent it on), Playwright sends it nowhere, so settle stops waiting on that one too. A handler that returns and leaves the request held might still send it on, so settle waits on it as on any other call. A context Playwright reuses between tests has its routes watched by the current test's settle alone. On a Playwright without this route handling, settle waits as before. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 4 +- playwright/src/settle.ts | 65 +++++++++++- playwright/test/fixtures/settle.spec.ts | 46 ++++++++ playwright/test/rows.test.mjs | 54 +++++++++- playwright/test/unit.test.mjs | 135 ++++++++++++++++++++++++ 5 files changed, 298 insertions(+), 6 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index f6e565d..ee029ca 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -43,7 +43,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim: each test is skipped at its begin, after the group's hooks have started. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. - **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped <titles>)`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. -- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. +- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor once a route of the test's own has answered it (its `route.fulfill` returned): the call has the route's whole answer, though the browser may report no end for it (Chromium for a Web Worker's script a route answers with a redirect, whose worker never starts, with no keploy too; Firefox for the first request of a redirect a route answers). Nor once the route's handler has thrown while the route still holds the call (WebKit refuses a redirect, and Playwright's `fulfill` throws its own error), as Playwright then sends it nowhere. A handler that returns and leaves its route holding the call might still send it on, so the call is waited for as any other. Settle reads this from Playwright's own route handling, past its API; with a Playwright that has none, such a call is waited for as any other. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see "What the run says when ..." below). @@ -98,7 +98,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - `given` needs the agent route `POST /agent/scope/given` (keploy/keploy#5016): keploy v3.6.192 and later serve it; with the enterprise keploy, that is an enterprise release on keploy v3.6.192 or later. An older keploy answers that route with a plain 404, and `given` fails the test with `keploy: this keploy agent answered 404 to POST /agent/scope/given, so it cannot take the given value <name>: given needs keploy v3.6.192 or later (keploy/keploy#5016), or an enterprise release on keploy v3.6.192 or later; an older keploy answers 404 there`. Claim, begin, settle and end do not need that route. - Seeds need a keploy that keeps them: keploy v3.6.217 (keploy/keploy#5121) and later; with the enterprise keploy, an enterprise release on keploy v3.6.217 or later. An older keploy reads no seed from a begin and answers none, so each test's browser and test process draw their own random numbers, and each test says so (`keploy: keploy answered the begin of <test> with no seed, ...`, above). Claim, begin, settle and end work with it as before. - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). -- **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it or the document that made it is gone (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. +- **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it, the document that made it is gone, or a route of the test's has answered it or holds it after its handler threw (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. - **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. A module Shared Worker whose seeding is on its script's first line (below) reads its marked URL so too. With seeding off, or for a classic Web Worker (its `location`) or another Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a module worker's script is served with a Content-Security-Policy that could refuse keploy's load.** When keploy cannot tell that the policy allows a script from the worker's own origin (`'self'` in `script-src` or `default-src` does; the rest under Workers, above), keploy puts the worker's seeding on its script's first line, and in a module that line runs after the modules the script imports: a random number one of those modules draws as it loads is the browser's own, so it may differ at replay. This holds for a module Web Worker and, in Chromium and WebKit, a module Shared Worker. The test says so once for each address (`keploy: in <test>, the module Web Worker at <address> is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, ...`, or `module Shared Worker`), with the next step: if one of those modules draws as it loads, have the policy allow scripts from the worker's own origin (`'self'` in `script-src`) where the tests run, or serve the worker's script with no Content-Security-Policy there, and keploy loads its seeding ahead of those modules. A classic worker's seeding on its first line runs before any of the app's code. diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index b8b2982..6f23141 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -4,7 +4,7 @@ // whole would be cut at a different byte each run). Counted by the // browser's own request events; no time window decides it. -import type { BrowserContext, Frame, Page, Request } from '@playwright/test'; +import type { BrowserContext, Frame, Page, Request, Route } from '@playwright/test'; import { workerMark } from './random'; export class Inflight { @@ -97,7 +97,15 @@ export class Inflight { this.docs.delete(f); this.drop((o) => o.frame === f); }); + onRoute(p, (r, threw) => this.routed(r, threw)); }; + // Nor for a request a route answers itself with a redirect (Chromium, + // for a Web Worker's script: its worker never starts, with no keploy + // too; Firefox, for the redirect's first request), nor for one a route + // holds after its handler threw (WebKit refuses a redirect, and + // Playwright's fulfill throws): routed watches each route a page's + // routes, and then its context's, are handed. + onRoute(context, (r, threw) => this.routed(r, threw)); context.pages().forEach(watch); context.on('page', watch); // Nor for any call when the test closes its own context, which ends its @@ -204,6 +212,32 @@ export class Inflight { return ds[ds.length - 1]; } + /** + * Watches route, a request's route. Once its fulfill returns, the + * request has the whole answer the route gave, and nothing of it is in + * flight, whatever the browser reports after. threw: the routes' handler + * threw (called again, after the first call); a request its route still + * holds then (no route answered it or sent it on: the browser refused + * its answer, as WebKit refuses a redirect, or the handler threw first) + * goes nowhere, as Playwright sends it on no further. A handler that + * returns and leaves the request held keeps it in flight, as one that + * still might send it on: the test's timeout ends that wait, naming it. + */ + private routed(route: Route, threw: boolean): void { + const r = route.request(); + if (threw) { + // Playwright's own, past its API: set while the handler may still + // answer the request, cleared once a route answered it or sent it on. + if ((route as unknown as { _handlingPromise?: unknown })._handlingPromise) this.answered(r); + return; + } + const fulfill = route.fulfill; + route.fulfill = async (o) => { + await fulfill.call(route, o); + this.answered(r); + }; + } + private done(r: Request): void { if (this.open.delete(r)) this.wake(); } @@ -224,6 +258,35 @@ export class Inflight { } } +// Each page's and context's route watcher: the newest Inflight's (a context +// Playwright reuses between tests has one test's at a time). +const routeWatch = new WeakMap<object, (route: Route, threw: boolean) => void>(); + +/** + * Has watch see each route Playwright hands target (a page, or a context) + * before any of target's routes does, and again (threw) when one of their + * handlers threw: target's _onRoute, Playwright's own, past its API. Where + * a version has none, no route is watched, and settle waits on such a + * request as it waits on any other. + */ +function onRoute(target: object, watch: (route: Route, threw: boolean) => void): void { + const t = target as { _onRoute?: (route: Route) => Promise<void> }; + const on = t._onRoute; + if (typeof on !== 'function') return; + const was = routeWatch.has(target); + routeWatch.set(target, watch); + if (was) return; + t._onRoute = async function (this: unknown, route: Route): Promise<void> { + routeWatch.get(target)?.(route, false); + try { + await on.call(this, route); + } catch (e) { + routeWatch.get(target)?.(route, true); + throw e; + } + }; +} + type Doc = Map<string, { loads: Request[]; stopped: Set<number> }>; function newDoc(): Doc { diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index ea6bf2c..97d6521 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -286,3 +286,49 @@ test('two Web Workers at one URL, the second terminated while their scripts load await reached(); await page.evaluate(() => { (globalThis as unknown as { w: Worker[] }).w[1].terminate(); }); }); +// The test's own routes answer a classic Web Worker's script with a 302, a +// module one's with a 307 and a page fetch with a 302 (each to a URL the +// page server does not know). Chromium reports no end for such a worker +// load (its worker never starts), Firefox none for the redirect's first +// request, and WebKit refuses the answer: Playwright's own error fails the +// test there, and the request stays with the route. +test("requests the test's route answers with a redirect", async ({ page }) => { + await page.goto(`${base}/page`); + await page.route('**/rw.js', (r) => r.fulfill({ status: 302, headers: { location: `${base}/v2/rw.js` } })); + await page.route('**/rwm.js', (r) => r.fulfill({ status: 307, headers: { location: `${base}/v2/rwm.js` } })); + await page.route('**/rf', (r) => r.fulfill({ status: 302, headers: { location: `${base}/v2/rf` } })); + const sent = Promise.all([/\/rw\.js(\?|$)/, /\/rwm\.js(\?|$)/, /\/rf$/].map((u) => page.waitForRequest(u))); + await page.evaluate(() => { + const g = globalThis as unknown as { w: Worker[] }; + g.w = [new Worker('/rw.js'), new Worker('/rwm.js', { type: 'module' })]; + void fetch('/rf').catch(() => {}); + }); + await sent; +}); +// A route whose answer the browser refused (WebKit refuses a redirect) +// sends the call on; the server has it before the body ends. +test("a call the test's route sends on after the browser refused its answer", async ({ page }) => { + await page.goto(`${base}/page`); + await page.route('**/slow', async (r) => { + await r.fulfill({ status: 302, headers: { location: `${base}/page` } }).catch(() => {}); + await r.continue(); + }); + const sent = page.waitForRequest(/\/slow$/); + await page.evaluate(() => { void fetch('/slow'); }); + await sent; + await reached(); +}); +// A route sends the call on, then tries to answer it too: Playwright's +// fulfill throws (the route is already handled), and so does the handler, +// which fails the test. +test("a call the test's route sends on and then tries to answer", async ({ page }) => { + await page.goto(`${base}/page`); + await page.route('**/slow', async (r) => { + await r.continue(); + await r.fulfill({ status: 302, headers: { location: `${base}/page` } }); + }); + const sent = page.waitForRequest(/\/slow$/); + await page.evaluate(() => { void fetch('/slow'); }); + await sent; + await reached(); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index eddcfa0..573de03 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -497,20 +497,68 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { } } +// A request the test's own route answers with a redirect (route.fulfill +// 302 or 307): Chromium reports no end for a Web Worker's script load so +// answered, Firefox none for the redirect's first request (a page fetch's +// too), and WebKit refuses the answer, with Playwright's own error, and the +// request stays with the route. Settle waits on none of them, seeded or off: +// a wait shows as settle's step naming the request (and a timeout at the +// config's 30 s: the row sets no short timeout). +for (const engine of ['chromium', 'firefox', 'webkit']) { + for (const seed of ['seeded', 'off']) { + test(`settle, in ${engine}, ${seed}: a request the test's own route answers with a redirect (a classic Web Worker's script with a 302, a module one's with a 307, a page fetch with a 302) is not waited for, and the test ${engine === 'webkit' ? "fails with Playwright's own error (WebKit refuses the answer)" : 'passes'}`, async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = "requests the test's route answers with a redirect"; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), ...(seed === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}), SLOW_BASE: p.url, PW_BROWSER: engine }); + const t = tests[title]; + if (engine === 'webkit') { + assert.equal(t.status, 'failed', out); + assert.match(t.error, /Cannot fulfill with redirect status/, out); + } else { + assert.equal(t.status, 'passed', out); + } + const waited = t.steps.filter((x) => x.startsWith('keploy: wait for') && /\/(rw\.js|rwm\.js|rf)(\?|,|\s|$)/.test(x)); + assert.deepEqual(waited, [], t.steps.join('\n')); + } finally { await p.close(); await a.close(); } + }); + } +} + +// A request the test's route sends on (continue) is waited for, though a +// fulfill of the route failed: in every engine, one the route had sent on +// before, whose handler then throws Playwright's "already handled" (which +// fails the test); in WebKit, one whose redirect the browser refused before +// the route sent it on. The server answers it once settle's step has begun +// (waitedFor). +for (const engine of ['chromium', 'firefox', 'webkit']) { + test(`settle, in ${engine}: a call the test's own route sends on is waited for, though the route's fulfill failed (after it was sent on, its handler throwing${engine === 'webkit' ? '; or refused before, as WebKit refuses a redirect' : ''})`, async () => { + await waitedFor("a call the test's route sends on and then tries to answer", 'GET', '/slow', engine, /Route is already handled/); + if (engine === 'webkit') await waitedFor("a call the test's route sends on after the browser refused its answer", 'GET', '/slow', engine); + }); +} + // Settle rows where a call is still waited for: the page server answers it // only once settle's step has begun (the rows' reporter asks /release then), // so the test ends by settle's own wait, with no deadline on its body. The // test passes, settle's step names the call (method, URL), and the test's // end comes after the call's answer. A settle that did not wait for it ends // the test with the call unanswered. -async function waitedFor(title, method, call) { +async function waitedFor(title, method, call, engine = '', error = null) { const j = journal(); const a = await fakeAgent(j); const p = await pageServer(j); try { - const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', RELEASE_AT_STEP: `${p.url}/release` }); + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', RELEASE_AT_STEP: `${p.url}/release`, ...(engine === '' ? {} : { PW_BROWSER: engine }) }); const t = tests[title]; - assert.equal(t.status, 'passed', out); + if (error === null) { + assert.equal(t.status, 'passed', out); + } else { + assert.equal(t.status, 'failed', out); + assert.match(t.error, error, out); + } const named = new RegExp(`^keploy: wait for the test's 1 call\\(s\\) in flight before its browser context closes: ${method} http://127\\.0\\.0\\.1:\\d+${call}$`); assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n')); const name = call.slice(1); diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 273f9bf..dab50e1 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -277,6 +277,141 @@ async function ticks() { for (let i = 0; i < 5; i++) await Promise.resolve(); } +// A fake Playwright route, as far as settle reads it: _handlingPromise is +// set while a handler may still answer it; fulfill throws as Playwright's +// does when the browser refuses the answer (refuse: WebKit's redirect, the +// route still the handler's) or when the route was already handled. +function fakeRoute(req) { + const route = { + _handlingPromise: {}, + request: () => req, + async fulfill(o = {}) { + if (!route._handlingPromise) throw new Error('Route is already handled!'); + if (o.refuse) throw new Error('Cannot fulfill with redirect status: 302'); + route._handlingPromise = null; + }, + async continue() { + if (!route._handlingPromise) throw new Error('Route is already handled!'); + route._handlingPromise = null; + }, + }; + return route; +} +// A page or context with Playwright's _onRoute, which keeps each route it +// is handed (seen) and runs target.handler on it, as Playwright runs the +// target's routes: a handler's throw rejects it; a page's handler that +// answers 'next' hands the route on to its context's (ctx). +function withRoutes(target, ctx = null) { + target.seen = []; + target._onRoute = async function (route) { + target.seen.push(route); + const next = target.handler ? await target.handler(route) : undefined; + if (next === 'next' && ctx !== null) await ctx._onRoute(route); + }; + return target; +} + +test('settle: a request a route of the test\'s answered (fulfill returned) has ended, though the browser reports no end; one the route still holds when its handler throws (the browser refused its answer, or nothing answered it) is not waited for; one the handler holds, or sent on, before or after a refused answer, is', async () => { + const ctx = withRoutes(fakeContext()); + const inflight = new Inflight(ctx); + const page = withRoutes(new EventEmitter()); + ctx.emit('page', page); + const sent = async (handler) => { + const r = fakeRequest(page); + ctx.emit('request', r); + const route = fakeRoute(r); + page.handler = handler; + const done = page._onRoute(route).then(() => 'returned', (e) => e.message); + return { r, route, done }; + }; + // Answered: fulfill returned. + const a = await sent((route) => route.fulfill({ status: 302 })); + assert.equal(await a.done, 'returned'); + assert.deepEqual(page.seen, [a.route]); + await ticks(); + assert.equal(inflight.size, 0); + // Refused, and the handler throws: held by the route, sent nowhere. + const b = await sent((route) => route.fulfill({ refuse: true })); + assert.match(await b.done, /Cannot fulfill/); + assert.equal(inflight.size, 0); + // A handler that throws before it answers. + const c = await sent(async () => { throw new Error('the handler'); }); + assert.equal(await c.done, 'the handler'); + assert.equal(inflight.size, 0); + // Refused, and the handler catches it and sends the request on. + const d = await sent(async (route) => { await route.fulfill({ refuse: true }).catch(() => {}); await route.continue(); }); + assert.equal(await d.done, 'returned'); + assert.equal(inflight.size, 1); + ctx.emit('requestfinished', d.r); + assert.equal(inflight.size, 0); + // Refused, and the handler returns holding it: it may still send it on. + const e = await sent((route) => route.fulfill({ refuse: true }).catch(() => {})); + assert.equal(await e.done, 'returned'); + assert.equal(inflight.size, 1); + await e.route.continue(); + ctx.emit('requestfinished', e.r); + assert.equal(inflight.size, 0); + // Sent on, then the handler throws (a fulfill Playwright refuses): in flight. + const f = await sent(async (route) => { await route.continue(); await route.fulfill({}); }); + assert.match(await f.done, /already handled/); + assert.equal(inflight.size, 1); + ctx.emit('requestfinished', f.r); + assert.equal(inflight.size, 0); +}); + +test('settle: a route the page\'s routes hand on reaches its context\'s, watched there too: one the context\'s handler answers has ended; one it still holds when it throws is not waited for', async () => { + const ctx = withRoutes(fakeContext()); + const inflight = new Inflight(ctx); + const page = withRoutes(new EventEmitter(), ctx); + ctx.emit('page', page); + page.handler = () => 'next'; + for (const [handler, ended] of [[(route) => route.fulfill({}), 'returned'], [(route) => route.fulfill({ refuse: true }), 'Cannot fulfill with redirect status: 302']]) { + const r = fakeRequest(page); + ctx.emit('request', r); + const route = fakeRoute(r); + ctx.handler = handler; + assert.equal(await page._onRoute(route).then(() => 'returned', (e) => e.message), ended); + assert.equal(ctx.seen.at(-1), route); + assert.equal(inflight.size, 0, ended); + } + // The context alone (a service worker's request). + const r = fakeRequest(null); + ctx.emit('request', r); + ctx.handler = (route) => route.fulfill({}); + await ctx._onRoute(fakeRoute(r)); + assert.equal(inflight.size, 0); +}); + +test('settle: a page or context with no _onRoute (a Playwright without it) gets none, and its requests are waited for as before', async () => { + const ctx = fakeContext(); + const inflight = new Inflight(ctx); + const page = new EventEmitter(); + ctx.emit('page', page); + assert.equal('_onRoute' in ctx, false); + assert.equal('_onRoute' in page, false); + const r = fakeRequest(page); + ctx.emit('request', r); + assert.equal(inflight.size, 1); +}); + +test('settle: of two Inflights on one context (Playwright reuses it between tests), the newest watches its routes; its _onRoute is wrapped once', async () => { + const ctx = withRoutes(fakeContext()); + const first = new Inflight(ctx); + const wrapped = ctx._onRoute; + const second = new Inflight(ctx); + assert.equal(ctx._onRoute, wrapped); + const r = fakeRequest(null); + ctx.emit('request', r); + assert.equal(first.size, 1); + assert.equal(second.size, 1); + const route = fakeRoute(r); + await ctx._onRoute(route); + assert.deepEqual(ctx.seen, [route]); + await route.fulfill({}); + assert.equal(second.size, 0); + assert.equal(first.size, 1); +}); + test('settle: a call keploy\'s route answered has ended, though the browser reports no end (Chromium, a Shared Worker\'s script); one reported after its answer is not waited for', async () => { const ctx = fakeContext(); const inflight = new Inflight(ctx); From 8cc6955f88d2d77f065b5ab18fff943f80a6fca3 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 23:40:24 +0530 Subject: [PATCH 66/73] fix(playwright): seed first under upgrade-insecure-requests on localhost helmet's default Content-Security-Policy has upgrade-insecure-requests. On an http worker script served with it, keploy counted the policy as one that could refuse its own load of the seeding in Firefox and WebKit, so a module worker there was seeded only after its imports ran, and the test said so, where Chromium kept the seeding first on a loopback host. Measured in Firefox and WebKit on localhost and on 127.0.0.1: neither upgrades a worker's http load of its own origin under that policy, and every Web Worker (classic and module, with importScripts and with imports) ran seeded with the seed's numbers. keploy now keeps the seeding first there too. Other loopback names (a name under localhost, the rest of 127.0.0.0/8, [::1]) are not measured in those engines and still count as refusing: seeded late, and named. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 2 +- playwright/src/random.ts | 11 ++++++++--- playwright/test/random.test.mjs | 30 +++++++++++++++++++++++------- playwright/test/rows.test.mjs | 12 ++++++------ 4 files changed, 38 insertions(+), 17 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index ee029ca..b7bb493 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -67,7 +67,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco The name names the instance's stream, so give each instance its own: two given one name make the same data. An instance given as a file loads, in `beforeAll` or while a worker fixture is set up is seeded in every test after; one given in a test (its `beforeEach`, its body, a test fixture) until that test's end. One the suite makes inside a test needs no call: it seeds its generator from the test's seeded `Math.random`. The worker's seeded line names `seedFaker` for these instances, and `seedFaker` throws, naming the next step, when it is given no faker instance it can seed or no name. - **A recording that kept no seed** (made before keploy kept seeds, or with seeding off), a test pulled from a Keploy branch (the branch keeps no seed), or a keploy from before seeds, answers none. That test's browser and test process then draw their own numbers, and the test prints `keploy: keploy answered the begin of <test> with no seed, so its browser and its test process draw their own random numbers: ...`. A recording made before seeds or with seeding off is seeded by recording it again with seeding on. - **What is seeded.** The seed is installed before any script of the page runs, in every frame of the test's `context`, cross-site frames included. Each function has its own stream, and each document its own. A document is named by its page, by its path (not its host, port or query, which a replay may change), and by how many documents of that path loaded before it: in its frame, for a frame of its parent document's origin; else in its origin in the tab. So a reload draws new numbers, the same ones each run, and two tabs of one test at one path draw different ones: an ID the browser makes for each tab (a tab or client ID, an idempotency key) stays its own, in the record as in the app. Only the test browser changes: not the app or its server. - - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy that could refuse that load (one keploy cannot tell allows a script from the worker's own origin: its source list for scripts (`script-src-elem`, else `script-src`, else `default-src`) lists `'self'`, `'strict-dynamic'`, `*`, the origin's scheme or the origin itself (its scheme, host and port, or a wildcard host that holds it, with a path that holds the script's, or none), and for a module worker so does its list for workers (`worker-src`, else `child-src`, else those); it has no `sandbox`; a classic worker's requires no Trusted Types for scripts; and it upgrades no http request (`upgrade-insecure-requests`), which Chromium never does for a loopback host (`localhost`, `127.0.0.1`), so helmet's default policy allows it there in Chromium, and in Firefox and WebKit on https), the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark; where the script's policy could refuse that load (as above), it is the app's script itself, as a Web Worker's is, with the seeding on its first line, run at the worker's marked URL, in a module Shared Worker after the modules the app's script imports, which the test says (below). Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark, but in a module one whose seeding is on its script's first line (above), where it is its marked URL. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a worker whose seeding is on the app's first line (a policy that could refuse keploy's load, above): there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it (a script the server redirects aside, below); the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker (once, by keploy, for one whose seeding is on its script's first line). Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. + - **Workers.** A Web Worker a page starts (`new Worker(url)`, classic or `type: 'module'`) draws from its own streams too, named by the document that started it, its script's path and how many workers that document started at that path before it, so it draws the same numbers at replay. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded so too, named by the worker that started it, its script's path and how many workers that worker started at that path before it. Chromium's and Firefox's routes reach no script of a worker that a worker starts, so there it starts as the app asked, with no mark, and draws the browser's own numbers. One a seeded worker starts from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark, does too. Each says so (below), once for each address: the worker that started it posts its address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. A worker that a worker named below starts draws the browser's own numbers too, and that worker's line says so. A Shared Worker, which a test's pages share, is named by its script's path, its name and how many times the test started it. keploy reaches a worker's script through the test's browser context: the page's `Worker` and `SharedWorker` add a `__keploy_worker` mark to the script URL's query, and a route of the context answers that request, asking the app's server for the script with no mark. A Web Worker's answer is the app's script itself, the bytes its server sent, never read and written again (so a script served in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding ahead of it on its first line, loaded from a URL keploy marks and answers itself (`importScripts` for a classic worker; an `import` ahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), and a `//# sourceURL` of the app's URL at its end unless the app names one: the worker loads the app's script once. When the script's response carries a Content-Security-Policy that could refuse that load (one keploy cannot tell allows a script from the worker's own origin: its source list for scripts (`script-src-elem`, else `script-src`, else `default-src`) lists `'self'`, `'strict-dynamic'`, `*`, the origin's scheme or the origin itself (its scheme, host and port, or a wildcard host that holds it, with a path that holds the script's, or none), and for a module worker so does its list for workers (`worker-src`, else `child-src`, else those); it has no `sandbox`; a classic worker's requires no Trusted Types for scripts; and it upgrades no http request (`upgrade-insecure-requests`), but on a loopback host, which Chromium never upgrades (`localhost`, a name under it, `127.0.0.0/8`, `[::1]`) and where Firefox and WebKit leave a worker's http load of its own origin alone on `localhost` and `127.0.0.1` (measured; their other loopback names count as refusing), so helmet's default policy allows it on those hosts and on https), the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so (below). A classic worker's answer starts with a `"use strict";` of keploy's when the app's script has `'use strict'` in its directive prologue, so the app's code stays strict. A Shared Worker's answer is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark; where the script's policy could refuse that load (as above), it is the app's script itself, as a Web Worker's is, with the seeding on its first line, run at the worker's marked URL, in a module Shared Worker after the modules the app's script imports, which the test says (below). Only the browser's load of a worker's script is answered so: a request the worker's own code makes at its script's URL (`fetch('')`, a fragment alone, an `XMLHttpRequest` of `''`, which resolve against the marked URL) goes on to the app's server with the mark taken off and gets what the server answers. A seeded worker's `fetch` and `Request` take the mark off too: a fetch or a `Request` at the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (the init names none, and a `Request` it is given was made with no referrer and the default policy) sends the script's URL with no mark as its `Referer`. JavaScript cannot set the rest, so the app's server sees the mark in these, the same at record and replay (the mark comes from the test's seed and the worker's place): the `Referer` of a worker's `XMLHttpRequest` and `importScripts`, of a fetch of a `Request` made with a `referrerPolicy` of its own and no referrer, and, in Chromium and WebKit, whose route reaches nothing a Shared Worker asks for after its script, of a Shared Worker's load of its own script (keploy's answer loads it so); in Chromium and WebKit, the URL of a Shared Worker's `XMLHttpRequest` of its script's own URL; and, for a worker that runs at its marked URL with no seeding (one whose script a route of the test's own answers itself, or a Shared Worker it sends on with options of its own, below), the `Referer` of each of its requests and, in Chromium and WebKit, the URL of a Shared Worker's `fetch('')`. So the app's script runs at its own URL: a stack trace through it names the app's URL with no mark (in Firefox, whose stacks go on through the code that queued what runs, a worker's `Error` and `DOMException` stacks leave keploy's frames out), the `filename` of an error event, in the worker and in the page, shows none, and the worker's `location` shows none. A module Web Worker's `import.meta.url` is its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL. The URLs its imports resolve against are the app's, as the mark is in the query, which a relative specifier other than a fragment alone replaces. A Shared Worker's `import.meta.url` shows no mark, but in a module one whose seeding is on its script's first line (above), where it is its marked URL. The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. An error the browser's own call throws in a seeded function or in the worker's `location` (a float array to `getRandomValues`, a `this` that is no `Crypto`) shows none of keploy's frames on its stack either, but in a worker whose seeding is on the app's first line (a policy that could refuse keploy's load, above): there its frames show at the app's URL, on that line. keploy wraps the page's `Worker`, its `terminate` and `SharedWorker` in every test, seeding off too, so their `toString()` shows keploy's code where it would show the browser's native code. keploy's answer has the status and headers the app's server gives the script (a worker's policy comes from them). The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it (a script the server redirects aside, below); the seeding's own load reaches neither. Chromium and WebKit route nothing a Shared Worker loads after its script, so the app's server is asked for a Shared Worker's script twice: by keploy, and by the worker (once, by keploy, for one whose seeding is on its script's first line). Chromium and WebKit let the route reach none of what a Shared Worker loads after its script, so a module Shared Worker imports its seeding as a `data:` module ahead of the app's script; where its script is served with a Content-Security-Policy that refuses `data:` scripts, keploy's answer runs the seeding itself and then imports the app's script, and the connections that came before that script ran reach its `connect` handlers after, in order, as events keploy dispatches (their `isTrusted` is false). The test's end never waits on a Shared Worker's script, whichever route answers it. So a seeded test's requests go through keploy's routes, and while a context has a route (any `context.route`), the browser asks the server for each request rather than answering it from its cache, in every engine. That holds at record and at replay alike, so a replay asks the app's server for what its recording asked: measured on one test's steps (a page, a second page on the same stylesheet, script and image, a reload, the first page again), the server got 20 calls in Chromium 158 and WebKit 27.2 (11 with no route) and 24 in Firefox 159 (12 with no route), the same ones at record and at replay in each engine. keploy's route stays ahead of every route the test adds to its context or its pages (with `route` or `routeFromHAR`, and through `unroute` and `unrouteAll`), so no route of the test's sees the mark: a worker's script load reaches the test's routes at the URL the app asked for (`**/worker.js` matches it). When the test's `unroute` or `unrouteAll` takes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route's `continue`, `fulfill` or `abort` then fails with Playwright's own error (`route.continue: Route is already handled!`, dropped after `unrouteAll({ behavior: 'ignoreErrors' })`) and its `fallback` goes on to the next route. keploy's route stays, so keploy does the same before the `unroute` or `unrouteAll` returns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's. `unrouteAll` takes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with `'wait'` it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. A route that continues it (a catch-all `route.continue()`) or falls back leaves it to keploy, which answers it as above, asking the app's server for the script with what that route asked: the URL, method, headers or post data it gave `continue` or `fallback` (a header the server needs, a stand-in's URL). A Web Worker runs the script that answer got, so a route that answers once (`times: 1`), counts its loads, or picks the worker's script by its destination (`sec-fetch-dest: worker`) sees what it sees with seeding off. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the one the app asked for, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole: it runs unseeded, at its marked URL. One sent on with the browser's own headers, or with no options, is seeded. One that aborts it aborts it; one that answers it itself (`route.fulfill`) answers it so, and that worker runs the test's script unseeded, at its marked URL. A request event is not a route: a `waitForRequest` or `page.on('request')` of the test's own sees a worker's script request with the mark. These say so as the test runs, once each, naming the worker and the next step: a worker started from a `blob:` or `data:` URL, or from a URL the browser refuses with the mark (a Trusted Types policy), in Chromium and Firefox a worker that a worker started, a worker whose script a route of the test's own answered itself, a Shared Worker whose script such a route sent on with options of its own, and in Firefox a Shared Worker, whose script Firefox's route does not reach. A Service Worker draws the browser's own numbers: keploy cannot put its seeding ahead of its script. Each one the browser registers says so in the test that registered it, once for each script address, naming the next step: if the tests do not need it, set `serviceWorkers: 'block'` in the project's `use`, so Playwright registers none and nothing is said. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and says so once for each address, naming the Service Worker and the same next step: a Web Worker of a page a Service Worker controls, and a Shared Worker of a page a Service Worker controls or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the pages of the test's context tell each other, on a `BroadcastChannel` of keploy's, the scope of each registration they find and of each `register` they call (before the browser registers it). Each `register` holds a Web Lock of keploy's (`keploy-sw-register`, which `navigator.locks.query()` shows) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock, so a page that begins while another is registering one is not told. A Shared Worker a page starts before the browser has answered (a script that runs as the page loads may), in a page that began while another page of its origin was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks (a sandboxed one), starts as the app asked, unseeded, and says so with the same next step: with `serviceWorkers: 'block'`, keploy knows at once that no Service Worker can answer, and seeds it. The worker still takes its place at its path, so the next worker there draws what it drew at record. - **How a page is named.** By its place in the test's own order, the same in every run: - a page the test opens (`context.newPage()`, the `page` fixture's among them), by the order in which the test asked for its pages, even when it asks for two at once, in each of its documents: its first one (the `about:blank` it has before its first `goto`, which `page.setContent` writes into) too; - a popup a page opens with `window.open`, at the opener's origin, as the window that the opener's document opened first, second, and so on (two popups opened at one moment keep their places); diff --git a/playwright/src/random.ts b/playwright/src/random.ts index 519f248..19582fa 100644 --- a/playwright/src/random.ts +++ b/playwright/src/random.ts @@ -1834,8 +1834,11 @@ export function cspAllowsData(headers: Record<string, string>): boolean { * engine may read a module's imports by it); for a * classic worker, whose importScripts takes a string, no policy requires * Trusted Types for scripts; and no policy upgrades an http url to https - * (upgrade-insecure-requests), but in Chromium for a loopback host, which - * Chromium never upgrades. A source keploy does not read so (a wildcard + * (upgrade-insecure-requests), but on a loopback host: in Chromium any + * (localhost, a name under it, 127.0.0.0/8, [::1]), which Chromium never + * upgrades; in Firefox and WebKit localhost and 127.0.0.1, where each + * leaves a worker's http load of its own origin alone (measured; the other + * loopback names are not). A source keploy does not read so (a wildcard * host aside: *.example.com), counts as one that refuses: keploy then * puts the seeding in the script itself (workerAnswer inline), which no * policy refuses. @@ -1848,6 +1851,8 @@ export function cspAllowsOwn(headers: Record<string, string>, url: string, class const scheme = u.protocol.slice(0, -1).toLowerCase(); const host = u.hostname.toLowerCase(); const loopback = host === 'localhost' || host.endsWith('.localhost') || /^127\.\d+\.\d+\.\d+$/.test(host) || host === '[::1]'; + // The hosts whose http loads upgrade-insecure-requests leaves alone. + const kept = engine === 'chromium' ? loopback : (engine === 'firefox' || engine === 'webkit') && (host === 'localhost' || host === '127.0.0.1'); const port = u.port !== '' ? u.port : scheme === 'https' ? '443' : scheme === 'http' ? '80' : ''; const source = (x: string): boolean => { const t = x.toLowerCase(); @@ -1875,7 +1880,7 @@ export function cspAllowsOwn(headers: Record<string, string>, url: string, class } if (ds.has('sandbox')) return false; if (classic && (ds.get('require-trusted-types-for') ?? []).some((x) => x.toLowerCase() === "'script'")) return false; - if (ds.has('upgrade-insecure-requests') && scheme === 'http' && !(loopback && engine === 'chromium')) return false; + if (ds.has('upgrade-insecure-requests') && scheme === 'http' && !kept) return false; const first = (...names: string[]): string[] | undefined => names.map((x) => ds.get(x)).find((x) => x !== undefined); for (const list of [first('script-src-elem', 'script-src', 'default-src'), classic ? undefined : first('worker-src', 'child-src', 'script-src', 'default-src')]) { if (list !== undefined && !list.some(source)) return false; diff --git a/playwright/test/random.test.mjs b/playwright/test/random.test.mjs index 89d9881..fe0f0f8 100644 --- a/playwright/test/random.test.mjs +++ b/playwright/test/random.test.mjs @@ -3163,7 +3163,7 @@ test('workerRoutes answers a Web Worker whose script carries a Content-Security- assert.match(lateLine('a test', 'http://h/m.js'), /^keploy: in a test, the module Web Worker at http:\/\/h\/m\.js is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, .* runs after the modules the script imports: .*; if one does, have the policy allow scripts from the worker's own origin \('self' in script-src\) where the tests run, or serve the worker's script with no Content-Security-Policy there/); }); -test('cspAllowsOwn: a policy that allows a script from the worker\'s own origin lets keploy load there; one it cannot tell does (none, another host or port, a nonce, a sandbox, Trusted Types for a classic worker, an upgrade of http but in Chromium on a loopback host) refuses', () => { +test('cspAllowsOwn: a policy that allows a script from the worker\'s own origin lets keploy load there; one it cannot tell does (none, another host or port, a nonce, a sandbox, Trusted Types for a classic worker, an upgrade of http but on a loopback host: any in Chromium, localhost and 127.0.0.1 in Firefox and WebKit) refuses', () => { const csp = (v) => ({ 'content-type': 'text/javascript', 'content-security-policy': v }); const helmet = "default-src 'self';base-uri 'self';font-src 'self' https: data:;form-action 'self';frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests"; assert.equal(cspAllowsOwn({}, 'http://h/w.js', true), true); @@ -3236,9 +3236,22 @@ test('cspAllowsOwn: a policy that allows a script from the worker\'s own origin [helmet, 'http://a.localhost:3000/w.js', false, 'chromium', true], [helmet, 'http://127.9.0.1/w.js', true, 'chromium', true], [helmet, 'http://[::1]:3000/w.js', false, 'chromium', true], - [helmet, 'http://127.0.0.1:3000/w.js', false, 'firefox', false], - [helmet, 'http://127.0.0.1:3000/w.js', false, 'webkit', false], + [helmet, 'http://127.0.0.1:3000/w.js', false, 'firefox', true], + [helmet, 'http://localhost:3000/w.js', true, 'firefox', true], + [helmet, 'http://LocalHost/w.js', false, 'firefox', true], + [helmet, 'http://127.0.0.1:3000/w.js', false, 'webkit', true], + [helmet, 'http://localhost:3000/w.js', true, 'webkit', true], + [helmet, 'http://a.localhost:3000/w.js', false, 'firefox', false], + [helmet, 'http://127.9.0.1/w.js', false, 'firefox', false], + [helmet, 'http://[::1]:3000/w.js', false, 'firefox', false], + [helmet, 'http://a.localhost:3000/w.js', false, 'webkit', false], + [helmet, 'http://127.0.0.2/w.js', true, 'webkit', false], + [helmet, 'http://[::1]:3000/w.js', false, 'webkit', false], + [helmet, 'http://localhost.test/w.js', false, 'webkit', false], + [helmet, 'http://app.test:3000/w.js', false, 'firefox', false], [helmet, 'http://127.0.0.1:3000/w.js', false, '', false], + [helmet, 'http://localhost:3000/w.js', false, '', false], + [helmet, 'http://localhost:3000/w.js', false, 'msedge', false], [helmet, 'http://app.test:3000/w.js', false, 'chromium', false], [helmet, 'http://128.0.0.1/w.js', false, 'chromium', false], [helmet, 'https://app.test/w.js', false, 'webkit', true], @@ -3263,18 +3276,21 @@ test('workerRoutes loads a Web Worker\'s seeding at its script\'s URL under a po assert.equal((await r.ask(markWorker('http://h/w.js', 'w', t), answerOf(200, 'b()', tt)))[1][1].body, `${oneLine(workerScript(t, 2, 'w'))}b()\n//# sourceURL=http://h/w.js\n`); assert.equal((await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'b()', tt)))[1][1].body, `import ${JSON.stringify(markWorker('http://h/m.js', 'p', t, 3))};b()\n//# sourceURL=http://h/m.js\n`); assert.deepEqual(late, []); - // helmet's policy upgrades http: in Chromium (the routes' engine here) not on a loopback host. + // helmet's policy upgrades http: in Chromium (the routes' engine here) not on any loopback host. const helmet = { 'content-type': 'text/javascript', 'content-security-policy': "default-src 'self';script-src 'self';upgrade-insecure-requests" }; assert.equal((await r.ask(markWorker('http://127.0.0.1:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `import ${JSON.stringify(markWorker('http://127.0.0.1:9/m.js', 'p', t, 4))};b()\n//# sourceURL=http://127.0.0.1:9/m.js\n`); assert.equal((await r.ask(markWorker('http://h/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `${oneLine(workerScript(t, 5, 'm'))}b()\n//# sourceURL=http://h/m.js\n`); assert.deepEqual(late, [['http://h/m.js', false]]); - // In Firefox and WebKit, a loopback host too. + // In Firefox and WebKit, not on localhost or 127.0.0.1 either (measured), + // but on another loopback name. for (const engine of ['firefox', 'webkit']) { const e = routed(); const said = []; await workerRoutes(e.context, { late: (...a) => said.push(a) }, engine); - assert.equal((await e.r.ask(markWorker('http://127.0.0.1:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `${oneLine(workerScript(t, 0, 'm', engine === 'webkit'))}b()\n//# sourceURL=http://127.0.0.1:9/m.js\n`, engine); - assert.deepEqual(said, [['http://127.0.0.1:9/m.js', false]], engine); + assert.equal((await e.r.ask(markWorker('http://127.0.0.1:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `import ${JSON.stringify(markWorker('http://127.0.0.1:9/m.js', 'p', t, 0))};b()\n//# sourceURL=http://127.0.0.1:9/m.js\n`, engine); + assert.equal((await e.r.ask(markWorker('http://localhost:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `import ${JSON.stringify(markWorker('http://localhost:9/m.js', 'p', t, 1))};b()\n//# sourceURL=http://localhost:9/m.js\n`, engine); + assert.equal((await e.r.ask(markWorker('http://127.0.0.2:9/m.js', 'm', t), answerOf(200, 'b()', helmet)))[1][1].body, `${oneLine(workerScript(t, 2, 'm', engine === 'webkit'))}b()\n//# sourceURL=http://127.0.0.2:9/m.js\n`, engine); + assert.deepEqual(said, [['http://127.0.0.2:9/m.js', false]], engine); } }); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 573de03..8fd1c97 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -1877,7 +1877,7 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`served, in ${engine}: a worker's script whose Content-Security-Policy allows a script from the worker's own origin keeps its seeding ahead of the modules it imports (default-src 'self'; a nonce and 'strict-dynamic', module and classic; helmet's default${engine === 'chromium' ? '' : ', which upgrades http, as one keploy cannot tell allows it here: seeded late and named, as it runs off'}), draws the seed's numbers there, its imports' too, and is named as late by no line; a Shared Worker under default-src 'none', classic and module, starts, as off, ${engine === 'firefox' ? 'unseeded (Firefox routes no Shared Worker\'s script)' : 'seeded, the module one named as seeded late once'}; the server never sees keploy's mark`, async () => { + test(`served, in ${engine}: a worker's script whose Content-Security-Policy allows a script from the worker's own origin keeps its seeding ahead of the modules it imports (default-src 'self'; a nonce and 'strict-dynamic', module and classic; helmet's default, which upgrades http but on this loopback host), draws the seed's numbers there, its imports' too, and is named as late by no line; a Shared Worker under default-src 'none', classic and module, starts, as off, ${engine === 'firefox' ? 'unseeded (Firefox routes no Shared Worker\'s script)' : 'seeded, the module one named as seeded late once'}; the server never sees keploy's mark`, async () => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: 'ce'.repeat(16) } } : undefined)); try { @@ -1889,28 +1889,28 @@ for (const engine of ENGINES) { const names = ['s', 'h', 'sd', 'sdc', 'sn', 'snm']; const kinds = (r) => r.drew.own.map((w) => (typeof w === 'string' ? w : w.k)); for (const [mode, r] of [['seeded', s1], ['seeded again', s2], ['off', o1], ['off again', o2]]) { - // Each starts as it does off; all but helmet's in every engine. + // Each starts as it does off, in every engine. assert.deepEqual(kinds(r), kinds(o1), mode); - assert.deepEqual(kinds(r).filter((k, i) => i !== 1), names.filter((k, i) => i !== 1), mode); + assert.deepEqual(kinds(r), names, mode); assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); // Each script loaded once by the browser, as off (a Shared Worker's twice seeded in Chromium and WebKit: keploy asks it too). for (const p of ['/mc/s.js', '/mc/sd.js', '/mc/sdc.js']) assert.equal(r.seen.filter((u) => u === p).length, 1, `${mode} ${p}`); } - if (engine === 'chromium') assert.equal(kinds(s1)[1], 'h'); const at = (r, i) => r.drew.own[i]; // Seeded: the seed's draws, the same in each run, an import's too; off, the browser's own. - const seededAt = engine === 'chromium' ? [0, 1, 2, 3, 4, 5] : engine === 'webkit' ? [0, 2, 3, 4, 5] : [0, 2, 3]; + const seededAt = engine === 'firefox' ? [0, 1, 2, 3] : [0, 1, 2, 3, 4, 5]; for (const i of seededAt) { assert.deepEqual(at(s1, i).draws, at(s2, i).draws, names[i]); if (at(s1, i).dep !== undefined) assert.deepEqual(at(s1, i).dep, at(s2, i).dep, `${names[i]} dep`); } assert.notDeepEqual(at(o1, 0).draws, at(o2, 0).draws); assert.notDeepEqual(at(o1, 0).dep, at(o2, 0).dep); + assert.notDeepEqual(at(o1, 1).dep, at(o2, 1).dep); if (engine === 'firefox') for (const i of [4, 5]) assert.notDeepEqual(at(s1, i).draws, at(s2, i).draws, names[i]); // The lines: a module worker seeded after its imports, once each. // served.spec.ts's other tests run too: only this test's addresses (/mc/, /sv/sn.js, /sv/snm.js). const late = (r) => r.lines.filter((l) => /is served with a Content-Security-Policy/.test(l) && /\/mc\/|\/sv\/snm?\.js/.test(l)).map((l) => /the (module (?:Web|Shared) Worker) at http:\/\/127\.0\.0\.1:\d+(\S+) is served/.exec(l)?.slice(1).join(' ')).sort(); - const want = { chromium: ['module Shared Worker /sv/snm.js'], firefox: ['module Web Worker /mc/h.js'], webkit: ['module Shared Worker /sv/snm.js', 'module Web Worker /mc/h.js'] }[engine]; + const want = { chromium: ['module Shared Worker /sv/snm.js'], firefox: [], webkit: ['module Shared Worker /sv/snm.js'] }[engine]; for (const r of [s1, s2]) assert.deepEqual(late(r), want); for (const r of [o1, o2]) assert.deepEqual(late(r), []); } finally { await a.close(); } From b1597272ad6e7d0c20807fc46cfcf5cd2fd4bfb0 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Sun, 15 Nov 2026 23:59:29 +0530 Subject: [PATCH 67/73] fix(playwright): say why a test's end waits on a busy Web Worker Chromium reports a Web Worker's script load finished only once the worker's top-level code returns. A worker whose top-level code runs long (a loop, or long setup as its script loads) so holds the test's end, with seeding off too, until that code returns or the test's timeout names the load. Playwright shows nothing else that says the script has arrived, so settle keeps waiting: it never stops waiting on a script that may still be arriving. The wait now says why. Settle keeps each Web Worker's script URL from Playwright's worker event (Chromium). Once the test's end has waited 3 s with only Web Workers' script loads in flight, and again as each call ends after that, the test prints one line for each such URL, naming the worker, why Chromium waits, and the next step: move the worker's long-running top-level work into its onmessage handler, or end the worker before the test ends. Firefox and WebKit report the load's end as the script arrives, and keploy's route ends a seeded Web Worker's load with its answer, so nothing is said there. Rows: a unit row for the line's timing, its once-per-URL rule, a fetch at a worker's URL and a page script beside the load, the stop, and another engine; a browser row per engine, seeded and off, with a worker whose top-level code runs on for 10 s after the body ends. The README names the case under "What the run says when ...". Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 1 + playwright/src/index.ts | 14 ++++- playwright/src/settle.ts | 63 ++++++++++++++++++- playwright/test/fixtures/settle.spec.ts | 11 ++++ playwright/test/rows.test.mjs | 34 ++++++++++- playwright/test/unit.test.mjs | 81 +++++++++++++++++++++++++ 6 files changed, 200 insertions(+), 4 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index b7bb493..3336893 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -100,6 +100,7 @@ keploy seeds, places and waits for what it can reach in the test browser and the - A replay's lines on how it draws need a keploy that keeps each test's note; a seeded replay that keploy answers with no note says so once per worker (above). - **... a call does not end.** Settle shows a step that names each call still in flight, and a call that never ends fails the test at Playwright's test timeout, named there. Settle drops a call only when the browser cancels it, the document that made it is gone, or a route of the test's has answered it or holds it after its handler threw (above), so it fails toward waiting, never toward a cut call. A call the old document starts after the new document's answer, as it unloads (a `pagehide` or `unload` handler's), is waited for too, since settle cannot tell it from a call of the new document; it is named the same way if the browser never reports its end. A Web Worker a page terminates (`worker.terminate()`) while its script still loads: Chromium reports no end for that load, ever, so the page's `Worker` tells settle which of its document's workers at that URL it terminated (each document numbers its Web Workers at a URL in the order it starts them), and settle stops waiting on that worker's load; Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test, keploy's route fetches a Web Worker's script, and its load ends when keploy's answer has gone to the browser or the browser has dropped it. - **... in Chromium, a page loads a script at the URL of a Web Worker it terminates.** Settle finds a terminated Web Worker's script load by its place among the script requests its document's frame reported at that URL: the k-th Web Worker the document started there is taken to be the k-th such request. A page's own script, or a worker's `importScripts`, at that URL in that document takes a place too, so settle may stop waiting on that script's load while it is in flight (the test can end before it does) and, with seeding off, go on waiting on the terminated worker's load, which Chromium never ends, until Playwright's test timeout fails the test, naming that load as a call that did not end. Playwright shows a Web Worker's script load as a script request of the page's frame, as it shows the page's own, so settle cannot tell them apart. Next step: serve the worker's script at a URL no other script of the page loads. +- **... in Chromium, a Web Worker's top-level code runs long.** Chromium reports a Web Worker's script load finished only once the worker's top-level code returns, and Playwright shows nothing else that says the script has arrived, so while that code runs (a loop, or long setup that runs as the script loads) the test's end waits on the load, with seeding off too: settle never stops waiting on a script that may still be arriving. A load that does not end by the test's timeout fails the test there, named as a call that did not end. Once the test's end has waited 3 s with only Web Workers' script loads in flight, the test says so once for each worker's URL (`keploy: in <test>, the test's end waits for the script of the Web Worker at <url>: Chromium reports a Web Worker's script loaded only once its top-level code returns, ...`), with the next step: move the worker's long-running top-level work into its `onmessage` handler, or end the worker (`terminate()`) before the test ends. Firefox and WebKit report the load's end as the script arrives, and a script load keploy's route answers (a seeded Web Worker's) ends when keploy's answer has gone to the browser, so neither waits on it. - **... a module Web Worker reads its `import.meta.url`.** On a seeded test it is the worker's marked URL, the app's URL with keploy's `__keploy_worker` mark added to its query: keploy answers that one load with its seeding ahead of the app's module, so the module runs at the URL the browser loaded it from. The path is the app's, so a URL the module builds from `import.meta.url` with a relative path (`new URL('./data.json', import.meta.url)`), or with a query of its own, is the app's URL with no mark, and so are the URLs its imports resolve to. Only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark; the mark comes from the test's seed and the worker's place, so it is the same at record and at replay. A module Shared Worker whose seeding is on its script's first line (below) reads its marked URL so too. With seeding off, or for a classic Web Worker (its `location`) or another Shared Worker (its `import.meta.url` too), the worker sees the app's URL. - **... a module worker's script is served with a Content-Security-Policy that could refuse keploy's load.** When keploy cannot tell that the policy allows a script from the worker's own origin (`'self'` in `script-src` or `default-src` does; the rest under Workers, above), keploy puts the worker's seeding on its script's first line, and in a module that line runs after the modules the script imports: a random number one of those modules draws as it loads is the browser's own, so it may differ at replay. This holds for a module Web Worker and, in Chromium and WebKit, a module Shared Worker. The test says so once for each address (`keploy: in <test>, the module Web Worker at <address> is served with a Content-Security-Policy that could refuse keploy's load of a script from the worker's own origin, ...`, or `module Shared Worker`), with the next step: if one of those modules draws as it loads, have the policy allow scripts from the worker's own origin (`'self'` in `script-src`) where the tests run, or serve the worker's script with no Content-Security-Policy there, and keploy loads its seeding ahead of those modules. A classic worker's seeding on its first line runs before any of the app's code. - **... a page starts a Shared Worker.** Settle does not wait for it, and the test says so once for each Shared Worker script (`keploy: in <test>, a page started the Shared Worker at <address>: keploy does not wait for its script or its calls ...`), with the next step: have the test wait for what the worker does before it ends. diff --git a/playwright/src/index.ts b/playwright/src/index.ts index 2490e82..e6cd83b 100644 --- a/playwright/src/index.ts +++ b/playwright/src/index.ts @@ -67,7 +67,7 @@ import { ENV_TEST_PID, decide } from './join'; import type { Joining } from './join'; import { NO_NOTE_LINE, differences, docLog, keepDocDraws, noteText, readNote, unreadNoteLine, type DocLog } from './note'; import { OFF_LINE, SEEDED_LINE, besideOf, fakerLine, fakersOf, freshSeed, initScript, lateLine, othersNow, pageAddress, placePages, processRandom, redirectLine, seatFaker, seatRandom, seedingFromEnv, setAside, tally, unplacedLine, unseededLine, workerRoutes, type Fakers, type Unplaced } from './random'; -import { Inflight, sharedWorkerLine, workersScript } from './settle'; +import { Inflight, busyWorkerLine, sharedWorkerLine, workersScript } from './settle'; export { expect, KeployError }; export { seedFaker } from './random'; @@ -101,6 +101,9 @@ let reports = 0; // How many contexts of a keploy test this worker opened: each one's Shared // Worker report function has its own name. let sharedReports = 0; +// How long a test's end waits on Web Workers' script loads alone before it +// says why (busyWorkerLine). +const BUSY_WORKER_MS = 3000; // How a test's browser draws its random numbers. type PageSeeding = { @@ -319,7 +322,14 @@ const keployTestType = base.extend<TestFixtures, WorkerFixtures>({ if (inflight.size > 0) { const calls = inflight.describe(); const shown = calls.slice(0, 5).join(', ') + (calls.length > 5 ? `, and ${calls.length - 5} more` : ''); - await base.step(`keploy: wait for the test's ${calls.length} call(s) in flight before its browser context closes: ${shown}`, () => inflight.settled()); + // Chromium ends a Web Worker's script load only when its top-level + // code returns: a wait on such loads alone says so, once for each. + const stop = inflight.busyWorkers(BUSY_WORKER_MS, (u) => { process.stdout.write(`${busyWorkerLine(name, pageAddress(u))}\n`); }); + try { + await base.step(`keploy: wait for the test's ${calls.length} call(s) in flight before its browser context closes: ${shown}`, () => inflight.settled()); + } finally { + stop(); + } } }, diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 6f23141..7be9de2 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -24,6 +24,11 @@ export class Inflight { // Each document's number (webWorkerDocument), the document it names. private readonly numbered = new Map<number, Doc>(); private documents = 0; + // Chromium (stops): the script URLs, with no fragment, of the Web Workers + // the test's pages started (Playwright's worker event), for busyWorkers. + private readonly workerUrls = new Set<string>(); + // While busyWorkers watches: its check, run again as each call ends. + private busy: (() => void) | null = null; /** * stops: the browser reports no end for the script load of a Web Worker @@ -98,6 +103,7 @@ export class Inflight { this.drop((o) => o.frame === f); }); onRoute(p, (r, threw) => this.routed(r, threw)); + if (this.stops) p.on('worker', (w) => this.workerUrls.add(noFragment(w.url()))); }; // Nor for a request a route answers itself with a redirect (Chromium, // for a Web Worker's script: its worker never starts, with no keploy @@ -129,6 +135,47 @@ export class Inflight { return new Promise((resolve) => this.waiters.push(resolve)); } + /** + * Chromium (stops) reports a Web Worker's script load finished only once + * the worker's top-level code returns, so settle waits as long as that + * code runs. Once ms have passed from this call, and again as each call + * ends after, when every call still in flight is the script load of a Web + * Worker (a script request at the URL, fragment aside, of a Web Worker one + * of the test's pages started), say gets each such URL, once for each. + * Returns the function that stops it. Other engines report the load's end + * as the script arrives, and nothing is said there. + */ + busyWorkers(ms: number, say: (url: string) => void): () => void { + const told = new Set<string>(); + const check = (): void => { + for (const u of this.workerLoads()) { + if (told.has(u)) continue; + told.add(u); + say(u); + } + }; + const t = setTimeout(() => { + this.busy = check; + check(); + }, ms); + return () => { + clearTimeout(t); + this.busy = null; + }; + } + + // The URLs (no fragment) of the calls in flight when every one is a Web + // Worker's script load (busyWorkers); none otherwise. + private workerLoads(): string[] { + const urls = new Set<string>(); + for (const r of this.open.keys()) { + const u = noFragment(r.url()); + if (r.resourceType() !== 'script' || !this.workerUrls.has(u)) return []; + urls.add(u); + } + return [...urls]; + } + /** * A call keploy's own route answered whole (a worker's script): nothing of * it is in flight, whatever the browser reports after. Chromium reports @@ -251,7 +298,10 @@ export class Inflight { } private wake(): void { - if (this.open.size !== 0) return; + if (this.open.size !== 0) { + this.busy?.(); + return; + } const w = this.waiters; this.waiters = []; w.forEach((f) => f()); @@ -422,6 +472,17 @@ export function workersScript(report: string): string { `; } +/** + * The line a test shows, once for each Web Worker script URL (at), when its + * end has waited busyWorkers' time with only Web Workers' script loads in + * flight (Chromium). + */ +export function busyWorkerLine(name: string, at: string): string { + return `keploy: in ${name}, the test's end waits for the script of the Web Worker at ${at}: Chromium reports a Web Worker's script loaded only once its top-level code returns, ` + + 'so keploy waits until that code returns (or the test times out, naming the load); ' + + "move the worker's long-running top-level work into its onmessage handler, or end the worker (terminate()) before the test ends."; +} + /** * The line a test shows, once for each Shared Worker script address (at, * origin and path), when one of its pages started a Shared Worker. diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index 97d6521..b112eab 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -286,6 +286,17 @@ test('two Web Workers at one URL, the second terminated while their scripts load await reached(); await page.evaluate(() => { (globalThis as unknown as { w: Worker[] }).w[1].terminate(); }); }); +// A Web Worker whose top-level code runs on (10 s) after its first message, +// which the body waits for (its URL's fragment is no part of the request). +test('a Web Worker whose top-level code runs long', async ({ page }) => { + await page.goto(`${base}/page`); + const said = await page.evaluate(() => new Promise<string>((resolve) => { + const g = globalThis as unknown as { w: Worker }; + g.w = new Worker('/busyw.js#f'); + g.w.onmessage = (e) => resolve(String(e.data)); + })); + if (said !== 'busy') throw new Error(`the Web Worker said ${said}`); +}); // The test's own routes answer a classic Web Worker's script with a 302, a // module one's with a 307 and a page fetch with a 302 (each to a URL the // page server does not know). Chromium reports no end for such a worker diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 8fd1c97..9d96090 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -11,7 +11,7 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, unplacedLine, unseededLine, markWorker, workerMark, workerScript } from '../dist/random.js'; -import { sharedWorkerLine } from '../dist/settle.js'; +import { busyWorkerLine, sharedWorkerLine } from '../dist/settle.js'; import { NO_NOTE_LINE, readNote, unreadNoteLine } from '../dist/note.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; @@ -70,6 +70,9 @@ async function pageServer(j) { // A Shared Worker that answers each page that connects. if (req.url === '/shared.js?v=1') { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end("onconnect = (e) => { e.ports[0].postMessage('up'); };"); return; } if (req.url === '/nocontent') { res.writeHead(204); res.end(); return; } + // A Web Worker whose top-level code runs on for 10 s after its first + // message. + if (req.url === '/busyw.js') { res.writeHead(200, { 'content-type': 'text/javascript' }); res.end("postMessage('busy'); const end = Date.now() + 10000; while (Date.now() < end) { /* busy */ }"); return; } // A Web Worker's script, classic or module, answered only once /release // is called. if (req.url === '/heldw.js' || req.url === '/heldw.mjs') { @@ -497,6 +500,35 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { } } +// A Web Worker whose top-level code runs on (10 s) after its first message, +// which the test's body waits for: Chromium reports its script loaded only +// once that code returns, so with seeding off settle waits on that load and, +// once its wait has run 3 s with only that load in flight, the test says why, +// once, naming the worker. Firefox and WebKit report the load's end as the +// script arrives, and seeded, keploy's route answers it whole: no wait on it, +// and nothing said. The wait ends by the worker's own code (no wall-clock +// bound in the row; the config's 30 s timeout would fail it). +for (const engine of ['chromium', 'firefox', 'webkit']) { + for (const seed of ['seeded', 'off']) { + const says = engine === 'chromium' && seed === 'off'; + test(`settle, in ${engine}, ${seed}: a test whose Web Worker's top-level code runs long after the body ends ${says ? 'waits for its script load and says once why, naming the worker and the next step' : 'does not wait for its script load, and says nothing of it'}`, async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = 'a Web Worker whose top-level code runs long'; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), ...(seed === 'off' ? { KEPLOY_SEED_RANDOM: 'off' } : {}), SLOW_BASE: p.url, PW_BROWSER: engine }); + const t = tests[title]; + assert.equal(t.status, 'passed', out); + const waited = t.steps.filter((x) => x.startsWith('keploy: wait for') && /\/busyw\.js/.test(x)); + assert.equal(waited.length, says ? 1 : 0, t.steps.join('\n')); + const said = t.stdout.split('\n').filter((l) => l.startsWith(`keploy: in ${title}, the test's end waits`)).map((l) => l.replace(/:\d+\//, ':<port>/')); + assert.deepEqual(said, says ? [busyWorkerLine(title, 'http://127.0.0.1:<port>/busyw.js')] : [], out); + } finally { await p.close(); await a.close(); } + }); + } +} + // A request the test's own route answers with a redirect (route.fulfill // 302 or 307): Chromium reports no end for a Web Worker's script load so // answered, Firefox none for the redirect's first request (a page fetch's diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index dab50e1..930155f 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -680,6 +680,87 @@ test('settle (Chromium): a commit with no answer (a route change inside the page assert.equal(inflight.size, 3); }); +test('settle (Chromium): busyWorkers names each Web Worker script URL (fragment aside) once, when its time has passed with only Web Workers\' script loads in flight, or as a call ends after; not before its time, not beside a fetch at a worker\'s URL or a script at a URL no worker of the test has, not after stop, and not in another engine', async (t) => { + t.mock.timers.enable({ apis: ['setTimeout'] }); + const setup = (stops) => { + const ctx = fakeContext(); + const inflight = new Inflight(ctx, stops); + const page = new EventEmitter(); + ctx.emit('page', page); + const f = { page: () => page }; + const load = (url, type = 'script') => ({ frame: () => f, isNavigationRequest: () => false, method: () => 'GET', url: () => url, resourceType: () => type }); + return { ctx, inflight, page, load }; + }; + // Only a Web Worker's load in flight: said once its time has passed. + { + const { ctx, inflight, page, load } = setup(true); + page.emit('worker', { url: () => 'http://h/w.js#a' }); + ctx.emit('request', load('http://h/w.js#b')); + const said = []; + const stop = inflight.busyWorkers(3000, (u) => said.push(u)); + t.mock.timers.tick(2999); + assert.deepEqual(said, []); + t.mock.timers.tick(1); + assert.deepEqual(said, ['http://h/w.js']); + stop(); + } + // Beside a page script at a URL no worker has, and then beside a fetch at + // the worker's URL: nothing; said as the last of them ends. + { + const { ctx, inflight, page, load } = setup(true); + page.emit('worker', { url: () => 'http://h/busy.js' }); + const busy = load('http://h/busy.js'); + const own = load('http://h/own.js'); + const fetched = load('http://h/busy.js', 'fetch'); + for (const r of [busy, own]) ctx.emit('request', r); + const said = []; + const stop = inflight.busyWorkers(3000, (u) => said.push(u)); + t.mock.timers.tick(3000); + assert.deepEqual(said, []); + ctx.emit('request', fetched); + ctx.emit('requestfinished', own); + assert.deepEqual(said, []); + ctx.emit('requestfinished', fetched); + assert.deepEqual(said, ['http://h/busy.js']); + // A second load at that URL and another worker's: the new URL alone is + // said, once, as a call ends; a request that starts says nothing. + page.emit('worker', { url: () => 'http://h/two.js' }); + const again = load('http://h/busy.js'); + ctx.emit('request', again); + ctx.emit('request', load('http://h/two.js')); + assert.deepEqual(said, ['http://h/busy.js']); + ctx.emit('requestfailed', again); + assert.deepEqual(said, ['http://h/busy.js', 'http://h/two.js']); + // After stop, nothing more. + stop(); + page.emit('worker', { url: () => 'http://h/three.js' }); + ctx.emit('request', load('http://h/three.js')); + ctx.emit('requestfinished', busy); + assert.deepEqual(said, ['http://h/busy.js', 'http://h/two.js']); + } + // Stopped before its time: nothing. + { + const { ctx, inflight, page, load } = setup(true); + page.emit('worker', { url: () => 'http://h/w.js' }); + ctx.emit('request', load('http://h/w.js')); + const said = []; + inflight.busyWorkers(3000, (u) => said.push(u))(); + t.mock.timers.tick(3000); + assert.deepEqual(said, []); + } + // Another engine reports the load's end as the script arrives: nothing. + { + const { ctx, inflight, page, load } = setup(false); + page.emit('worker', { url: () => 'http://h/w.js' }); + ctx.emit('request', load('http://h/w.js')); + const said = []; + const stop = inflight.busyWorkers(3000, (u) => said.push(u)); + t.mock.timers.tick(3000); + assert.deepEqual(said, []); + stop(); + } +}); + test('settle: a cancel that comes while settle already waits wakes it (its page closes; then, for a service worker\'s call, the test closes its context)', async () => { for (const worker of [false, true]) { const c = worker ? 'a page\'s call and a service worker\'s' : 'a page\'s call'; From edfdb414e0af65f12add9b84e45b2a68053643be Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Mon, 16 Nov 2026 00:29:51 +0530 Subject: [PATCH 68/73] fix(playwright): keep Playwright's own error from a watched fulfill Settle's route watch called each route's fulfill through Function.prototype.call. Playwright names an API call by how its caller invoked it, so a fulfill that failed read "route.call: Route is already handled!" where Playwright's own error, with no keploy, reads "route.fulfill: ...". A test's route that took itself off and then answered a worker's script load failed with that wrong text, seeded and off. Settle now calls the bound method, as keploy's own route wrappers do, and the error is Playwright's own again. The settle rows for a route that answers with a redirect asserted that settle's step names none of those requests, while the fixture's body could end before the routes' handlers had: the step then named calls that ended a moment later. The body now ends once each handler has ended (its fulfill returned, or threw as WebKit refuses a redirect), and the rows match only the first requests, not a redirect's next hop. The "sends on, then fulfill" row asserts Playwright's whole error text. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/src/settle.ts | 7 +++++-- playwright/test/fixtures/settle.spec.ts | 21 ++++++++++++++++++--- playwright/test/rows.test.mjs | 4 ++-- 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 7be9de2..6af5281 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -278,9 +278,12 @@ export class Inflight { if ((route as unknown as { _handlingPromise?: unknown })._handlingPromise) this.answered(r); return; } - const fulfill = route.fulfill; + // Bound, not called through Function.prototype.call: Playwright names + // an API call by how its caller invoked it, and its error must read as + // Playwright's own (route.fulfill: ...), as with no keploy. + const fulfill = route.fulfill.bind(route); route.fulfill = async (o) => { - await fulfill.call(route, o); + await fulfill(o); this.answered(r); }; } diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index b112eab..e81caa9 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -305,9 +305,23 @@ test('a Web Worker whose top-level code runs long', async ({ page }) => { // test there, and the request stays with the route. test("requests the test's route answers with a redirect", async ({ page }) => { await page.goto(`${base}/page`); - await page.route('**/rw.js', (r) => r.fulfill({ status: 302, headers: { location: `${base}/v2/rw.js` } })); - await page.route('**/rwm.js', (r) => r.fulfill({ status: 307, headers: { location: `${base}/v2/rwm.js` } })); - await page.route('**/rf', (r) => r.fulfill({ status: 302, headers: { location: `${base}/v2/rf` } })); + // Each route's handler says when it has ended (its fulfill returned, or + // threw), and the body ends only then. + const ended: Promise<void>[] = []; + const redirect = (glob: string, status: number, to: string): Promise<void> => { + let end = (): void => {}; + ended.push(new Promise<void>((resolve) => { end = resolve; })); + return page.route(glob, async (r) => { + try { + await r.fulfill({ status, headers: { location: to } }); + } finally { + end(); + } + }); + }; + await redirect('**/rw.js', 302, `${base}/v2/rw.js`); + await redirect('**/rwm.js', 307, `${base}/v2/rwm.js`); + await redirect('**/rf', 302, `${base}/v2/rf`); const sent = Promise.all([/\/rw\.js(\?|$)/, /\/rwm\.js(\?|$)/, /\/rf$/].map((u) => page.waitForRequest(u))); await page.evaluate(() => { const g = globalThis as unknown as { w: Worker[] }; @@ -315,6 +329,7 @@ test("requests the test's route answers with a redirect", async ({ page }) => { void fetch('/rf').catch(() => {}); }); await sent; + await Promise.all(ended); }); // A route whose answer the browser refused (WebKit refuses a redirect) // sends the call on; the server has it before the body ends. diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 9d96090..17c3b23 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -552,7 +552,7 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { } else { assert.equal(t.status, 'passed', out); } - const waited = t.steps.filter((x) => x.startsWith('keploy: wait for') && /\/(rw\.js|rwm\.js|rf)(\?|,|\s|$)/.test(x)); + const waited = t.steps.filter((x) => x.startsWith('keploy: wait for') && /:\d+\/(rw\.js|rwm\.js|rf)(\?|,|\s|$)/.test(x)); assert.deepEqual(waited, [], t.steps.join('\n')); } finally { await p.close(); await a.close(); } }); @@ -567,7 +567,7 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { // (waitedFor). for (const engine of ['chromium', 'firefox', 'webkit']) { test(`settle, in ${engine}: a call the test's own route sends on is waited for, though the route's fulfill failed (after it was sent on, its handler throwing${engine === 'webkit' ? '; or refused before, as WebKit refuses a redirect' : ''})`, async () => { - await waitedFor("a call the test's route sends on and then tries to answer", 'GET', '/slow', engine, /Route is already handled/); + await waitedFor("a call the test's route sends on and then tries to answer", 'GET', '/slow', engine, /route\.fulfill: Route is already handled!/); if (engine === 'webkit') await waitedFor("a call the test's route sends on after the browser refused its answer", 'GET', '/slow', engine); }); } From 558f5670e3b04c3e0dbff03c55aec6cb88676a07 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Mon, 16 Nov 2026 00:58:45 +0530 Subject: [PATCH 69/73] test(playwright): assert Playwright's own text for a refused fulfill The settle rows for a call the test's route sends on and then tries to answer asserted "route.fulfill: Route is already handled!". Playwright's own check refuses that fulfill before it makes any API call, so its error names no API, with keploy or without it: the rows now assert the first line Playwright gives, "Error: Route is already handled!". The rows for a route that takes itself off are the ones that see how a fulfill is called (their error comes back through Playwright's API). Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/test/rows.test.mjs | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 17c3b23..291fb76 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -562,12 +562,13 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { // A request the test's route sends on (continue) is waited for, though a // fulfill of the route failed: in every engine, one the route had sent on // before, whose handler then throws Playwright's "already handled" (which -// fails the test); in WebKit, one whose redirect the browser refused before -// the route sent it on. The server answers it once settle's step has begun -// (waitedFor). +// fails the test; Playwright's own check refuses that fulfill before it +// makes any API call, so its error names no API, as with no keploy); in +// WebKit, one whose redirect the browser refused before the route sent it +// on. The server answers it once settle's step has begun (waitedFor). for (const engine of ['chromium', 'firefox', 'webkit']) { test(`settle, in ${engine}: a call the test's own route sends on is waited for, though the route's fulfill failed (after it was sent on, its handler throwing${engine === 'webkit' ? '; or refused before, as WebKit refuses a redirect' : ''})`, async () => { - await waitedFor("a call the test's route sends on and then tries to answer", 'GET', '/slow', engine, /route\.fulfill: Route is already handled!/); + await waitedFor("a call the test's route sends on and then tries to answer", 'GET', '/slow', engine, /^Error: Route is already handled!\n/); if (engine === 'webkit') await waitedFor("a call the test's route sends on after the browser refused its answer", 'GET', '/slow', engine); }); } From 7b860397b0dd37cd492c21f035bdd2532e9db1cb Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Mon, 16 Nov 2026 01:42:07 +0530 Subject: [PATCH 70/73] test(playwright): end a refused redirect's route handlers together Playwright ends a test at the first error a route's handler throws (the handler's rejection is unhandled, and the test runner stops the test), while the test's other handlers may still wait on their fulfill. Their requests are then in flight, held by a running handler, and settle's step named them for the moment until WebKit refused their answers too. The rows for a test whose routes answer with a redirect asserted that no step names them, and in WebKit they failed in about half the runs. The fixture's handlers now each wait until every route's first fulfill has returned or thrown, and only then throw what their fulfill threw, so in WebKit all three end together. In WebKit no route can reach a request whose redirect the browser refused: Playwright marks the route handled before WebKit refuses, so a continue() of it fails with Playwright's own "route.continue: Route is already handled!". The WebKit row that expected such a call sent on and waited for now asserts that error and that settle does not wait on it. WebKit holds the call until the test's browser context is deleted and may then send it to the server (Playwright sends nothing for it, with or without keploy), so the row asserts nothing of the server. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/test/fixtures/settle.spec.ts | 45 ++++++++++++++++++------- playwright/test/rows.test.mjs | 39 ++++++++++++++++----- 2 files changed, 63 insertions(+), 21 deletions(-) diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index e81caa9..234f108 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -305,23 +305,38 @@ test('a Web Worker whose top-level code runs long', async ({ page }) => { // test there, and the request stays with the route. test("requests the test's route answers with a redirect", async ({ page }) => { await page.goto(`${base}/page`); - // Each route's handler says when it has ended (its fulfill returned, or - // threw), and the body ends only then. - const ended: Promise<void>[] = []; + // Playwright ends a test at the first error a route's handler throws, + // while the other handlers may still wait on their fulfill: their + // requests are then in flight, and settle's step names them. So each + // handler, once its fulfill has returned or thrown, waits until every + // route's first fulfill has, and then throws what its fulfill threw: in + // WebKit the three handlers throw together. The body ends once each + // route's first handler has returned (never, in WebKit). + const fulfilled: Promise<void>[] = []; + const returned: Promise<void>[] = []; + let together: Promise<unknown> = Promise.resolve(); const redirect = (glob: string, status: number, to: string): Promise<void> => { + let done = (): void => {}; let end = (): void => {}; - ended.push(new Promise<void>((resolve) => { end = resolve; })); + fulfilled.push(new Promise<void>((resolve) => { done = resolve; })); + returned.push(new Promise<void>((resolve) => { end = resolve; })); return page.route(glob, async (r) => { + let refused: { error: unknown } | null = null; try { await r.fulfill({ status, headers: { location: to } }); - } finally { - end(); + } catch (error) { + refused = { error }; } + done(); + await together; + if (refused !== null) throw refused.error; + end(); }); }; await redirect('**/rw.js', 302, `${base}/v2/rw.js`); await redirect('**/rwm.js', 307, `${base}/v2/rwm.js`); await redirect('**/rf', 302, `${base}/v2/rf`); + together = Promise.all(fulfilled); const sent = Promise.all([/\/rw\.js(\?|$)/, /\/rwm\.js(\?|$)/, /\/rf$/].map((u) => page.waitForRequest(u))); await page.evaluate(() => { const g = globalThis as unknown as { w: Worker[] }; @@ -329,20 +344,26 @@ test("requests the test's route answers with a redirect", async ({ page }) => { void fetch('/rf').catch(() => {}); }); await sent; - await Promise.all(ended); + await Promise.all(returned); }); -// A route whose answer the browser refused (WebKit refuses a redirect) -// sends the call on; the server has it before the body ends. -test("a call the test's route sends on after the browser refused its answer", async ({ page }) => { +// WebKit refuses a redirect, and no route can then reach the request: +// Playwright's continue() of it fails with Playwright's own error, which +// the handler throws and which fails the test. The body ends once the +// handler has returned (never, in WebKit). WebKit holds the request until +// the context is deleted, and may then send it to the server. +test("a call the test's route tries to send on after the browser refused its answer", async ({ page }) => { await page.goto(`${base}/page`); + let end = (): void => {}; + const returned = new Promise<void>((resolve) => { end = resolve; }); await page.route('**/slow', async (r) => { await r.fulfill({ status: 302, headers: { location: `${base}/page` } }).catch(() => {}); await r.continue(); + end(); }); const sent = page.waitForRequest(/\/slow$/); - await page.evaluate(() => { void fetch('/slow'); }); + await page.evaluate(() => { void fetch('/slow').catch(() => {}); }); await sent; - await reached(); + await returned; }); // A route sends the call on, then tries to answer it too: Playwright's // fulfill throws (the route is already handled), and so does the handler, diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 291fb76..cb0c148 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -535,7 +535,9 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { // too), and WebKit refuses the answer, with Playwright's own error, and the // request stays with the route. Settle waits on none of them, seeded or off: // a wait shows as settle's step naming the request (and a timeout at the -// config's 30 s: the row sets no short timeout). +// config's 30 s: the row sets no short timeout). The fixture's handlers end +// together: Playwright ends a test at a handler's first throw, and a +// handler still waiting on its fulfill then holds its request in flight. for (const engine of ['chromium', 'firefox', 'webkit']) { for (const seed of ['seeded', 'off']) { test(`settle, in ${engine}, ${seed}: a request the test's own route answers with a redirect (a classic Web Worker's script with a 302, a module one's with a 307, a page fetch with a 302) is not waited for, and the test ${engine === 'webkit' ? "fails with Playwright's own error (WebKit refuses the answer)" : 'passes'}`, async () => { @@ -560,19 +562,38 @@ for (const engine of ['chromium', 'firefox', 'webkit']) { } // A request the test's route sends on (continue) is waited for, though a -// fulfill of the route failed: in every engine, one the route had sent on -// before, whose handler then throws Playwright's "already handled" (which -// fails the test; Playwright's own check refuses that fulfill before it -// makes any API call, so its error names no API, as with no keploy); in -// WebKit, one whose redirect the browser refused before the route sent it -// on. The server answers it once settle's step has begun (waitedFor). +// fulfill of the route failed after: its handler then throws Playwright's +// "already handled" (which fails the test; Playwright's own check refuses +// that fulfill before it makes any API call, so its error names no API, as +// with no keploy). The server answers it once settle's step has begun +// (waitedFor). for (const engine of ['chromium', 'firefox', 'webkit']) { - test(`settle, in ${engine}: a call the test's own route sends on is waited for, though the route's fulfill failed (after it was sent on, its handler throwing${engine === 'webkit' ? '; or refused before, as WebKit refuses a redirect' : ''})`, async () => { + test(`settle, in ${engine}: a call the test's own route sends on is waited for, though the route's fulfill failed after it was sent on, its handler throwing`, async () => { await waitedFor("a call the test's route sends on and then tries to answer", 'GET', '/slow', engine, /^Error: Route is already handled!\n/); - if (engine === 'webkit') await waitedFor("a call the test's route sends on after the browser refused its answer", 'GET', '/slow', engine); }); } +// WebKit refuses a route's redirect (route.fulfill 302), and Playwright then +// lets no route reach the request: a continue() of it fails with +// Playwright's own error (route.continue: Route is already handled!), which +// the handler throws and which fails the test. Settle does not wait on the +// call. WebKit holds it until the test's browser context is deleted, and +// then may send it to the server (Playwright sends nothing for it, with or +// without keploy), so the row asserts nothing of the server. +test("settle, in webkit: a call the test's own route tries to send on after WebKit refused its redirect fails with Playwright's own error, and is not waited for", async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = "a call the test's route tries to send on after the browser refused its answer"; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, PW_BROWSER: 'webkit' }); + const t = tests[title]; + assert.equal(t.status, 'failed', out); + assert.match(t.error, /^Error: route\.continue: Route is already handled!\n/, out); + assert.deepEqual(t.steps.filter((x) => x.startsWith('keploy: wait for') && /\/slow(,|\s|$)/.test(x)), [], t.steps.join('\n')); + } finally { await p.close(); await a.close(); } +}); + // Settle rows where a call is still waited for: the page server answers it // only once settle's step has begun (the rows' reporter asks /release then), // so the test ends by settle's own wait, with no deadline on its body. The From eca89f8aad40875e8d7a388c0694737ac879fff4 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Mon, 16 Nov 2026 04:22:54 +0530 Subject: [PATCH 71/73] test(playwright): take WebKit's dropped Shared Worker only on its record WebKit sometimes never runs a Shared Worker whose script the server redirects: the server answers the script's URL with the redirect and the redirect's URL with the script, and then WebKit asks for nothing more, with no error event and no message. Plain Playwright with no keploy does the same (1 of 90 runs of the spec file in a busy 2-CPU container), so WebKit's served row with seeding off failed now and then on that Shared Worker's missing answer. The row keeps every assertion, and takes a missing answer from that Shared Worker only on exactly WebKit's record: the engine is WebKit, neither a message nor an error event came, and the server served /rds/s.js and /rds/v2/s.js once each and never the h.js the script imports first. The check lives in test/webkit.mjs with a row of its own (the recorded run is taken; another engine, an error event, a message, the first URL asked twice as a seeded WebKit run asks it, either URL missing or asked twice, or the import asked, are not). A run it takes prints a line in the row's output naming WebKit's drop. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/test/rows.test.mjs | 12 +++++++++--- playwright/test/webkit.mjs | 24 ++++++++++++++++++++++++ playwright/test/webkit.test.mjs | 33 +++++++++++++++++++++++++++++++++ 3 files changed, 66 insertions(+), 3 deletions(-) create mode 100644 playwright/test/webkit.mjs create mode 100644 playwright/test/webkit.test.mjs diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index cb0c148..535034e 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -14,6 +14,7 @@ import { LOADS_KEY, OFF_LINE, SEEDED_LINE, fakerLine, initScript, processRandom, import { busyWorkerLine, sharedWorkerLine } from '../dist/settle.js'; import { NO_NOTE_LINE, readNote, unreadNoteLine } from '../dist/note.js'; import { BLANK_DRAWS, DRAWS, documentAt, workerAt } from './draws.mjs'; +import { webkitDroppedLine, webkitDroppedSharedWorker } from './webkit.mjs'; const PKG = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); const CLI = path.join(PKG, 'node_modules', '@playwright', 'test', 'cli.js'); @@ -1860,7 +1861,7 @@ for (const engine of ENGINES) { } for (const engine of ENGINES) { - test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and two module ones under default-src 'none') runs seeded, its draws the seed's, the module ones named as seeded late once; one the server redirects (classic, module, a Shared Worker, and the classic one again) ${engine === 'chromium' ? 'does not start seeded (keploy stops its load, which Chromium would send round the redirect again and again) and is named once for each address, the server asked for its first URL once for each start and never for the next, and off it ' : ''}runs at the URL the redirect gives, loading what sits beside it there, as off${engine === 'chromium' ? '' : ', unseeded and named, the server asked for its first URL ' + (engine === 'webkit' ? 'twice' : 'once')}; the server never sees keploy's mark`, async () => { + test(`served, in ${engine}: a worker's script in Latin-1 reads the same seeded as off (Chromium: as Latin-1); one served with a Content-Security-Policy (default-src 'none', script-src 'none', require-trusted-types-for 'script', and two module ones under default-src 'none') runs seeded, its draws the seed's, the module ones named as seeded late once; one the server redirects (classic, module, a Shared Worker, and the classic one again) ${engine === 'chromium' ? 'does not start seeded (keploy stops its load, which Chromium would send round the redirect again and again) and is named once for each address, the server asked for its first URL once for each start and never for the next, and off it ' : ''}runs at the URL the redirect gives, loading what sits beside it there, as off${engine === 'chromium' ? '' : ', unseeded and named, the server asked for its first URL ' + (engine === 'webkit' ? 'twice' : 'once')}${engine === 'webkit' ? " (a run where WebKit itself dropped the Shared Worker, as it sometimes does with no keploy, is taken only on exactly that record, and said)" : ''}; the server never sees keploy's mark`, async (t) => { const j = journal(); const a = await fakeAgent(j, (c) => (c.path === '/agent/scope/begin' ? { body: { v: 1, status: 'ok', seed: 'cd'.repeat(16) } } : undefined)); try { @@ -1880,12 +1881,17 @@ for (const engine of ENGINES) { assert.deepEqual(r.drew.redirect, ['error', 'error', 'error', 'error'], mode); } else { const [rd, rdm, rds, rd2] = r.drew.redirect; - assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rds.k, rds.H, rd2.k, rd2.H], ['rd', 'v2', 'rdm', 'v2', 'rds', 'v2', 'rd', 'v2'], mode); + // WebKit sometimes never runs this Shared Worker, with no keploy too (test/webkit.mjs): only that record is + // taken in place of its answer, and the row says so. + const dropped = webkitDroppedSharedWorker(engine, rds, r.seen); + if (dropped) t.diagnostic(webkitDroppedLine(mode)); + assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rd2.k, rd2.H], ['rd', 'v2', 'rdm', 'v2', 'rd', 'v2'], mode); + if (!dropped) assert.deepEqual([rds.k, rds.H], ['rds', 'v2'], mode); assert.match(rd.href, at('/rd/v2/w.js'), mode); assert.match(rd2.href, at('/rd/v2/w.js'), mode); assert.match(rdm.href, at('/rdm/v2/m.js'), mode); assert.match(rdm.meta, at('/rdm/v2/m.js'), mode); - assert.match(rds.href, at('/rds/v2/s.js'), mode); + if (!dropped) assert.match(rds.href, at('/rds/v2/s.js'), mode); } assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); // Each Web Worker's script loaded once (two start at /sv/l1.js, three at /sv/cn.js). diff --git a/playwright/test/webkit.mjs b/playwright/test/webkit.mjs new file mode 100644 index 0000000..484cd7e --- /dev/null +++ b/playwright/test/webkit.mjs @@ -0,0 +1,24 @@ +// WebKit's own defects that the browser rows meet, with keploy or without. + +/** + * Whether WebKit dropped served.spec.ts's redirected Shared Worker. WebKit + * (27.2, under Playwright 1.65) sometimes never runs a Shared Worker whose + * script the server redirects: it asks for /rds/s.js, follows the 302 to + * /rds/v2/s.js, gets that script, and then does nothing more, with no + * request for the h.js the script imports first, no error event and no + * message. The same happens with plain Playwright and no keploy, on a busy + * machine. The rows accept a missing answer only on exactly that record: + * engine is WebKit, got (what the fixture kept for the worker) is 'none' + * (neither a message nor an error event came), and the server (seen: every + * URL it was asked for in the run) served /rds/s.js and /rds/v2/s.js once + * each and never /rds/v2/h.js. Anything else is no drop: the rows assert + * it as they would. + */ +export function webkitDroppedSharedWorker(engine, got, seen) { + if (engine !== 'webkit' || got !== 'none') return false; + const asked = (p) => seen.filter((u) => u === p).length; + return asked('/rds/s.js') === 1 && asked('/rds/v2/s.js') === 1 && asked('/rds/v2/h.js') === 0; +} + +/** The line a row prints for each run where it accepted that drop. */ +export const webkitDroppedLine = (run) => `WebKit dropped the redirected Shared Worker /rds/s.js in the ${run} run: the server served /rds/s.js and /rds/v2/s.js once each, nothing asked for /rds/v2/h.js, and no message or error event came; WebKit's own defect (the same with no keploy), accepted on exactly this record`; diff --git a/playwright/test/webkit.test.mjs b/playwright/test/webkit.test.mjs new file mode 100644 index 0000000..7b89482 --- /dev/null +++ b/playwright/test/webkit.test.mjs @@ -0,0 +1,33 @@ +// Rows for test/webkit.mjs: the browser rows take WebKit's dropped Shared +// Worker only on exactly its record. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { webkitDroppedLine, webkitDroppedSharedWorker } from './webkit.mjs'; + +// What the server saw in a run of served.spec.ts where WebKit dropped the +// Shared Worker (as recorded with plain Playwright, no keploy): /rds/s.js and +// /rds/v2/s.js once each, then the next worker's script; no /rds/v2/h.js. +const DROPPED = ['/sv', '/sv/l1.js', '/sv/l1.js', '/sv', '/sv/cn.js', '/sv/csn.js', '/sv/ctt.js', '/sv/cn.js', '/sv/cn.js', '/sv', '/rd/w.js', '/rd/v2/w.js', '/rd/v2/h.js', '/rdm/m.js', '/rdm/v2/m.js', '/rdm/v2/h.mjs', '/rds/s.js', '/rds/v2/s.js', '/rd/w.js', '/rd/v2/w.js', '/rd/v2/h.js', '/sv', '/sv/sn.js', '/sv/snm.js']; +const at = (i, ...us) => [...DROPPED.slice(0, i), ...us, ...DROPPED.slice(i)]; +const without = (u) => DROPPED.filter((x) => x !== u); +const S = DROPPED.indexOf('/rds/v2/s.js') + 1; + +test("webkitDroppedSharedWorker: WebKit's dropped Shared Worker is taken only on exactly its record (WebKit, no message and no error event, /rds/s.js and /rds/v2/s.js served once each, /rds/v2/h.js never); any other engine, answer or record is not", () => { + assert.equal(webkitDroppedSharedWorker('webkit', 'none', DROPPED), true); + const not = [ + ['chromium', 'none', DROPPED, 'another engine'], + ['firefox', 'none', DROPPED, 'another engine'], + ['webkit', 'error', DROPPED, 'an error event'], + ['webkit', { k: 'rds', H: 'v2' }, DROPPED, 'a message'], + ['webkit', undefined, DROPPED, 'nothing kept'], + ['webkit', 'none', at(S, '/rds/s.js'), "the first URL asked twice (seeded WebKit's record)"], + ['webkit', 'none', without('/rds/s.js'), 'the first URL never asked'], + ['webkit', 'none', without('/rds/v2/s.js'), "the redirect's URL never asked"], + ['webkit', 'none', at(S, '/rds/v2/s.js'), "the redirect's URL asked twice"], + ['webkit', 'none', at(S, '/rds/v2/h.js'), 'the import asked'], + ['webkit', 'none', at(S, '/rds/v2/h.js', '/rds/v2/h.js'), 'the import asked twice'], + ['webkit', 'none', [], 'nothing asked'], + ]; + for (const [engine, got, seen, why] of not) assert.equal(webkitDroppedSharedWorker(engine, got, seen), false, why); + assert.match(webkitDroppedLine('off'), /^WebKit dropped the redirected Shared Worker \/rds\/s\.js in the off run: .*WebKit's own defect \(the same with no keploy\)/); +}); From 1a9780f5608bd682211ba3726c4307e735c32a90 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Mon, 16 Nov 2026 05:49:59 +0530 Subject: [PATCH 72/73] fix(playwright): end a request whose route handler throws after fallback A route handler that called route.fallback() and then threw left settle waiting on its request until the test's timeout: Playwright hands such a request to no other route and sends it nowhere, so with no keploy the test fails at once with the handler's error. Settle read Playwright's private _handlingPromise to tell a held request apart, and fallback() clears it. Settle now wraps each route's continue() and abort() as it wraps fulfill(), bound so Playwright names them as the test called them. When a handler throws, a request no continue() or abort() of its route returned for is not waited for; one they sent on or stopped is waited for until the browser reports its end. Rows: fallback() then throw fails at once with the handler's error in every engine; abort() after WebKit refused a redirect fails with Playwright's own route.abort error; unit cases for a fallback, a failed continue() and an abort() before the throw, and a throw on a context two settles share (the newest ends the request). README and comments updated; the WebKit drop record names microsoft/playwright#44134. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 2 +- playwright/src/settle.ts | 44 ++++++++++++++-------- playwright/test/fixtures/settle.spec.ts | 30 +++++++++++++++ playwright/test/rows.test.mjs | 43 ++++++++++++++++++++-- playwright/test/unit.test.mjs | 49 ++++++++++++++++++++++--- playwright/test/webkit.mjs | 2 +- 6 files changed, 144 insertions(+), 26 deletions(-) diff --git a/playwright/README.md b/playwright/README.md index 3336893..af171b8 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -43,7 +43,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim: each test is skipped at its begin, after the group's hooks have started. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. - **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped <titles>)`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. -- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor once a route of the test's own has answered it (its `route.fulfill` returned): the call has the route's whole answer, though the browser may report no end for it (Chromium for a Web Worker's script a route answers with a redirect, whose worker never starts, with no keploy too; Firefox for the first request of a redirect a route answers). Nor once the route's handler has thrown while the route still holds the call (WebKit refuses a redirect, and Playwright's `fulfill` throws its own error), as Playwright then sends it nowhere. A handler that returns and leaves its route holding the call might still send it on, so the call is waited for as any other. Settle reads this from Playwright's own route handling, past its API; with a Playwright that has none, such a call is waited for as any other. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. +- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor once a route of the test's own has answered it (its `route.fulfill` returned): the call has the route's whole answer, though the browser may report no end for it (Chromium for a Web Worker's script a route answers with a redirect, whose worker never starts, with no keploy too; Firefox for the first request of a redirect a route answers). Nor once the route's handler has thrown before a `route.continue()` or `route.abort()` of it returned (WebKit refuses a redirect, and Playwright's `fulfill` throws its own error; a handler calls `route.fallback()` and then throws), as Playwright then hands the call to no other route and sends it nowhere; a call that a `continue()` sent on, or that an `abort()` stopped, is waited for until the browser reports its end. A handler that returns and leaves its route holding the call might still send it on, so the call is waited for as any other. Settle reads this from Playwright's own route handling, past its API; with a Playwright that has none, such a call is waited for as any other. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see "What the run says when ..." below). diff --git a/playwright/src/settle.ts b/playwright/src/settle.ts index 6af5281..3199574 100644 --- a/playwright/src/settle.ts +++ b/playwright/src/settle.ts @@ -14,6 +14,8 @@ export class Inflight { private waiters: (() => void)[] = []; // The calls keploy's own route answered whole (answered). private readonly ended = new WeakSet<Request>(); + // The routes a continue() or abort() of returned for (routed). + private readonly sent = new WeakSet<Route>(); // The script URLs, with no fragment, that each frame started a Shared // Worker at, for the test's life (sharedWorker). private readonly shared = new Map<Frame | null, Set<string>>(); @@ -107,9 +109,10 @@ export class Inflight { }; // Nor for a request a route answers itself with a redirect (Chromium, // for a Web Worker's script: its worker never starts, with no keploy - // too; Firefox, for the redirect's first request), nor for one a route - // holds after its handler threw (WebKit refuses a redirect, and - // Playwright's fulfill throws): routed watches each route a page's + // too; Firefox, for the redirect's first request), nor for one whose + // route's handler threw before a continue() or abort() of it returned + // (WebKit refuses a redirect, and Playwright's fulfill throws; a handler + // falls back and then throws): routed watches each route a page's // routes, and then its context's, are handed. onRoute(context, (r, threw) => this.routed(r, threw)); context.pages().forEach(watch); @@ -263,29 +266,40 @@ export class Inflight { * Watches route, a request's route. Once its fulfill returns, the * request has the whole answer the route gave, and nothing of it is in * flight, whatever the browser reports after. threw: the routes' handler - * threw (called again, after the first call); a request its route still - * holds then (no route answered it or sent it on: the browser refused - * its answer, as WebKit refuses a redirect, or the handler threw first) - * goes nowhere, as Playwright sends it on no further. A handler that + * threw (called again, after the first call). Once a handler throws, + * Playwright hands the request to no other route and sends it on only + * through a continue() or abort() of its route: a request no continue() + * or abort() of its route returned for by then goes nowhere (the browser + * refused its answer, as WebKit refuses a redirect; the handler fell + * back, or threw first), and is not waited for. One that one of them + * sent on is waited for, as the browser goes on with it. A handler that * returns and leaves the request held keeps it in flight, as one that * still might send it on: the test's timeout ends that wait, naming it. */ private routed(route: Route, threw: boolean): void { const r = route.request(); if (threw) { - // Playwright's own, past its API: set while the handler may still - // answer the request, cleared once a route answered it or sent it on. - if ((route as unknown as { _handlingPromise?: unknown })._handlingPromise) this.answered(r); + if (!this.sent.has(route)) this.answered(r); return; } - // Bound, not called through Function.prototype.call: Playwright names - // an API call by how its caller invoked it, and its error must read as - // Playwright's own (route.fulfill: ...), as with no keploy. - const fulfill = route.fulfill.bind(route); + // Each bound and called by its own name (own.fulfill), never through + // Function.prototype.call or another name: Playwright can name an API + // call by the name its caller called it by, and its error must read as + // Playwright's own (route.fulfill: ..., route.continue: ...), as with no + // keploy. + const own = { fulfill: route.fulfill.bind(route), continue: route.continue.bind(route), abort: route.abort.bind(route) }; route.fulfill = async (o) => { - await fulfill(o); + await own.fulfill(o); this.answered(r); }; + route.continue = async (o) => { + await own.continue(o); + this.sent.add(route); + }; + route.abort = async (e) => { + await own.abort(e); + this.sent.add(route); + }; } private done(r: Request): void { diff --git a/playwright/test/fixtures/settle.spec.ts b/playwright/test/fixtures/settle.spec.ts index 234f108..4ff8d29 100644 --- a/playwright/test/fixtures/settle.spec.ts +++ b/playwright/test/fixtures/settle.spec.ts @@ -365,6 +365,36 @@ test("a call the test's route tries to send on after the browser refused its ans await sent; await returned; }); +// The same with the route's abort(), which fails with Playwright's own +// error too. +test("a call the test's route tries to fail after the browser refused its answer", async ({ page }) => { + await page.goto(`${base}/page`); + let end = (): void => {}; + const returned = new Promise<void>((resolve) => { end = resolve; }); + await page.route('**/slow', async (r) => { + await r.fulfill({ status: 302, headers: { location: `${base}/page` } }).catch(() => {}); + await r.abort(); + end(); + }); + const sent = page.waitForRequest(/\/slow$/); + await page.evaluate(() => { void fetch('/slow').catch(() => {}); }); + await sent; + await returned; +}); +// A route falls back, then throws: Playwright hands the call to no other +// route and sends it nowhere, and the handler's error fails the test. The +// body never ends: Playwright's runner ends the test at that error. +test("a call the test's route falls back on and then throws", async ({ page }) => { + await page.goto(`${base}/page`); + await page.route('**/slow', async (r) => { + await r.fallback(); + throw new Error("the test's route handler threw after its fallback"); + }); + const sent = page.waitForRequest(/\/slow$/); + await page.evaluate(() => { void fetch('/slow').catch(() => {}); }); + await sent; + await new Promise<void>(() => {}); +}); // A route sends the call on, then tries to answer it too: Playwright's // fulfill throws (the route is already handled), and so does the handler, // which fails the test. diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 535034e..4b47519 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -595,6 +595,43 @@ test("settle, in webkit: a call the test's own route tries to send on after WebK } finally { await p.close(); await a.close(); } }); +// So does its abort() there: route.abort: Route is already handled!, named +// as the test called it, and settle does not wait on the call. +test("settle, in webkit: a call the test's own route tries to fail after WebKit refused its redirect fails with Playwright's own error, and is not waited for", async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = "a call the test's route tries to fail after the browser refused its answer"; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, PW_BROWSER: 'webkit' }); + const t = tests[title]; + assert.equal(t.status, 'failed', out); + assert.match(t.error, /^Error: route\.abort: Route is already handled!\n/, out); + assert.deepEqual(t.steps.filter((x) => x.startsWith('keploy: wait for') && /\/slow(,|\s|$)/.test(x)), [], t.steps.join('\n')); + } finally { await p.close(); await a.close(); } +}); + +// A test's route falls back and then throws: Playwright sends the call +// nowhere, so settle does not wait on it, and the test fails at once with +// the handler's error, as with no keploy (0.1.1 waited on the call until +// the test's timeout, naming it in settle's step). +for (const engine of ['chromium', 'firefox', 'webkit']) { + test(`settle, in ${engine}: a call the test's own route falls back on and then throws is not waited for, and the test fails at once with the handler's error`, async () => { + const j = journal(); + const a = await fakeAgent(j); + const p = await pageServer(j); + try { + const title = "a call the test's route falls back on and then throws"; + const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, PW_BROWSER: engine, PW_TIMEOUT: '20000' }); + const t = tests[title]; + assert.equal(t.status, 'failed', out); + assert.match(t.error, /^Error: the test's route handler threw after its fallback(\n|$)/, out); + assert.doesNotMatch(t.error, /exceeded|Test timeout/, out); + assert.deepEqual(t.steps.filter((x) => x.startsWith('keploy: wait for') && /\/slow(,|\s|$)/.test(x)), [], t.steps.join('\n')); + } finally { await p.close(); await a.close(); } + }); +} + // Settle rows where a call is still waited for: the page server answers it // only once settle's step has begun (the rows' reporter asks /release then), // so the test ends by settle's own wait, with no deadline on its body. The @@ -1881,8 +1918,8 @@ for (const engine of ENGINES) { assert.deepEqual(r.drew.redirect, ['error', 'error', 'error', 'error'], mode); } else { const [rd, rdm, rds, rd2] = r.drew.redirect; - // WebKit sometimes never runs this Shared Worker, with no keploy too (test/webkit.mjs): only that record is - // taken in place of its answer, and the row says so. + // WebKit sometimes never runs this Shared Worker, with no keploy too (test/webkit.mjs, + // microsoft/playwright#44134): only that record is taken in place of its answer, and the row says so. const dropped = webkitDroppedSharedWorker(engine, rds, r.seen); if (dropped) t.diagnostic(webkitDroppedLine(mode)); assert.deepEqual([rd.k, rd.H, rdm.k, rdm.H, rd2.k, rd2.H], ['rd', 'v2', 'rdm', 'v2', 'rd', 'v2'], mode); @@ -1953,7 +1990,7 @@ for (const engine of ENGINES) { assert.deepEqual(kinds(r), kinds(o1), mode); assert.deepEqual(kinds(r), names, mode); assert.deepEqual(r.seen.filter((u) => u.includes('__keploy_worker')), [], mode); - // Each script loaded once by the browser, as off (a Shared Worker's twice seeded in Chromium and WebKit: keploy asks it too). + // Each Web Worker's script loaded once by the browser, as off (the row counts no Shared Worker's loads). for (const p of ['/mc/s.js', '/mc/sd.js', '/mc/sdc.js']) assert.equal(r.seen.filter((u) => u === p).length, 1, `${mode} ${p}`); } const at = (r, i) => r.drew.own[i]; diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index 930155f..e7b6425 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -277,10 +277,12 @@ async function ticks() { for (let i = 0; i < 5; i++) await Promise.resolve(); } -// A fake Playwright route, as far as settle reads it: _handlingPromise is -// set while a handler may still answer it; fulfill throws as Playwright's -// does when the browser refuses the answer (refuse: WebKit's redirect, the -// route still the handler's) or when the route was already handled. +// A fake Playwright route, as far as a handler drives it: _handlingPromise +// is set while a handler may still answer it (settle reads it no more); +// fulfill throws as Playwright's does when the browser refuses the answer +// (refuse: WebKit's redirect, the route still the handler's), and each +// call throws when the route was already handled (fallback hands it to the +// next route, as Playwright's does). function fakeRoute(req) { const route = { _handlingPromise: {}, @@ -294,6 +296,14 @@ function fakeRoute(req) { if (!route._handlingPromise) throw new Error('Route is already handled!'); route._handlingPromise = null; }, + async abort() { + if (!route._handlingPromise) throw new Error('Route is already handled!'); + route._handlingPromise = null; + }, + async fallback() { + if (!route._handlingPromise) throw new Error('Route is already handled!'); + route._handlingPromise = null; + }, }; return route; } @@ -311,7 +321,7 @@ function withRoutes(target, ctx = null) { return target; } -test('settle: a request a route of the test\'s answered (fulfill returned) has ended, though the browser reports no end; one the route still holds when its handler throws (the browser refused its answer, or nothing answered it) is not waited for; one the handler holds, or sent on, before or after a refused answer, is', async () => { +test('settle: a request a route of the test\'s answered (fulfill returned) has ended, though the browser reports no end; one whose handler throws before a continue() or abort() of its route returned (the browser refused its answer, the handler fell back, its continue() failed, or nothing answered it) is not waited for; one the handler holds, sent on or aborted, before or after a refused answer, is', async () => { const ctx = withRoutes(fakeContext()); const inflight = new Inflight(ctx); const page = withRoutes(new EventEmitter()); @@ -357,6 +367,23 @@ test('settle: a request a route of the test\'s answered (fulfill returned) has e assert.equal(inflight.size, 1); ctx.emit('requestfinished', f.r); assert.equal(inflight.size, 0); + // Fallen back, then the handler throws: Playwright hands the request to no + // other route and sends it nowhere. + const g = await sent(async (route) => { await route.fallback(); throw new Error('after the fallback'); }); + assert.equal(await g.done, 'after the fallback'); + assert.equal(inflight.size, 0); + // Fallen back, then a continue() that fails (already handled), which the + // handler throws: sent nowhere. + const h = await sent(async (route) => { await route.fallback(); await route.continue(); }); + assert.match(await h.done, /already handled/); + assert.equal(inflight.size, 0); + // Aborted, then the handler throws: the browser fails the request, and it + // ends as the browser reports that. + const i = await sent(async (route) => { await route.abort(); throw new Error('after the abort'); }); + assert.equal(await i.done, 'after the abort'); + assert.equal(inflight.size, 1); + ctx.emit('requestfailed', i.r); + assert.equal(inflight.size, 0); }); test('settle: a route the page\'s routes hand on reaches its context\'s, watched there too: one the context\'s handler answers has ended; one it still holds when it throws is not waited for', async () => { @@ -394,7 +421,7 @@ test('settle: a page or context with no _onRoute (a Playwright without it) gets assert.equal(inflight.size, 1); }); -test('settle: of two Inflights on one context (Playwright reuses it between tests), the newest watches its routes; its _onRoute is wrapped once', async () => { +test('settle: of two Inflights on one context (Playwright reuses it between tests), the newest watches its routes, a handler that throws holding its request too; its _onRoute is wrapped once', async () => { const ctx = withRoutes(fakeContext()); const first = new Inflight(ctx); const wrapped = ctx._onRoute; @@ -410,6 +437,16 @@ test('settle: of two Inflights on one context (Playwright reuses it between test await route.fulfill({}); assert.equal(second.size, 0); assert.equal(first.size, 1); + // The second test's handler throws while its route holds the request: the + // newest Inflight ends it, and the first's is left as it was. + const held = fakeRequest(null); + ctx.emit('request', held); + assert.equal(second.size, 1); + assert.equal(first.size, 2); + ctx.handler = async () => { throw new Error('the second test\'s handler'); }; + assert.equal(await ctx._onRoute(fakeRoute(held)).then(() => 'returned', (e) => e.message), 'the second test\'s handler'); + assert.equal(second.size, 0); + assert.equal(first.size, 2); }); test('settle: a call keploy\'s route answered has ended, though the browser reports no end (Chromium, a Shared Worker\'s script); one reported after its answer is not waited for', async () => { diff --git a/playwright/test/webkit.mjs b/playwright/test/webkit.mjs index 484cd7e..0301e3f 100644 --- a/playwright/test/webkit.mjs +++ b/playwright/test/webkit.mjs @@ -7,7 +7,7 @@ * /rds/v2/s.js, gets that script, and then does nothing more, with no * request for the h.js the script imports first, no error event and no * message. The same happens with plain Playwright and no keploy, on a busy - * machine. The rows accept a missing answer only on exactly that record: + * machine (microsoft/playwright#44134). The rows accept a missing answer only on exactly that record: * engine is WebKit, got (what the fixture kept for the worker) is 'none' * (neither a message nor an error event came), and the server (seen: every * URL it was asked for in the run) served /rds/s.js and /rds/v2/s.js once From c88bfa83794162e4136f7290fc47a29eebe45758 Mon Sep 17 00:00:00 2001 From: slayerjain <shubhamkjain@outlook.com> Date: Mon, 16 Nov 2026 07:32:19 +0530 Subject: [PATCH 73/73] test(playwright): check a route abort's error code against no keploy Settle wraps each route's abort(), and keploy's worker routes wrap the abort of a worker script's load they hand to the test's routes, but no row checked that the error code reaches the browser: a wrapper that dropped it left every row green. Rows: in every engine, a fetch the test's own route aborts with accessdenied (on its page and on its context) or with no code, and a Web Worker's script it aborts with accessdenied, with blockedbyclient or with no code, fail with the page's requestfailed text they have with no keploy, seeded and with seeding off, and never reach the server. In Chromium, keploy's stop of a seeded Web Worker's script the server redirects fails it as a route's abort('blockedbyclient') does with no keploy. Engines name codes apart only in part (WebKit gives every abort one text; Chromium reports every failed Web Worker script load as net::ERR_ABORTED), so the row checks that a dropped code would show where they do: a fetch's text in Chromium and Firefox, and a worker script's in Firefox. A unit row checks that the watched route's fulfill, continue and abort hand Playwright what the test gave them, on a page's route and on its context's. README: in a Playwright trace, the source link of a route action points into @keploy/playwright, as settle wraps the route's calls. Signed-off-by: slayerjain <shubhamkjain@outlook.com> --- playwright/README.md | 2 +- playwright/test/fixtures/aborts.spec.ts | 76 +++++++++++++++++++++++++ playwright/test/rows.test.mjs | 50 ++++++++++++++++ playwright/test/unit.test.mjs | 34 ++++++++++- 4 files changed, 158 insertions(+), 4 deletions(-) create mode 100644 playwright/test/fixtures/aborts.spec.ts diff --git a/playwright/README.md b/playwright/README.md index af171b8..b3f6e77 100644 --- a/playwright/README.md +++ b/playwright/README.md @@ -43,7 +43,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco - **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again. Calls that `beforeAll`/`afterAll` hooks and worker fixtures make outside every test are answered from the worker's claim, one slice per Playwright project. The `beforeAll` and `afterAll` hooks of a describe group whose tests keploy has all set aside still run, and their calls are answered from the claim: each test is skipped at its begin, after the group's hooks have started. - **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `. - **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`). The test's output shows it too (`keploy: ... (keploy skipped <titles>)`), since Playwright's list, line and dot reporters print a test's output but not its skip's reason. -- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor once a route of the test's own has answered it (its `route.fulfill` returned): the call has the route's whole answer, though the browser may report no end for it (Chromium for a Web Worker's script a route answers with a redirect, whose worker never starts, with no keploy too; Firefox for the first request of a redirect a route answers). Nor once the route's handler has thrown before a `route.continue()` or `route.abort()` of it returned (WebKit refuses a redirect, and Playwright's `fulfill` throws its own error; a handler calls `route.fallback()` and then throws), as Playwright then hands the call to no other route and sends it nowhere; a call that a `continue()` sent on, or that an `abort()` stopped, is waited for until the browser reports its end. A handler that returns and leaves its route holding the call might still send it on, so the call is waited for as any other. Settle reads this from Playwright's own route handling, past its API; with a Playwright that has none, such a call is waited for as any other. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. +- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. A context the test makes itself (`browser.newContext()`) is the test's own to close. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor once a route of the test's own has answered it (its `route.fulfill` returned): the call has the route's whole answer, though the browser may report no end for it (Chromium for a Web Worker's script a route answers with a redirect, whose worker never starts, with no keploy too; Firefox for the first request of a redirect a route answers). Nor once the route's handler has thrown before a `route.continue()` or `route.abort()` of it returned (WebKit refuses a redirect, and Playwright's `fulfill` throws its own error; a handler calls `route.fallback()` and then throws), as Playwright then hands the call to no other route and sends it nowhere; a call that a `continue()` sent on, or that an `abort()` stopped, is waited for until the browser reports its end. To see this, settle wraps each route's `fulfill`, `continue` and `abort`, which give Playwright's own what the test gave them (an abort's error code fails the call with the text it has with no keploy), so in a Playwright trace the source link of a route action (Fulfill, Continue or Abort request) points into @keploy/playwright rather than at the test's line; the test's reports (HTML, list, JSON) show no route action and do not change. A handler that returns and leaves its route holding the call might still send it on, so the call is waited for as any other. Settle reads this from Playwright's own route handling, past its API; with a Playwright that has none, such a call is waited for as any other. When a frame loads another document, the calls it drops are those the browser reported before the new document's answer: a call the new document makes as it loads (a script or an image its parser asks for) is waited for, even when the browser reports it before the frame's commit, as Firefox can. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker. A Shared Worker's are not: Playwright reports none of a Shared Worker's calls in any engine, and Chromium reports no end for its script. So in every keploy test, seeded or not, the page's `SharedWorker` starts each worker as the app asked and tells the test its script's URL, and settle does not wait on a request of that frame at that URL. The test says so once for each Shared Worker script, naming it and the next step: have the test wait for what the worker does (a message it posts, or what the page shows from it) before it ends, as a call the worker still has in flight then is cut there. - **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text. - **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see "What the run says when ..." below). diff --git a/playwright/test/fixtures/aborts.spec.ts b/playwright/test/fixtures/aborts.spec.ts new file mode 100644 index 0000000..b3c5373 --- /dev/null +++ b/playwright/test/fixtures/aborts.spec.ts @@ -0,0 +1,76 @@ +import fs from 'node:fs'; +import type { Page } from '@playwright/test'; +import { test } from '../../dist'; + +// Run in Chromium, Firefox and WebKit (PW_BROWSER), under keploy and with no +// keploy. Each test has one request of its page fail, from the page at /sv, +// and keeps what the page saw of it (a fetch's 'failed', a Web Worker's +// 'error' or its message) and the text the page's requestfailed event gave +// for it (its failure's errorText; 'finished' when it ended as the browser's +// requestfinished instead), under the test's title: a fetch the test's own +// route aborts with an error code, on its page and on its context, and with +// none; a Web Worker's script the test's route aborts with an error code, +// with blockedbyclient and with none; and (Chromium, seeded: keploy stops +// its load) a Web Worker's script the server redirects. +const base = process.env.DRAWS_BASE ?? ''; +const keep = (name: string, v: unknown): void => fs.writeFileSync(`${process.env.MARKER_DIR}/${name}.json`, JSON.stringify(v)); + +// The end the page's request events give the first request of the page at +// path: the failure's text, or 'finished'. +const ended = (page: Page, path: string): Promise<string> => new Promise((resolve) => { + const at = (u: string): boolean => new URL(u).pathname === path; + page.on('requestfailed', (r) => { if (at(r.url())) resolve(r.failure()?.errorText ?? 'none'); }); + page.on('requestfinished', (r) => { if (at(r.url())) resolve('finished'); }); +}); + +const fetched = async (page: Page, path: string): Promise<void> => { + await page.goto(`${base}/sv`); + const end = ended(page, path); + const saw = await page.evaluate((p) => fetch(p).then(() => 'ok', () => 'failed'), path); + keep(test.info().title, [saw, await end]); +}; + +const started = async (page: Page, path: string): Promise<void> => { + await page.goto(`${base}/sv`); + const end = ended(page, path); + const saw = await page.evaluate((p) => new Promise((resolve) => { + const w = new Worker(p); + w.onmessage = (e) => resolve((e.data as { k?: string }).k ?? 'message'); + w.onerror = () => resolve('error'); + }), path); + keep(test.info().title, [saw, await end]); +}; + +test('abort with a code on page', async ({ page }) => { + await page.route('**/ab/x', (r) => r.abort('accessdenied')); + await fetched(page, '/ab/x'); +}); + +test('abort with a code on context', async ({ context, page }) => { + await context.route('**/ab/x', (r) => r.abort('accessdenied')); + await fetched(page, '/ab/x'); +}); + +test('abort with no code on page', async ({ page }) => { + await page.route('**/ab/x', (r) => r.abort()); + await fetched(page, '/ab/x'); +}); + +test('abort a worker script with a code', async ({ page }) => { + await page.route('**/rd/w.js', (r) => r.abort('accessdenied')); + await started(page, '/rd/w.js'); +}); + +test('abort a worker script with blockedbyclient', async ({ page }) => { + await page.route('**/rd/w.js', (r) => r.abort('blockedbyclient')); + await started(page, '/rd/w.js'); +}); + +test('abort a worker script with no code', async ({ page }) => { + await page.route('**/rd/w.js', (r) => r.abort()); + await started(page, '/rd/w.js'); +}); + +test('a worker script the server redirects', async ({ page }) => { + await started(page, '/rd/w.js'); +}); diff --git a/playwright/test/rows.test.mjs b/playwright/test/rows.test.mjs index 4b47519..0f07920 100644 --- a/playwright/test/rows.test.mjs +++ b/playwright/test/rows.test.mjs @@ -2068,6 +2068,56 @@ for (const engine of ENGINES) { }); } +// aborts.spec.ts's tests of a route of the test's own that aborts a request, +// and the one where keploy stops a seeded Web Worker's script load (Chromium). +const AB = ['abort with a code on page', 'abort with a code on context', 'abort with no code on page', 'abort a worker script with a code', 'abort a worker script with blockedbyclient', 'abort a worker script with no code']; +const AB_G = 'abort with a code|abort with no code|abort a worker script'; +const AB_STOP = 'a worker script the server redirects'; + +// Settle wraps each route's abort (and keploy's worker routes the abort of +// a worker script's load they hand to the test's routes), so the error code +// the test gives reaches the browser: the page's requestfailed text is what +// it is with no keploy. keploy's own stop of a seeded Web Worker's script +// the server redirects (Chromium) fails the load with blockedbyclient, as a +// route's abort('blockedbyclient') does with no keploy. Not every engine +// names codes apart (measured): WebKit gives every abort one text (Blocked by +// Web Inspector); Firefox gives blockedbyclient the text of no code +// (NS_ERROR_FAILURE), and accessdenied its own; Chromium names a fetch's code +// (net::ERR_ACCESS_DENIED, net::ERR_FAILED) but reports every failed Web +// Worker script load to the page as net::ERR_ABORTED, whatever the code. So a +// code dropped on the way shows in a fetch's text in Chromium and Firefox, +// and in a worker script's in Firefox, for accessdenied: the row checks that +// it would there. +for (const engine of ENGINES) { + test(`aborts, in ${engine}: a route of the test's own that aborts a fetch (with an error code, on its page or its context, or with none) or a Web Worker's script (with an error code, with blockedbyclient or with none) fails it with the requestfailed text it has with no keploy, seeded and with seeding off, and the server never sees it${engine === 'chromium' ? "; keploy's stop of a seeded Web Worker's script the server redirects fails it as a route's abort('blockedbyclient') does with no keploy" : ''}`, async (t) => { + const j = journal(); + const a = await fakeAgent(j); + try { + const only = ['aborts.spec.ts', '-g', AB_G]; + const plain = await drawRun(j, { KEPLOY_MOCK_AGENT: undefined, KEPLOY_TEST_PID: undefined, PW_BROWSER: engine }, only, AB); + const off = await drawRun(j, { ...keployEnv(a.url), PW_BROWSER: engine, PW_NOSEED: '1' }, only, AB); + const on = await drawRun(j, { ...keployEnv(a.url), PW_BROWSER: engine }, engine === 'chromium' ? ['aborts.spec.ts'] : only, engine === 'chromium' ? [...AB, AB_STOP] : AB); + const text = (r, n) => r.drew[n][1]; + for (const [mode, r] of [['no keploy', plain], ['seeding off', off], ['seeded', on]]) t.diagnostic(`${engine}, ${mode}: ${[...AB, AB_STOP].filter((n) => r.drew[n] !== undefined).map((n) => `${n}: ${r.drew[n].join(' ')}`).join('; ')}`); + // Each failed, as the page saw it, and the page's requestfailed event named it. + assert.deepEqual(AB.map((n) => plain.drew[n][0]), ['failed', 'failed', 'failed', 'error', 'error', 'error']); + for (const n of AB) assert.notEqual(text(plain, n), 'finished', n); + for (const n of AB) { + assert.deepEqual(on.drew[n], plain.drew[n], `seeded: ${n}`); + assert.deepEqual(off.drew[n], plain.drew[n], `seeding off: ${n}`); + } + // Where the engine names the code apart, a code dropped on the way would show. + if (engine !== 'webkit') assert.notEqual(text(plain, 'abort with a code on page'), text(plain, 'abort with no code on page')); + if (engine === 'firefox') assert.notEqual(text(plain, 'abort a worker script with a code'), text(plain, 'abort a worker script with no code')); + if (engine === 'chromium') assert.deepEqual(on.drew[AB_STOP], plain.drew['abort a worker script with blockedbyclient']); + // No request a route aborted reached the server; keploy asked it once + // for the redirected script (to see the redirect), never for the next. + for (const r of [plain, off]) assert.deepEqual(r.seen.filter((u) => u.startsWith('/ab/') || u.startsWith('/rd/')), []); + assert.deepEqual(on.seen.filter((u) => u.startsWith('/ab/') || u.startsWith('/rd/')), engine === 'chromium' ? ['/rd/w.js'] : []); + } finally { await a.close(); } + }); +} + // once.spec.ts's tests, and the workers each one's page starts, in order // (each script's URL, and w a classic Web Worker, m a module one, s a // Shared Worker), and the doc tests' document paths (/ov for the others). diff --git a/playwright/test/unit.test.mjs b/playwright/test/unit.test.mjs index e7b6425..581c5d4 100644 --- a/playwright/test/unit.test.mjs +++ b/playwright/test/unit.test.mjs @@ -282,21 +282,26 @@ async function ticks() { // fulfill throws as Playwright's does when the browser refuses the answer // (refuse: WebKit's redirect, the route still the handler's), and each // call throws when the route was already handled (fallback hands it to the -// next route, as Playwright's does). +// next route, as Playwright's does). calls: each fulfill, continue and +// abort it got, with what it was given. function fakeRoute(req) { const route = { _handlingPromise: {}, + calls: [], request: () => req, async fulfill(o = {}) { + route.calls.push(['fulfill', o]); if (!route._handlingPromise) throw new Error('Route is already handled!'); if (o.refuse) throw new Error('Cannot fulfill with redirect status: 302'); route._handlingPromise = null; }, - async continue() { + async continue(o) { + route.calls.push(['continue', o]); if (!route._handlingPromise) throw new Error('Route is already handled!'); route._handlingPromise = null; }, - async abort() { + async abort(code) { + route.calls.push(['abort', code]); if (!route._handlingPromise) throw new Error('Route is already handled!'); route._handlingPromise = null; }, @@ -409,6 +414,29 @@ test('settle: a route the page\'s routes hand on reaches its context\'s, watched assert.equal(inflight.size, 0); }); +test('settle: the watched route\'s fulfill, continue and abort reach Playwright\'s own with what the test gave them (an abort\'s error code, a continue\'s options, a fulfill\'s answer), on a page\'s route and on its context\'s', async () => { + const ctx = withRoutes(fakeContext()); + const inflight = new Inflight(ctx); + const page = withRoutes(new EventEmitter(), ctx); + ctx.emit('page', page); + const given = [['abort', 'accessdenied'], ['abort', undefined], ['continue', { url: 'http://h/m.js', headers: { 'x-auth': 'k' } }], ['continue', undefined], ['fulfill', { status: 201, body: 'b' }]]; + for (const on of ['page', 'context']) { + for (const [name, arg] of given) { + const r = fakeRequest(page); + ctx.emit('request', r); + const route = fakeRoute(r); + const handler = (rt) => rt[name](arg); + page.handler = on === 'page' ? handler : () => 'next'; + ctx.handler = handler; + assert.equal(await page._onRoute(route).then(() => 'returned', (e) => e.message), 'returned'); + assert.deepEqual(route.calls, [[name, arg]], `${on} ${name} ${JSON.stringify(arg)}`); + } + } + // Each one aborted or sent on is in flight until the browser reports its + // end; each fulfilled one has ended. + assert.equal(inflight.size, 8); +}); + test('settle: a page or context with no _onRoute (a Playwright without it) gets none, and its requests are waited for as before', async () => { const ctx = fakeContext(); const inflight = new Inflight(ctx);