Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion playwright/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ With `KEPLOY_MOCK_AGENT` unset or `off`, nothing changes. Only `keploy mock reco
- **Claim.** Each worker claims keploy's run before its first hook or test (`POST /agent/instance`). The claim uses the worker's Playwright project name as its key, or `.` for an unnamed project, and names the project directory keploy names (`KEPLOY_MOCK_PROJECT`) as the folder of its tests, the folder the test files are named from (the current directory only when keploy names none). So a suite run from a folder above its keploy project, such as `cd .. && npx playwright test` in `mock.command`, is filed under its project. A worker that Playwright starts again after a failed test claims again.
- **Begin.** Each test begins (`POST /agent/scope/begin`) before its `beforeEach` hooks. keploy builds the test's ID as `[project::]file::titles`. The file is relative to the project directory keploy names (`KEPLOY_MOCK_PROJECT`). The titles are the describe groups and the test's own title, joined with ` › `.
- **Skip.** When keploy has set a test aside, the test is skipped through Playwright's own skip, and the report shows keploy's reason (`keploy: ...`).
- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker.
- **Settle.** Before the test's `context` (and its `page`) closes, the test waits until none of its pages' calls is in flight. Calls are counted by the browser's own request events, and no time window decides. A call that never ends keeps the test waiting until Playwright's test timeout fails it. The test shows a step that names each call still in flight. A call is not waited for once the browser has cancelled it, which happens when its page loads another document, its frame is removed, its page closes or the test closes its own context. Nor are the calls a page's frames had in flight when the page loaded another document: the frames go with the old document, and Playwright reports no end for their calls, ordinary or keepalive (`navigator.sendBeacon`, `fetch` with `keepalive`, the telemetry a payment form's frame sends, which the browser goes on sending), nor for a frame's own document request still loading, so waiting for them would hold the test until its timeout. The browser goes on sending a keepalive call only while the test's context lives: one still in flight when the test ends is cut by the context close, so its answer may not be recorded. A closing page cancels only its own calls: a popup that closes cancels nothing of its opener's. A route change inside the page (the history API) cancels nothing, even after an answered call. A service worker's calls are counted too. No page's change drops them, but the test closing its own context does, as it ends the worker.
- **End.** The test ends (`POST /agent/scope/end`) after its last teardown, with its own outcome. When keploy saw a call with no recorded answer, or a recorded call the test no longer made, the test fails with keploy's text.
- **Given values.** `await given(name, value)` tells keploy a value the test made itself. Examples are a session token it signs and a name it stamps with the time. keploy then follows the value wherever the test's calls carry it, as it follows an incoming request's values in a server test. Give a distinctive value, whole, as the calls carry it. `given` must be called inside a test, or in its `beforeEach` or `afterEach` hooks. It returns the value. It needs a keploy whose agent serves `POST /agent/scope/given` (see the limits below).

Expand Down
12 changes: 10 additions & 2 deletions playwright/src/settle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,17 @@ export class Inflight {
this.drop((o) => o.frame === f && !o.nav);
});
// Nor for the calls of a page that closes. Only that page's calls: a
// popup that closes cancels nothing of its opener. (A removed frame's
// calls the browser does end, as failed requests.)
// popup that closes cancels nothing of its opener.
p.on('close', () => this.drop((o) => o.frame !== null && pageOf(o.frame) === p));
// Nor for the calls of a frame that is gone: removed, or gone with
// the document it was in when the page loaded another (a payment
// form's frame). Playwright reports no end for the calls such a frame
// still had in flight when it went with the page's old document:
// ordinary, keepalive (navigator.sendBeacon, fetch with keepalive,
// which the browser goes on sending only until the test closes its
// context), or the frame's own document request, still loading. Each
// frame that goes is reported apart, a frame inside it too.
p.on('framedetached', (f) => this.drop((o) => o.frame === f));
};
context.pages().forEach(watch);
context.on('page', watch);
Expand Down
8 changes: 6 additions & 2 deletions playwright/test/fixtures/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ export default defineConfig({
reporter: [['json'], ['./steps-reporter.cjs']],
outputDir: process.env.PW_OUTPUT_DIR || 'test-results',
// PW_UNNAMED=1: one unnamed project, Playwright's default when a config
// lists none.
projects: [{ name: process.env.PW_UNNAMED ? '' : 'p1', use: { headless: true } }],
// lists none. PW_BROWSER=firefox or webkit runs the rows in that browser
// (Chromium, Playwright's default, otherwise).
projects: [{
name: process.env.PW_UNNAMED ? '' : 'p1',
use: { headless: true, browserName: (process.env.PW_BROWSER || 'chromium') as 'chromium' | 'firefox' | 'webkit' },
}],
});
64 changes: 64 additions & 0 deletions playwright/test/fixtures/settle.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import type { Frame, Page } from '@playwright/test';
import { test } from '../../dist';

const base = process.env.SLOW_BASE ?? '';
Expand Down Expand Up @@ -93,6 +94,69 @@ test('a call its removed cross-site frame had in flight', async ({ page }) => {
await page.evaluate(() => { document.getElementById('child')?.remove(); });
await detached;
});
// A frame from another site, loaded and ready (the payment form's frame).
const crossSiteChild = async (page: Page): Promise<Frame> => {
await page.goto(`${base}/page`);
const attached = page.waitForEvent('frameattached');
await page.evaluate((src) => new Promise<void>((resolve) => {
const f = document.createElement('iframe');
f.id = 'child';
f.onload = () => resolve();
f.src = src;
document.body.appendChild(f);
}), `${base.replace('//127.0.0.1:', '//localhost:')}/page?child`);
return attached;
};
// The page loads another document: its frames go with the old one.
const nextDocument = (page: Page) => page.goto(`${base}/page?next`);
test('a beacon its frame sends as the page loads another document', async ({ page }) => {
// Telemetry a frame sends as its page hides (a payment form's). A
// keepalive call outlives its document: the browser still sends it.
const child = await crossSiteChild(page);
await child.evaluate(() => { addEventListener('pagehide', () => { navigator.sendBeacon('/beacon', 'b'); }); });
await nextDocument(page);
await reached();
});
test('a beacon its frame had in flight when the page loads another document', async ({ page }) => {
const child = await crossSiteChild(page);
const sent = page.waitForRequest(/\/beacon$/);
await child.evaluate(() => { navigator.sendBeacon('/beacon', 'b'); });
await sent;
await reached();
await nextDocument(page);
});
test('a keepalive fetch its frame had in flight when the page loads another document', async ({ page }) => {
const child = await crossSiteChild(page);
const sent = page.waitForRequest(/\/beacon$/);
await child.evaluate(() => { void fetch('/beacon', { method: 'POST', body: 'b', keepalive: true }); });
await sent;
await reached();
await nextDocument(page);
});
test('a call its frame had in flight when the page loads another document', async ({ page }) => {
const child = await crossSiteChild(page);
const sent = page.waitForRequest(/\/slow$/);
await child.evaluate(() => { void fetch('/slow'); });
await sent;
await reached();
await nextDocument(page);
});
test('a document its frame was still loading when the page loads another document', async ({ page }) => {
// A frame whose own document has not finished loading (a payment form's
// frame, loaded lazily) goes with the page's old document: Playwright
// reports no end for the frame's own navigation request either. The
// frame commits /slowdoc, whose last byte the page server holds.
const child = await crossSiteChild(page);
await child.goto(`${base.replace('//127.0.0.1:', '//localhost:')}/slowdoc`, { waitUntil: 'commit' });
await nextDocument(page);
});
test('a beacon in flight from the page', async ({ page }) => {
await page.goto(`${base}/page`);
const sent = page.waitForRequest(/\/beacon$/);
await page.evaluate(() => { navigator.sendBeacon('/beacon', 'b'); });
await sent;
await reached();
});
test('a call its closed page had in flight', async ({ page, context }) => {
await page.goto(`${base}/page`);
const other = await context.newPage();
Expand Down
52 changes: 52 additions & 0 deletions playwright/test/rows.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,16 @@ async function pageServer(j) {
void released.then(() => res.end('slow'));
return;
}
// A keepalive call (navigator.sendBeacon, fetch with keepalive): a POST,
// answered as /slow is.
if (req.url === '/beacon' && req.method === 'POST') {
req.resume();
req.on('end', () => { j.add({ page: 'beacon sent' }); saw(); });
res.on('close', () => { if (!res.writableFinished) j.add({ page: 'beacon cut' }); });
res.on('finish', () => j.add({ page: 'beacon answered' }));
void released.then(() => res.end('beacon'));
return;
}
res.writeHead(404); res.end();
});
await new Promise((r) => srv.listen(0, '127.0.0.1', r));
Expand Down Expand Up @@ -362,6 +372,48 @@ for (const [title, why] of [
});
}

// Settle rows where the page loads another document while one of its
// frames has a call in flight: the frame goes with the old document, and
// Playwright reports no end for its call, ordinary or keepalive
// (navigator.sendBeacon, fetch with keepalive, which the browser still
// sends: the page server sees it), nor for the frame's own document request
// when that document was still loading. The call is not waited for, so the
// test does not hang until its timeout.
for (const [title, why, call] of [
['a beacon its frame sends as the page loads another document', 'its frame sends it as the page loads another document', 'beacon'],
['a beacon its frame had in flight when the page loads another document', 'the page loads another document, though a beacon outlives its frame', 'beacon'],
['a keepalive fetch its frame had in flight when the page loads another document', 'the page loads another document, though a keepalive fetch outlives its frame', 'beacon'],
['a call its frame had in flight when the page loads another document', 'the page loads another document, with an ordinary call of its frame in flight', 'slow'],
['a document its frame was still loading when the page loads another document', 'the page loads another document while the frame\'s own document is still loading', 'slowdoc'],
]) {
test(`settle: a frame's call is not waited for once ${why}`, async () => {
const j = journal();
const a = await fakeAgent(j);
const p = await pageServer(j);
try {
const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '20000' });
const t = tests[title];
assert.equal(t.status, 'passed', out);
assert.deepEqual(t.steps.filter((s) => s.startsWith('keploy:')), []);
assert.ok(j.events.some((e) => e.page === `${call} sent`));
} finally { await p.close(); await a.close(); }
});
}

test('settle: a beacon from a page that stays is still waited for', async () => {
const j = journal();
const a = await fakeAgent(j);
const p = await pageServer(j);
try {
const title = 'a beacon in flight from the page';
const { tests, out } = await playwright(['settle.spec.ts', '-g', title], { ...keployEnv(a.url), SLOW_BASE: p.url, NO_RELEASE: '1', PW_TIMEOUT: '4000' });
const t = tests[title];
assert.equal(t.status, 'timedOut', out);
const named = /^keploy: wait for the test's 1 call\(s\) in flight before its browser context closes: POST http:\/\/127\.0\.0\.1:\d+\/beacon$/;
assert.ok(t.steps.some((s) => named.test(s)), t.steps.join('\n'));
} finally { await p.close(); await a.close(); }
});

// Settle rows where nothing was cancelled: the call is still waited for, and
// the test times out naming it.
for (const [title, why, call] of [
Expand Down
28 changes: 28 additions & 0 deletions playwright/test/unit.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,34 @@ test('settle: a cancel that comes while settle already waits wakes it (its page
}
});

test('settle: a frame that is gone drops only its own calls, its own document request too, and wakes settle once none is left', async () => {
// Two frames of one page, each with a call Playwright reports no end for
// once the frame is gone, and each with its own document still loading.
const ctx = fakeContext();
const inflight = new Inflight(ctx);
const page = new EventEmitter();
ctx.emit('page', page);
const a = { page: () => page };
const b = { page: () => page };
const beacon = (frame, url) => ({ frame: () => frame, isNavigationRequest: () => false, method: () => 'POST', url: () => url });
const doc = (frame, url) => ({ frame: () => frame, isNavigationRequest: () => true, method: () => 'GET', url: () => url });
ctx.emit('request', doc(a, 'http://localhost/slowdoc'));
ctx.emit('request', beacon(a, 'http://localhost/a'));
ctx.emit('request', doc(b, 'http://localhost/bdoc'));
ctx.emit('request', beacon(b, 'http://localhost/b'));
let settled = false;
inflight.settled().then(() => { settled = true; });
await ticks();
page.emit('framedetached', a);
await ticks();
assert.deepEqual(inflight.describe(), ['GET http://localhost/bdoc', 'POST http://localhost/b'], 'the gone frame\'s calls are dropped, its document request too; the other frame\'s call and document request are not');
assert.equal(settled, false, 'settle still waits for the other frame\'s calls');
page.emit('framedetached', b);
await ticks();
assert.equal(inflight.size, 0, 'no call is left in flight');
assert.equal(settled, true, 'settle is woken');
});

test('decide: inert, refused, active', () => {
const never = () => { throw new Error('runsInside called'); };
assert.deepEqual(decide({}, 10, never), { mode: 'inert' });
Expand Down
Loading