feat(protocol,electron): export the pixel video as MP4 instead of WebM

docs/design/backend-beta-readiness.md

@@ -205,8 +205,9 @@ Found by the same audit; each is a change in this repo:
   allowed-headers list; Firefox is expected to fail the preflight.
 - An unrecognised OS is sent as `platform.type: ''`, which the session endpoint rejects as missing
   (permanent); send `'unknown'`.
-- Electron video is WebM, but the manifest carries no codec/format attributes, so the worker and
-  viewer label it `video/mp4`.
+- ~~Electron video is WebM, but the manifest carries no codec/format attributes, so the worker and
+  viewer label it `video/mp4`.~~ Resolved: the Electron pixel video is `video.mp4` (H.264) with `format: "mp4"`, and the SDK drops
+  (and reports) any video entry whose bytes are not MP4, so a WebM can no longer be labelled MP4 (`docs/design/electron.md` §5).
 - A cross-origin "Script error." with no filename sends zero frames, so each occurrence is its own
   issue.
 - The performance uploader sends no `x-bugsee-internal` header and ignores `{ok:false}`.

docs/design/electron.md

@@ -151,20 +151,25 @@ backend symbolicates. App native-module symbols upload at build time via `bugsee
   (renderer replay is on by default, E1/E3) and needs no slice; this is the OPT-IN pixel path, built as a report-time snapshot
   source (the CPU-profile pattern — pulled at report assembly, never streamed). Landed across three packages:
   - **protocol**: the mobile-canonical `'video'` FileType (now also JS-emitted) + `DEFAULT_FILENAMES.video =
-    'video.webm'`.
+    'video.mp4'`.
   - **node**: `launchCore` forwards `reportSnapshots` + `fileEncoders` from the launch options into `createClient`
     (concatenated AFTER the internal profiling snapshot) — the generic extension seam for an async-produced
     report-time artifact + its binary encoder.
   - **electron**: two pluggable `VideoCaptureSource`s behind injected seams — `createCapturePageVideoSource` (MAIN,
     periodic `webContents.capturePage()` → bounded frame ring → injected muxer; own-window, no TCC) and
-    `createMediaRecorderVideoSource` (RENDERER, `MediaRecorder` over a MediaStream → webm). **Per the user, the two
+    `createMediaRecorderVideoSource` (RENDERER, `MediaRecorder` over a MediaStream → MP4, H.264). **Per the user, the two
     combine**: capturePage supplies frames (no Screen-Recording permission) that a canvas→`MediaRecorder` encodes to
-    webm — composed into ONE source by the adapter, so `createPixelVideoController` stays source-agnostic. The
+    MP4 — composed into ONE source by the adapter, so `createPixelVideoController` stays source-agnostic. The
     controller is permission-gated (`hasPermission` TCC seam → inert when denied) and exposes the `ReportSnapshotSource`;
     `encodePixelVideo` is the `video` file encoder. `launchMain` accepts `video: { source, hasPermission? }`, builds the
     controller, forwards its snapshot + encoder into the node launch, starts it (async permission), and stops it on
-    `client.stop`. The real Electron/DOM capturers + webm muxer are the app-/adapter-supplied injected seams (no
-    electron/DOM dependency in the package). The e2e proves an opt-in `video.webm` lands in the SAME merged bundle.
+    `client.stop`. The real Electron/DOM capturers + MP4 muxer are the app-/adapter-supplied injected seams (no
+    electron/DOM dependency in the package). The e2e proves an opt-in `video.mp4` lands in the SAME merged bundle.
+    **Contract (breaking for a host that supplied WebM):** the member is `video.mp4` with `format: "mp4"` taken from the file TYPE,
+    and the worker skips its transcode for an MP4, so the bytes MUST be MP4 (H.264): the `encode` seam and the `MediaRecorder` `mimeType`
+    must produce it. The pixel-video controller checks the `ftyp` box of every video entry and DROPS one that is not MP4 (a WebM is named
+    in the message), reporting through `onError`; the bundle still builds, without the video. Chromium's `MediaRecorder` default is WebM,
+    so a host must ask for an MP4 mime type explicitly.
 
 ## 6. Open questions / to confirm at slice time
 - **Minidump upload path** — `crashReporter.submitURL` posting directly to a Bugsee collector minidump endpoint vs

docs/design/per-window-replay.md

@@ -25,7 +25,7 @@ base across windows, and a byte-cap reset restarts every recorder. The replay of
 | `CaptureDataEntry` | `lane?: string` copied from the stored record by the reifier. |
 | Export | members are grouped by `(type, lane)`. The first lane (by first record) keeps the canonical name (`replay.bin`); later lanes get `replay.<n>.bin` with a manifest entry `{type: 'replay', attrs: {window: <lane>}}`. Each member has its own stream encoder session (orphan trim and clamp are per window). A viewer that reads only `replay.bin` still gets a coherent single-window replay. |
 | Electron main | tags each renderer record with its window id as `lane`; `onVisualReset(lane)` restarts ONLY that window's recorder (`checkout` carries the window id; the gate paces per lane). |
-| Pixel video | separate slice: main composes the windows of a display with `getBounds()` and z-order into one frame per tick; one `video` member per display (`video.<n>.webm`, attrs `{display: id}`). |
+| Pixel video | separate slice: main composes the windows of a display with `getBounds()` and z-order into one frame per tick; one `video` member per display (`video.<n>.mp4`, attrs `{display: id}`). |
 
 ## Slices
 

packages/core/src/bundle-assembler.test.ts

@@ -515,15 +515,15 @@ describe('assembleBundle — binary file encoders (replay)', () => {
 });
 
 describe('assembleBundle — the video entry names its container', () => {
-  // `video.webm` is a WebM file, but a manifest entry with no attrs is read by the worker and the viewer as
-  // the default (an h264 MP4): the worker skipped the transcode a WebM needs and the viewer labelled it
-  // `video/mp4`. The format is stated on the entry, taken from the file's own extension.
+  // The container is stated on the manifest entry, taken from the file's own extension, rather than left to a reader default
+  // (an entry with no attrs is read by the worker and the viewer as an h264 MP4, and a WebM mislabelled that way skipped the
+  // transcode it needs). `video.mp4` is the mobile-canonical name, so the default needs no transcode.
   const manifestOf = (bundle: ReturnType<typeof assembleBundle>) =>
     JSON.parse(strFromU8(unzipSync(bundle.body)['manifest.json'] as Uint8Array)) as {
       files: Array<{ filename: string; type: string; attrs?: Record<string, unknown> }>;
     };
 
-  it('puts format "webm" on the video entry', () => {
+  it('puts format "mp4" on the video entry', () => {
     const map = new Map<FileType, CaptureDataEntry[]>([
       ['video', [entry('video', 1, new Uint8Array([1, 2, 3]))]],
     ]);
@@ -534,8 +534,8 @@ describe('assembleBundle — the video entry names its container', () => {
     );
 
     const video = manifestOf(bundle).files.find((f) => f.type === 'video');
-    expect(video?.filename).toBe('video.webm');
-    expect(video?.attrs).toEqual({ format: 'webm' });
+    expect(video?.filename).toBe('video.mp4');
+    expect(video?.attrs).toEqual({ format: 'mp4' });
   });
 
   it('leaves every other entry without attrs', () => {

packages/core/src/bundle-stream-assembler.test.ts

@@ -1213,7 +1213,7 @@ describe('assembleBundleStream — one stream-encoded member per visual source l
       files: Array<{ type: string; attrs?: Record<string, unknown> }>;
     };
     expect(manifest.files.find((f) => f.type === 'video')?.attrs).toEqual({
-      format: 'webm',
+      format: 'mp4',
       displayId: 1,
     });
   });
@@ -1244,27 +1244,27 @@ describe('assembleBundleStream — one member per display for a buffered-encoder
 
   it('writes each display its own video member, encoded on its own, with attrs.displayId', async () => {
     const { files, manifest } = await run([video('0', [1, 2]), video('1', [3, 4])]);
-    expect([...(files['video.webm'] as Uint8Array)]).toEqual([1, 2]);
-    expect([...(files['video.1.webm'] as Uint8Array)]).toEqual([3, 4]);
+    expect([...(files['video.mp4'] as Uint8Array)]).toEqual([1, 2]);
+    expect([...(files['video.1.mp4'] as Uint8Array)]).toEqual([3, 4]);
     expect(manifest.files.filter((f) => f.type === 'video')).toEqual([
-      expect.objectContaining({ filename: 'video.webm', attrs: { format: 'webm', displayId: 0 } }),
+      expect.objectContaining({ filename: 'video.mp4', attrs: { format: 'mp4', displayId: 0 } }),
       expect.objectContaining({
-        filename: 'video.1.webm',
-        attrs: { format: 'webm', displayId: 1 },
+        filename: 'video.1.mp4',
+        attrs: { format: 'mp4', displayId: 1 },
       }),
     ]);
   });
 
   it('an unkeyed video stays the one canonical member with no displayId', async () => {
     const { files, manifest } = await run([video(undefined, [9])]);
-    expect([...(files['video.webm'] as Uint8Array)]).toEqual([9]);
-    expect(manifest.files.find((f) => f.type === 'video')?.attrs).toEqual({ format: 'webm' });
+    expect([...(files['video.mp4'] as Uint8Array)]).toEqual([9]);
+    expect(manifest.files.find((f) => f.type === 'video')?.attrs).toEqual({ format: 'mp4' });
   });
 
   it('a lane that is not a number is recorded as a window, not a display', async () => {
     const { manifest } = await run([video('win-a', [1])]);
     expect(manifest.files.find((f) => f.type === 'video')?.attrs).toEqual({
-      format: 'webm',
+      format: 'mp4',
       window: 'win-a',
     });
   });

packages/electron/src/display-video.integration.test.ts

@@ -22,6 +22,9 @@ const win = (id: number, bounds: ReturnType<typeof rect>, focused = false): Wind
   webContents: { capturePage: async () => ({ toJPEG: () => new Uint8Array([id]) }) },
 });
 
+// A box size, `ftyp`, a brand: the smallest bytes the pixel-video controller accepts as MP4 (it drops anything else).
+const MP4_HEAD = [0, 0, 0, 12, 0x66, 0x74, 0x79, 0x70, 0x69, 0x73, 0x6f, 0x6d];
+
 describe('a two-display session, end to end', () => {
   it('ships one video member per display, each holding only its own display frames, named by attrs.displayId', async () => {
     const windows = [win(1, rect(10, 10, 100, 100), true), win(2, rect(1100, 10, 100, 100))];
@@ -40,7 +43,7 @@ describe('a two-display session, end to end', () => {
         // "compose": the first layer's bytes stand for the composed frame of the display
         compose: async (display, layers) =>
           new Uint8Array([display.id, ...layers.map((l) => l.bytes?.[0] as number)]),
-        encode: (frames) => Uint8Array.from(frames.flatMap((f) => [...f.bytes])),
+        encode: (frames) => Uint8Array.from([...MP4_HEAD, ...frames.flatMap((f) => [...f.bytes])]),
         scheduler: {
           setInterval: (cb) => {
             tick = cb;
@@ -77,16 +80,16 @@ describe('a two-display session, end to end', () => {
     );
     const files = unzipSync(await bundle.body.bytes());
     // display 0 = the focused window's display (native 30, window 1); display 1 = native 70 (window 2)
-    expect([...(files['video.webm'] as Uint8Array)]).toEqual([0, 1]);
-    expect([...(files['video.1.webm'] as Uint8Array)]).toEqual([1, 2]);
+    expect([...(files['video.mp4'] as Uint8Array)]).toEqual([...MP4_HEAD, 0, 1]);
+    expect([...(files['video.1.mp4'] as Uint8Array)]).toEqual([...MP4_HEAD, 1, 2]);
     const manifest = JSON.parse(strFromU8(files['manifest.json'] as Uint8Array)) as {
       files: Array<{ filename: string; type: string; attrs?: Record<string, unknown> }>;
     };
     expect(
       manifest.files.filter((f) => f.type === 'video').map((f) => [f.filename, f.attrs]),
     ).toEqual([
-      ['video.webm', { format: 'webm', displayId: 0 }],
-      ['video.1.webm', { format: 'webm', displayId: 1 }],
+      ['video.mp4', { format: 'mp4', displayId: 0 }],
+      ['video.1.mp4', { format: 'mp4', displayId: 1 }],
     ]);
   });
 });

packages/electron/src/electron-e2e.test.ts

@@ -199,6 +199,9 @@ async function bootRenderer(
   };
 }
 
+/** The start of an MP4 file: a box of size 24 and type `ftyp` (what an H.264 encoder's output begins with). */
+const MP4_FTYP = [0x00, 0x00, 0x00, 0x18, 0x66, 0x74, 0x79, 0x70];
+
 describe('Electron main + 2 renderers converge into one session/bundle', () => {
   it('handshake, native-crash correlation, ONE merged bundle, and stop/flush propagation', async () => {
     const bus = fakeElectronBus();
@@ -207,13 +210,13 @@ describe('Electron main + 2 renderers converge into one session/bundle', () => {
     const crashStarts: CrashReporterStartOptions[] = [];
     const crashReporter = { start: (o: CrashReporterStartOptions) => crashStarts.push(o) };
 
-    // A pixel-video source (fake capturer + encoder) — proves the opt-in path lands a video.webm.
+    // A pixel-video source (fake capturer + encoder) — proves the opt-in path lands a video.mp4.
     const videoSource: VideoCaptureSource = {
       start: () => {},
       stop: () => {},
       clear: () => {},
       snapshot: async (now: number): Promise<CaptureDataEntry[]> => [
-        new CaptureDataEntryBase('video', now, new Uint8Array([0x1a, 0x45, 0xdf, 0xa3])),
+        new CaptureDataEntryBase('video', now, new Uint8Array(MP4_FTYP)),
       ],
     };
 
@@ -263,8 +266,8 @@ describe('Electron main + 2 renderers converge into one session/bundle', () => {
       expect.arrayContaining([{ from: 'renderer-1' }, { from: 'renderer-2' }, { from: 'main' }]),
     );
     // The opt-in pixel video rode the report-snapshot path into the SAME bundle (binary, via the encoder).
-    expect('video.webm' in files).toBe(true);
-    expect([...(files['video.webm'] as Uint8Array)]).toEqual([0x1a, 0x45, 0xdf, 0xa3]);
+    expect('video.mp4' in files).toBe(true);
+    expect([...(files['video.mp4'] as Uint8Array)]).toEqual([...MP4_FTYP]);
 
     // ── Control propagation: flush + stop from the main reach both renderers ──
     await mainClient.flush();
@@ -290,7 +293,7 @@ describe('the main blackout reaches every renderer and the pixel video (privacy)
       stop: () => void videoCalls.push('stop'),
       clear: () => void videoCalls.push('clear'),
       snapshot: async (now: number): Promise<CaptureDataEntry[]> => [
-        new CaptureDataEntryBase('video', now, new Uint8Array([0x1a, 0x45, 0xdf, 0xa3])),
+        new CaptureDataEntryBase('video', now, new Uint8Array(MP4_FTYP)),
       ],
     };
     const mainClient = launchMain('tok', {
@@ -325,13 +328,13 @@ describe('the main blackout reaches every renderer and the pixel video (privacy)
     // A report built DURING the blackout carries no pixel video.
     await mainClient.logException(new Error('during blackout'));
     expect(puts).toHaveLength(1);
-    expect('video.webm' in unzipSync(puts[0] as Uint8Array)).toBe(false);
+    expect('video.mp4' in unzipSync(puts[0] as Uint8Array)).toBe(false);
 
     mainClient.endBlackout();
     for (const r of [r1, r2, r3]) expect(r.blackout.isActive()).toBe(false);
     expect(videoCalls).toEqual(['start', 'stop', 'clear', 'start']);
     await mainClient.logException(new Error('after blackout'));
-    expect('video.webm' in unzipSync(puts[1] as Uint8Array)).toBe(true);
+    expect('video.mp4' in unzipSync(puts[1] as Uint8Array)).toBe(true);
 
     await mainClient.stop();
   });

packages/electron/src/launch-main.test.ts

@@ -524,7 +524,12 @@ describe('launchMain', () => {
       },
       clear() {},
       snapshot: vi.fn(async (now: number) => [
-        new CaptureDataEntryBase('video', now, new Uint8Array([1])),
+        // MP4-shaped (a box size, `ftyp`, a brand): the controller drops any other container
+        new CaptureDataEntryBase(
+          'video',
+          now,
+          new Uint8Array([0, 0, 0, 12, 0x66, 0x74, 0x79, 0x70, 1, 2, 3, 4]),
+        ),
       ]),
     };
     const client = launchMain('tok', {