Skip to content

Participant portal: indexDB outbox that fully persists audio on device down to 1-second intervals - #1264

Draft
patcon wants to merge 13 commits into
refactor/portal-cleanupfrom
feat/frontend-audio-outbox
Draft

patcon wants to merge 13 commits into
refactor/portal-cleanupfrom
feat/frontend-audio-outbox

Conversation

@patcon

@patcon patcon commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Real human words: this add an indexDB outbox that stores raw chunks of audio, persists them through refreshes and even to the next visit, and then sends them to the site after. this survives page refreshes and phone restarts. This means that it will essentially be impossible to lose participant recording chunks. (it stores it up to 7 days, and sends in the background when someone visits the portal while connected.

This worked for full chunks before (created only at 30s intervals or stop button presses), but it would only keep these in memory and retry for 8 minutes on same page. This will store every 1s "slice" interval, gather them into chunks, and send them not just in the tab session, but after page refresh, or even system restart.

small UX fixes, so it reassures that recording works offline, and tells status of upload parts.

Note: I did make the offline waveform brighter yellow (to connect it more clearly to the yellow call-out message), but I can revert that.

Before After
Screen.Recording.2026-10-14.at.4.15.46.PM.mov
Screen.Recording.2026-10-14.at.4.12.53.PM.mov

Still working on this, as I'd like to have assurances down to 1 second that no audio will ever be lost from a phone.


Stacked on #1259. Review that one first; this PR's diff is only the 12 commits on top of it. When #1259 merges, GitHub retargets this one.

Draft: phases 1–2 of the audio outbox. The recorder doesn't write 1s slices yet (T6), so the crash-recovery code is in place but has nothing to recover until then.

A participant's recorded audio now stays on their device, in IndexedDB, until the server has confirmed it. Before, every chunk lived only in memory: React Query gave up after 20 upload / 5 confirm retries, and Finish waited 30s and then finished anyway (portal_uploads_incomplete_on_finish). A long offline stretch or a reload lost whatever hadn't been sent.

The outbox

  • Store (outbox/outboxStore.ts): chunks as ArrayBuffer (older iOS Safari loses Blobs in IndexedDB), plus 1s slices of the chunk being recorded, sealed into one chunk in a single transaction.
  • Drain loop (outbox/drainOutbox.ts): sends in order within a conversation, with backoff per conversation (2s doubling to 60s, no limit), so one failing conversation never holds up another. 403/404/410 drop the item; a chunk S3 already has is only re-confirmed; anything over 7 days is purged.
  • Crash recovery: at start, a slice group whose recording lock no tab holds is sealed into a recovered chunk. Live once T6 writes slices.
  • Provider (outbox/OutboxProvider.tsx, mounted in ParticipantLayout): one tab drains under a Web Lock. It wakes on new chunks, on coming back online (which skips the backoff), when the tab becomes visible, and when other tabs post on a BroadcastChannel. If IndexedDB can't be used, chunks fall back to the old in-memory upload.

Finish

  • Finish stops the recorder, waits for its last chunk to be handed over (no more 100ms grace), then waits on outbox.settled() with no timeout, then calls finish().
  • No finish item in the queue. A first cut queued the finish behind the chunks (54d08cbb); 2ae4a5d5 takes that back out. The backend already finishes abandoned conversations: conversations.finish-idle runs every 2 minutes and finishes anything with no chunk in the last 5 minutes. A chunk that lands on a finalized conversation reopens it. So if the tab closes mid-wait, the audio still sends later and the sweep finishes the conversation.
  • Resume stays enabled while finishing and calls the finish off. Finish (and Skip on the verify reminder) is disabled while offline.
  • Reconnect after an interruption no longer waits on uploads; queued audio sends after the reload.

What participants see

  • Pause / finish modal: one progress line, counted over the whole page session.
    • "Uploading audio: 7 of 8 parts sent" (spinner)
    • "Awaiting connection: 7 of 8 parts sent" (spinner), under a yellow banner: "Connection lost. Your audio will continue to be saved on device."
    • "Upload complete: 8 of 8 parts sent" (check), which stays up rather than flashing by.
  • Recording screen banners: the "Something went wrong… try refreshing the page" banner is gone, since a refresh never helps now. Offline gets "You seem to be offline, but keep going…". If the server stops answering while the device is online, a separate banner says so. Text mode keeps its plain offline warning, since typed text isn't saved on the device.
  • Meter: a connection warning draws in amber (#e9bd5c, a new warningMark role) rather than the dark brown meant for text.

Telemetry

  • New: portal_outbox_enqueued, portal_outbox_delivered, portal_outbox_dropped, portal_outbox_recovered and portal_outbox_store_failed.
  • Replacing portal_uploads_incomplete_on_finish:
    • portal_finish_slow fires after 30s of waiting, the old give-up point.
    • portal_finish_cancelled fires when Resume calls a finish off.
    • conversation_finished gains finish_wait_ms and pending_uploads_at_start.
    • portal_abandoned also fires when the tab closes mid-finish.
  • Gap: a conversation the idle sweep finishes has no conversation_finished event.

Testing

  • src/portal: 18 files, 168 tests passing; tsc clean; Biome adds no new findings.
  • Store and drain loop: tested against fake-indexeddb (new devDependency), with a fake API and a fake clock.
  • New tests checked against the old code: a session-wide count test was dropped after it passed against the old code anyway. React batching hides the double count it was meant to catch.
  • Not run: the full frontend suite, pnpm build, check-portal-bundle.mjs, and a manual pass in Chrome or on iOS Safari.

Not in this PR

  • T6: the recorder writes 1s slices.
  • T7: crash recovery end to end on Chrome and iOS Safari.
  • T8: cleanup and device pass. enqueueFinish, the drainer's finish item type and ConnectionHealthStatus.tsx are unused now.
  • Translations: the new strings aren't translated outside English yet.
  • Backend follow-up: confirm-upload creates a new row per confirm, so a retried confirm duplicates the chunk. React Query already retried confirms, so this isn't new. The fix is one line: pass chunkId.
  • Idle sweep and offline backlogs: the sweep goes by each chunk's client timestamp, so a backlog sent after a long offline stretch can be finished and reopened between uploads. That costs extra merges and summaries, but no audio is lost.

🤖 Generated with Claude Code (code and ~850 words of PR description from ~500 words of human prompts across this session)

patcon and others added 12 commits October 14, 2026 02:43
Keeps the participant's audio on the device until the server has it:
1s slices of the chunk being recorded, sealed atomically into a queued
chunk, plus queued finishes. Audio is stored as ArrayBuffer, since older
iOS Safari loses Blobs in IndexedDB. Not wired in yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sends queued chunks and finishes one at a time: in order within a
conversation, with backoff per conversation (2s doubling to 60s, no
limit) so one that keeps failing never holds up another's audio. A
finish goes once it's its conversation's oldest item. A chunk S3 already
has is only re-confirmed; a 400 on confirm uploads it again; 403/404/410
drop the item; anything over 7 days is purged on start. Not wired in yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On start, the drainer seals any slice group whose recording lock no tab
holds into a recovered chunk, so a crash loses at most the last second.
Groups under 1KB (an MP4 header and nothing else) or started over 7
days ago are discarded and reported. Without Web Locks every group is
left alone, since a live recording would look the same as a dead one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The recording screen now hands each chunk to an OutboxProvider mounted
in the participant layout, which keeps it in IndexedDB and sends it from
whichever portal page is open: one tab drains under a Web Lock, woken by
new chunks, coming back online (which skips the backoff), the tab
becoming visible, and other tabs over a BroadcastChannel. If the store
can't be used, chunks fall back to the old in-memory upload.

A chunk counts as queued from the moment it's handed over, so Finish
and Reconnect can't slip past one still being written. Their 30s wait
now races outbox.settled(); usePendingUploads is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finish now stops the recorder, waits for its last chunk to be handed over,
then queues a finish behind the conversation's chunks and stays on the
modal until the server has it. The modal shows N of M chunks sent, says
when it's waiting for a connection, and Cancel returns to the paused
choice. Reconnect no longer waits on uploads: queued audio is on the
device and sends after the reload.

The 30s give-up and portal_uploads_incomplete_on_finish go away. In their
place: portal_finish_slow after 30s, portal_finish_cancelled, wait time on
conversation_finished, and portal_abandoned when the tab closes mid-finish.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drops the finish item from the outbox. The backend already finishes
abandoned conversations: the conversations.finish-idle job (every 2 min)
finishes any conversation with no chunk in the last 5 min, and a chunk
landing on a finalized conversation reopens it. So the queued finish and
its plumbing (abort and take-back, finishWrites, finish counts, the
no-store path) weren't needed.

Finish now waits on outbox.settled() with no timeout, then calls finish()
directly, as before the outbox. If the tab closes mid-wait, the audio
still sends later and the idle sweep finishes the conversation.

The Cancel/Resume swap goes too: Resume stays enabled while finishing and
calls the finish off. The recorder's stop order, N of M progress, offline
line and telemetry stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The meter used --app-warning, which in light mode is the dark brown
meant for text, so a connection warning read as brown rather than
yellow. A new warningMark role (--app-warning-mark) holds the amber
dark mode already uses for warning text, #e9bd5c, for marks like the
meter. The existing fill (the same brown) and tint (a pale background,
too faint for thin lines) don't suit a graphic. Dark mode is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Offline, the recording screen showed the Wi-Fi warning and, as the
liveness pings failed too, "Something went wrong... try refreshing the
page". Now that audio waits on the device until the server has it, a
refresh never helps, so that banner goes.

In audio mode there are now two separate banners:
- Offline: "You seem to be offline, but keep going: your audio is saved
  on this device and will send once you're back online."
- Online but pings failing: "We're having trouble reaching our server,
  but keep going: ..." It only shows when online, since offline already
  explains the failed pings.

Text mode keeps its plain offline warning: typed text isn't kept on the
device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pause and finish modals now show one of three lines, counted over the
whole page session rather than since the queue was last empty (which
showed "0 of 1" for the one chunk sent at pause):
- online, parts waiting: spinner, "Uploading audio: X of Y parts sent"
- offline, parts waiting: Wi-Fi off, "Awaiting connection: X of Y parts sent"
- all sent: check, "Upload complete: Y of Y parts sent", which stays up
  rather than flashing by

The provider keeps the counts: every stored item it has seen since the
page loaded (by seq), plus chunks on their way in and direct uploads that
landed. useOutboxProgress goes; the screen reads outbox.progress(). A
chunk moving from memory into the store now updates the counts in one
step, so it's never counted twice. The separate "Waiting for a
connection" line goes, as the progress line now says it. The uploading
message gets a new id, as extract kept the old id's stale English text.

Finish, and Skip on the verify reminder, are disabled while offline: the
audio can't be sent, so finishing can't either. Resume still works, and a
finish already waiting carries on through a lost connection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anch

participant.button.stop.cancel_finish, participant.modal.offline_waiting
and participant.modal.uploading_progress were added and dropped within
this branch, so main never had them and their obsolete entries are noise.
Obsolete entries that came from main stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Offline, the pause and finish modal now shows a yellow banner with the
Wi-Fi-off icon: "Connection lost. Your audio will continue to be saved
on device." It shows whenever the device is offline, even with every
part sent, as it's also why Finish is greyed out.

The progress line goes back to a spinner for "Awaiting connection", like
"Uploading audio"; only "Upload complete" gets the check. The spinner
drops from Mantine's sm (22px) to 16px, the size of the check's drawn
circle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 16px spinner had no padding, while the check icon draws its circle
inside a 20px box, so text after the spinner sat 2px closer. A 2px side
margin gives both the same footprint, so the text lines up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@patcon patcon changed the title Participant portal: audio waits on the device until the server has it Participant portal: indexDB outbox that fully persists audio on device down to 1-second intervals Oct 14, 2026
Stacked on #1264. Once recording has started, a failed conversation
refetch keeps the recorder and shows #1264's server banner instead of
the error page, since the audio waits in the outbox; 403/404/410 still
end the session. ECHO-1018.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants