[iamfatness]

Closes #1030

On the 2026-10-23 live show (beta-2026-10-23-1072a50), a Zoom ISO participant left during a recording. His ISO kept muxing audio silence with no video. Media Foundation's CMFSinkWriter::ThrottleStream blocked that WriteSample forever. When the shell re-formed the session, closeWriters() joined every RecordingTrackWorker with an unbounded thread_.join(), and the AsyncEncoderSink file-control lane hung for the rest of the show. A second Record press failed, and the core missed the shell's close grace. Thread 81 and thread 428 in the dump show exactly these two stacks.

The four parts

1. Throttling off. Mp4Writer::createFragmentedSink is the only sink-writer creation path. Program.mp4 and every ISO file use it. It now sets MF_SINK_WRITER_DISABLE_THROTTLING = TRUE, and a failed set fails the open loudly. The adapter already paces each file with its own bounded queue on the shared capture epoch. Throttling did nothing except turn a stalled sibling stream into a block with no end. Program is covered by the same change.
2. No silence-fill after a participant departs. During audio gather (under coreMutex), the core compares the selected zoom: ISO ids with the engine's floor roster, the same signal the shell logs as roster-drop. The comparison is core::departedIsoSources in core/IsoDeparture.h, and it sets the new IsoSourceAudio::departed. The MF sink neither queues nor writes a departed tick that has no PCM, and it logs the departure once. The stem stops at its last real media, and the existing tail reconcile aligns it with the last video frame, so the file finalizes with its real duration. An empty roster counts as "unknown", never as "everyone left". If the participant returns, the stem resumes and isoAudioAdvance fills the gap with silence at that point. A participant with the camera off who is still in the meeting is not treated as departed.
3. Bounded joins. RecordingTrackWorker gains joinFor(bound), abandon() and exited():
   • abandon() discards queued work (it may reference the sink) and detaches the thread. If the blocked write ever returns, the finalize still runs late.
   • closeWriters() gives all joins one shared deadline: 4000 ms after the last close, matching AsyncEncoderSink's finalize grace. COREVIDEO_RECORDING_TRACK_JOIN_BOUND_MS (50..60000) overrides it for drills and tests.
   • An ISO writer that misses the deadline is logged [iso-writer] ABANDONED and named in the ISO status warning and in recordingError. Its entry is parked and released once its thread exits. MFShutdown is skipped while any abandoned writer is still running.
   • A Program writer that misses the deadline makes the next Start throw a named error until its thread exits, and recording then recovers by itself. The new IEncoderSink::mustOutliveTeardown() makes AsyncEncoderSink::State leak the sink instead of destroying it under a live thread. The same bound covers the destructor path at core shutdown.
4. Accurate loss text. The recording warning no longer blames "encoder load". It now says a file writer's queue was full because that writer was blocked or slower than real time. The per-ISO loss text gives the same cause for that file.

The recording stall QA knob (COREVIDEO_QA_RECORDING_STALL_MS) now allows up to 10 s instead of 2 s, so a drill can hold a writer past the bound. validate-iso-record.mjs accepts the same range.

Tests (Windows dev core, npm run build:native-dev, Release)

• Full native suite with COREVIDEO_CAPTURE_TESTS=1: 1888 passed, 0 failed (after the review rounds).
• New, compiled everywhere: RecordingTrackWorker.BoundedJoin* (3 tests: healthy join; blocked write abandoned and finalized late, with discarded work never run; writer stuck in finalize) and IsoDeparture.OnlyZoomSourcesMissingFromAKnownRosterHaveDeparted.
• New, Windows MF only, behind the !COREVIDEO_STUB && COREVIDEO_ENABLE_DEV_ADAPTERS && COREVIDEO_WITH_MF_ENCODER guard:
  • MediaFoundationIsoWhoseVideoStopsWhileAudioContinuesClosesWithinBound: 1 s live, then 30 s of silence ticks with no video. Stop must finish in under 3.5 s and must not abandon anything. Red with the throttling attribute removed: the writer blocked in the silence write and was abandoned at 4 s.
  • MediaFoundationDepartedIsoParticipantIsNotSilenceFilled: the stem stays under 3 s instead of 31 s.
  • MediaFoundationStopAbandonsAnIsoWriterThatMissesTheJoinBound: the stall QA holds the ISO for 8 s against a 3 s bound. Stop returns, the file is named, the next take runs, and the late writer finalizes and is released.
  • MediaFoundationAbandonedProgramWriterRefusesThenRecovers: Program is held past the bound. Start is refused with a named error, then recording recovers after the late finalize.

Review round (fix-first)

• Blocker 1, engine mutex under coreMutex. The roster read moved out of gatherAudioOutputWork. The audio worker now reads the roster ids before taking coreMutex (MediaCore::readZoomRosterIdsUnlocked through ZoomEngineRuntime::rosterIdsIfConfigured), the same pattern as pollZoomAudioUnlocked. ZoomEngineRuntime::mutex_ is now an EngineMutex (modules/EngineMutexGuardrail.h), and the gather opens a ForbidEngineMutexScope that counts every acquisition inside it. The guardrail immediately found a violation that predates this PR: configured() took the engine mutex on every gather, via observeZoomGuestAvEpoch. configured() now reads a lock-free atomic mirror. The new test MediaCoreIsoDeparture.RosterDropMarksTheIsoDepartedThroughTheGatherWithoutTheEngineMutex sends a real roster drop and return through the gather and asserts the counter does not move.
• Blocker 2, back-pressure. The reviewer was right. With throttling off and nothing else, 30 s of audio ahead of a stopped video stream left 4.7 MB queued in the sink writer and zero bytes on disk, because the fMP4 sink holds audio for interleave.
  • First fix, IMFSinkWriter::SendStreamTick: replaced. The follow-up test MediaFoundationIsoFrameStampedBehindTheLastHeldFrame showed it corrupted the timeline. File PTS stayed monotonic, but the sink lays video out by accumulated sample durations and the tick collapsed the 28 s gap, so a camera that resumed at 31 s was muxed at 2.8 s against its audio.
  • Final fix, Mp4Writer::holdStalledVideo. While video is more than 1 s behind the audio, the writer re-submits the last real picture once per second of media, trailing the audio by 0.5 s. Held frames count in the mux frame count, never in framesWritten. The variable-frame-rate logic gives each sample its exact duration, so the gap is kept.
  • Resume behaviour. A resumed frame stamped at or behind the last hold takes that hold's slot (the pending held picture is discarded). It is never dropped, PTS stay monotonic, and the following frames land on their own capture times.
  • Measured over the 30 s audio-only phase: +472,886 bytes reached disk before Stop (about 57 MB per hour), queued_bytes was 0 at Finalize, and the sink writer's own queue peaked at 7,680 bytes for the stalled file. The Finalize drain line now reports held_frames (28 to 29) and stall_max_queued_bytes.
  • Evidence logging. [recording-writer] queued_bytes now also fires from the audio path (via=audio).
• Nits:
  • A late Program finalize now settles the take as stopped with a "finalized late" warning, and start() recovery adopts that state.
  • Every abandoned ISO is named in the error, including when Program was also abandoned.
  • The MediaCore roster test above covers the gather path.
  • Join timings are logged. validate-iso-record.mjs --sources 8 --seconds 20 --allow-capacity-warning passed. Every ISO joined at 38 ms of the 4 s deadline and Program at 20 to 38 ms, because the ISOs finalize during the 8 x 100 ms close stagger. The 4 s bound was kept. Without --allow-capacity-warning the run fails only on the known one-ISO software-spill admission note.
• A GPU timing test (D3DProgramBuffer.MultiviewProgramPixelsAdvanceAtFixedConsumerPhases) failed once in one full run. It passed 3/3 when rerun alone, and the next full run was clean; it failed once more in a later full run and was clean again on the run after. It does not touch recording code.

Not verified

• The live wedge was not reproduced. No live meeting was run. Acceptance still needs a real participant to leave during a take: the departed ISO must finalize at its real duration, the re-formed take must start, and Record must work again without a restart.
• Shell close timing was not measured. A healthy close is now bounded by the 4 s join (plus the existing 4 s AsyncEncoderSink grace), so it no longer hangs. The shell's 2000 ms grace can still be shorter than a normal Program finalize; this PR does not change that.
• In the small 640x360 tests, Program's existing Mp4Writer::finalize drain uses its full 2 s, because the NVENC MFT held about 20 of 51 frames after end-of-stream. That behavior predates this PR and is why the default bound is 4 s and not lower.

🤖 Generated with Claude Code