Skip to content

fix(json): end a stream track when a record cannot be written - #3142

Merged
kixelated merged 1 commit into
claude/json-tracks-catalog-api-a13c8cfrom
claude/compassionate-rubin-cb5da3
Aug 28, 2026
Merged

fix(json): end a stream track when a record cannot be written#3142
kixelated merged 1 commit into
claude/json-tracks-catalog-api-a13c8cfrom
claude/compassionate-rubin-cb5da3

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Stacked on #3109 (base is claude/json-tracks-catalog-api-a13c8c, not main). rs/moq-binary/ and rs/moq-mux/src/json.rs only exist there. Merge after #3109.

Summary

  • Root cause. moq_json::stream::Producer::append recovered from a failed frame write by closing the group and letting the next append open a fresh one. A stream track is a single group, so the roll produces a log with a hole in it. Worse, a subscriber joining after the roll starts at the latest group (Subscription::group_start defaults to None) and never sees the records before it, and latency_max defaults to Duration::ZERO so the publisher skips the older group as soon as a newer one exists. A visible failure beats a silently gapped log.
  • Contradicted the spec in two places: the Timeline Track section of drafts/draft-lcurley-moq-hang.md says the timeline track is "a single compressed group that is never rolled", and the Data Tracks section (added in feat(hang): add json and binary data tracks to the catalog #3109) says a publisher that cannot write a payload MUST close the track. feat(hang): add json and binary data tracks to the catalog #3109 fixed this for moq-binary and worked around it at the moq_mux::json::Stream wrapper, but deliberately left moq-json alone.
  • Any append failure now ends the track. A write into the live group aborts it, and so does a record that never encodes: nothing was published either way, but the log is missing a record all the same. Opening the group is the one remaining non-aborting path, and only because append_group fails when the track has already ended.
  • Both failure paths reset the DEFLATE encoder. Without that, the uncommitted Pending's desync latch answers the next append ahead of the track and masks the real reason the log stopped. With it, an append onto an aborted track falls into the open branch, where modify() returns the stored abort error verbatim.
  • Aborting goes through a cloned track handle in both crates. track::Producer::abort consumes self, which tempts an Option<Producer> and an Option<Subscriber> from consume; track::Producer is Clone, so the abort reaches the shared state while the producer keeps its handle. Producer::subscribe already documents that info survives a close/abort and the subscriber surfaces it on first read, which is what tells a late reader the log is truncated. This also removes the Option<Subscriber> that feat(hang): add json and binary data tracks to the catalog #3109 introduced in moq_binary::stream, so both crates now match every other producer in the workspace.
  • Inner::finish finalizes the group and the track independently in both crates, so an error finalizing the group can't leave the track open (the bug fixed for moq-binary in 1664399).
  • moq_mux::json::Stream::append drops its workaround and simply retires the catalog entry, which is now correct for every error rather than only the ones that reached the wire.

Callers cope: container::Producer::write already drops the Recorder on the first record() error and carries on, so terminal-vs-recoverable changes nothing for the timeline. timeline::Producer::finish() after an abort returns the abort error rather than panicking, since its unreachable! arm only covers non-Net errors.

One imprecision worth flagging: an encode failure aborts with moq_net::Error::Cancel, whose doc says "Not a failure". Delivered through abort the consumer still gets an Err rather than a clean end, so the behavior is right, but no variant means "the publisher could not encode this record". Adding one means touching moq_net::Error, a #[non_exhaustive] public enum in a core crate, which felt out of scope here.

Public API changes

No new, renamed, removed, or signature-changed pub items. Two behavior changes to published surfaces:

  • moq_json::stream::Producer::append (moq-json 0.3.6): a failure is now terminal for the track instead of rolling a group. Not a semver break under CONTRIBUTING's rule, which reserves dev for a renamed/removed/signature-changed item in the listed crates.
  • moq_binary::stream::Producer::consume and moq_mux::{json,binary}::Stream::consume revert from Option<moq_net::track::Subscriber> to moq_net::track::Subscriber. Both are unreleased, introduced by feat(hang): add json and binary data tracks to the catalog #3109.

Cross-package sync

  • No draft change. drafts/draft-lcurley-moq-hang.md already specifies exactly this behavior in both the Data Tracks and Timeline Track sections; the code was the thing out of step. just drafts check passes.
  • No wire format change, so no js/ mirror is needed: this is publisher-side failure handling, and the bytes a conforming publisher emits are unchanged.

Test plan

  • just check and just test (1728 passed, up from 1727), just drafts check, all green.
  • New/updated in rs/moq-json/src/stream/mod.rs: a_failed_write_aborts_the_track (a subscriber must see an error, not the clean end a completed log looks like), a_failed_write_ends_the_track (the retry reports the abort rather than Error::Desync, and a subscriber taken after the abort still surfaces it), appending_after_finish_fails_without_aborting (a clean finish must stay readable), and a_rejected_record_does_not_open_a_group extended to assert the track ends.
  • New in rs/moq-mux/src/json.rs: an_unserializable_record_ends_the_track_and_the_entry, which fails without the encode-path fix because the second append used to succeed on a track whose catalog entry had already been retired.

(Written by Claude Opus 5)

`moq_json::stream::Producer::append` recovered from a failed frame write by
closing the group and letting the next append open a fresh one. That contradicts
the format: a `stream` is a single group, so the roll yields a log with a hole in
it, and a subscriber joining afterwards starts at the latest group and never sees
the records before it.

Any failure now ends the track, matching `moq_binary::stream::Producer`. A write
into the live group aborts it, and so does a record that never encodes: nothing
was published either way, but the log is missing a record all the same. Opening
the group stays the one recoverable case, and only because a failure there means
the track had already ended. Both write paths reset the encoder, so the desync
latch doesn't answer the next append ahead of the track.

Both crates abort through a cloned track handle rather than consuming their own,
so `consume` keeps returning a plain `Subscriber` the way every other producer in
the workspace does. A subscriber taken after the abort surfaces it on the first
read, which is what tells a late reader the log is truncated.

`moq_mux::json::Stream::append` drops its workaround and simply retires the
catalog entry, which is now correct for every error rather than only the ones
that reached the wire.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kixelated
kixelated merged commit 95ed3fd into claude/json-tracks-catalog-api-a13c8c Aug 28, 2026
2 checks passed
@kixelated
kixelated deleted the claude/compassionate-rubin-cb5da3 branch August 28, 2026 19:16
kixelated added a commit that referenced this pull request Aug 28, 2026
…the decode limit

Follows #3142, which made the Rust producer end the track on a failed record.
The browser producer still rolled: a serialization failure threw before the try
and left the track writable, and a write failure closed only the group and reset
the encoder, so a later append continued in a second group with the rejected
record silently missing. It now mirrors the Rust semantics exactly, including
the distinction that matters: a write failure had a live group and is terminal,
while an `appendGroup` failure published nothing and stays retryable.

Neither language enforced moq-flate's decode cap. A value over 64 MiB can
compress under the 32 MiB group limit, so it published successfully and then
failed for every consumer, since they all decode with the default and neither
crate exposes an override. Both encoders now reject it, which lands the right
semantics for free: a stream's encode failure routes through the abort, while a
snapshot's rejection happens before anything is published, so the previously
published value stands.

`a rejected record does not open a group` asserted the track stayed writable
after a serialization failure, which is no longer true. Renamed and matched to
the Rust test: no group opened, and the track ended.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant