Skip to content

DDIR: a projected reducer keeps its spelling's graph, so equal spellings merge - #1082

Merged
frankmcsherry merged 1 commit into
TimelyDataflow:master-nextfrom
frankmcsherry:ddir-projected-reducer-graphs
Nov 10, 2026
Merged

frankmcsherry merged 1 commit into
TimelyDataflow:master-nextfrom
frankmcsherry:ddir-projected-reducer-graphs

Conversation

@frankmcsherry

Copy link
Copy Markdown
Member

Two reducers that projection narrows to the same spelling now merge in the plan, instead of only after the printed plan is planned again.
Answers were never wrong: the plan kept one extra reduce and its output arrangement, and the emitted DDP was not quite what ran.

Why

A reported program failed ddir-plan --passes none,structural,hoist-invariant,projection --emit-ddp --check-roundtrip with root: item count 6 -> 5.
After projection its plan held two items with the same input and the same printed text, i1_idx | reduce(_ -> ((), 1) enlist): an unread fold-sum and a collect whose list nobody read.
Structural merging runs after each projection round but left them apart; planning the printed text merged them.

Reducer::project (interactive/src/reducers.rs) replaces the source with a readable spelling once it compiles to the same graph at the reducer's input type, but kept the composed graph.
That graph still holds the original reducer's work, and Lambda compares and hashes by graph, which is what structural merging keys on.
For the pair above, the spellings and the compiled graphs were equal, and the stored graphs were not.

With every pass on, dead-code removes the unread reduce, so the reported program passes; a version with both reduces read fails under all too.

What changes

  • A projected reducer keeps the spelling's parsed graph, as planning the printed text would build it. canonical.rs (compose_run) already does this for composed maps.
  • The comment on that step said "Keep the composed graph"; it now says the spelling's graph is kept, and why.
  • composed_spelling_preserves_the_exact_graph asserted the old choice. Renamed composed_spelling_is_its_own_graph, it asserts the spelling's graph, and the same compiled graph as the composition.
  • The minimized case joins REPROS in tests/pass_roundtrip.rs, which round-trips every repro under all 128 pass subsets. It fails without the change.

Tests

  • cargo test --release -p interactive -p ddir-server: 334 passed, 0 failed, 9 ignored; no warnings. No expected plan in tests/plan_pairs changed.
  • DIFFTEST_SEED=0 DIFFTEST_CASES=1000, all oracles: 1,000 of 1,000 pass.

Diagnosed, implemented and tested by Claude Code.

🤖 Generated with Claude Code

https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL

…ngs merge

`Reducer::project` replaced a projected reducer's source with a readable spelling, once that spelling compiled to the same graph at its input type, but kept the composed graph.
That graph still holds the original reducer's work, and lambdas compare by graph, so two reducers projected to the same spelling stayed apart in the plan and merged only when the printed plan was planned again.
A sum and a collect, both masked to `()`, printed as `reduce(_ -> ((), 1) enlist)` over one arrangement and stayed two reduces.

The projected reducer now keeps the spelling's parsed graph, as `canonical.rs` already does for composed maps.
The test that asserted the composed graph now asserts the spelling's graph, and the same compiled graph as the composition.
The minimized case joins the round-trip repros, checked under every pass subset.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL
@frankmcsherry
frankmcsherry merged commit bdfe924 into TimelyDataflow:master-next Nov 10, 2026
7 checks passed
@frankmcsherry
frankmcsherry deleted the ddir-projected-reducer-graphs branch November 10, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant