Repository navigation
DDIR: join fragments as a separate, opt-in step from surface text to plain DDP (spike) - #1090
Closed
frankmcsherry wants to merge 1 commit into
Closed
frankmcsherry wants to merge 1 commit into
frankmcsherry wants to merge 1 commit into
Conversation
…plain DDP (spike) The same join fragments as TimelyDataflow#1085, held apart from the core language. `surface::expand` finds each fragment in the text and replaces its source range with plain DDP, leaving every other byte unchanged; the core parser then reads the result. The core lexer, parser, syntax tree, lowering and printers are master-next's, except that `lambda_in` becomes crate-visible so the scanner finds lambda boundaries as the lexer does. `ddir-plan --fragments` and the server's `load … --fragments` opt in; everything else reads plain DDP, and saved expanded text loads without the flag. Errors keep their messages, prefixed with the line and column in the surface text. The expansion is TimelyDataflow#1085's: written join order, key-slot reads, direct first joins, each test as a plain filter at its earliest join, variants through `enlist unweave`, and constant later keys rejected. Eight tests keep the essentials; the emitted plans match TimelyDataflow#1085's byte for byte. Co-Authored-By: GPT-6 Sol (Codex) <noreply@openai.com> Claude-Session: https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The join fragments of #1085, held apart from the core language as a separate, opt-in step from surface text to plain DDP. A draft, for comparison with #1085; both are spikes.
How it differs from #1085
*,[,]tokens, thename(rule,Expr::Fragment, a lowering call, a printer arm (+44 lines)lambda_inbecomes crate-visible (+1/−1)parse/pipe/fragment.rs, 480 lines, extending the core parsersrc/surface.rs, 672 lines: its own scanner, a small printer for the generated text, surface positionsddir-plan --fragments, serverload … --fragments; everything else reads plain DDPHow it works
surface::expand(text)replaces each fragment's source range with plain DDP and leaves every other byte unchanged; comments inside a fragment are copied to the start of its replacement.The expansion is #1085's: written join order, key-slot reads, direct first joins, each test as a plain
filterat its earliest join, variants throughenlist unweave, and constant later keys rejected.Checks
a(x, y) * b(y, z), an experimental surface form (spike) #1085's tests, the eight spike programs, the demux and six placement examples give byte-identical plans to DDIR: join fragments,a(x, y) * b(y, z), an experimental surface form (spike) #1085 at the lowered and final stages (94 plans); the 35 rejected programs keep their errors.cargo test --release -p interactive -p ddir-server: 354 passed, 9 ignored.load --fragments.a(x, y) * b(y, z), an experimental surface form (spike) #1085's suite: dedicated tests for nested variants, the open-Resultconstruction limit, string and list runtime cases, and printer round trips. Their programs still take part in the plan comparison.Not intended to land as is.
Implemented by GPT-6 Sol (Codex); reviewed, tested and opened by Claude Code.
🤖 Generated with Claude Code
https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL