[frankmcsherry]

A spike, as a draft: a join fragment syntax for DDIR that expands to the joins, filters and arrangements a hand-written plan would use, with each test at the same point in the joins.
Fragments are an experimental surface form, not part of the plan language or a stable interface: the parser expands each one into plain DDP as it reads it, so no syntax tree, later stage or emitted plan contains one, not even the parsed view. They may change or be removed.
Three commits on master-next 8b03a29: the fragments (374ad91), expansion at parse time (eb0f63f), and the direct-first-join rule (ab26688). A text-to-text alternative that kept the core untouched (#1090, closed) was compared; a review preferred this structure with expansion at parse time.

The syntax

let trim = trans(src, dst) * labels(src, lbl) * labels(dst, lbl)
         | map(((src, dst, lbl), ()) -> (dst, src));

enum Member = User int | Group int;
let by_user = grants(doc, (User(u), level)) | map(((doc, u, level), ()) -> (u, (doc, level)));
let below   = members(doc, Group(g)) * groups(g, m) | map(((doc, g, m), ()) -> (doc, m));

• An atom names an existing binding (a let, set, input, import or scope path) and gives a pattern for its key and its value.
• Shared variables become join keys. Literals, variants (User(u), Type::Variant(x)) and repeated variables become tests. A bare constructor such as None means its empty payload; a name that is not a constructor is a variable.
• A fragment is an ordinary pipeline source. Its rows are every bound variable once, in order of first appearance, as the key, with a unit value. The following map builds the head.
• Weights multiply as in any join. Nothing implies distinct.

The expansion

The plan is fixed by the text; there is no planner.

• Atoms join in written order, each against its binding's own arrangement: one new arrangement per *, of the rows so far; none at the first * when the first two keys are the same unrestricted variables.
• A restricted first key (a literal, a variant, a repeated variable) is tested before its arrangement, so rejected rows are never paired; the survivors keep their key values, so they stay on their workers. With a(Some(x), v) * b(Some(x), w) and 10,000 None keys on each side, reusing a's arrangement and testing afterwards took 2.32 s; testing first takes 4.4 ms.
• Every later key must mention a variable bound by an earlier atom. A constant key (a literal, (), Empty()) would move every incoming row to one key, so it is rejected; a cross join is written as plain DDP.
• Key variables are read from the join's key slot.
• Each literal or equality test is a plain filter as soon as its fields are bound: on the first atom's rows before its first rekey, or right after the join that binds a later field. After a direct first join, the first atom's value tests follow that join. Tested fields leave the stream before the next arrangement.
• A variant binds its payload through enlist unweave in a flatmap, with no head and no tag column. That flatmap can produce the next join's key directly.
• When nothing is left to bind, the output layout goes into the last join's lambda; the author's head map stays its own stage.
• No passes, runtime operators or corgi code change.

For example, message(_, (1, who, parent)) * message(parent, (_, to, _)) filters comments before rekeying by parent:

let out_rows = message | filter((k, v) -> (v.0, 1) eq) | map((k, v) -> (v.2, (v.1)));
let out_idx  = out_rows | arrange;
let out_join = out_idx | join(message_idx, … -> ((who, parent, to), ()));

and members(doc, Group(g)) * groups(g, m) binds and rekeys in one step:

let below_rows = members | flatmap((k, v) -> ((k), v enlist unweave .2) cap_list map (((doc), g) -> (g, (doc))));
let below_join = below_rows | arrange | join(groups_idx, …);

Evidence

Eight programs were rewritten with fragments: SCC, access control, LDBC IC3, IS6, BI11, BI14 and IC13, and the transaction resolver.

• They build the same joins and arrangements as the hand-written originals.
• Across their 38 fragments and the demux, no test runs later in the joins than in the original. IC3's two country tests follow their own place joins; BI14's message-kind test runs before the parent-key arrangement.
• Answers match: SCC's expected output, the access grant/block session, the authored-plan checker on its fixture for the five LDBC plans, and the resolver's rollback and delta.
• cargo test --release -p interactive passes: 337 passed, 9 ignored, including 26 fragment tests that pin each placement, check each error message, and run a fragment over its own feedback variable inside a loop.
• A critical review fuzzed about 70,000 random fragments against a brute-force evaluator, with all passes, none, and inside a loop scope, with no mismatch; its correctness findings (a bare None bound as a variable, nested []) are fixed.

What remains

• The head map. The author's map after a fragment is a separate stage, where hand-written plans often put the head in the last join's lambda. In 30 of the 39 fragments that is one more linear stage after the last join than the original (8 tie, and the demux has one fewer); there is no fusion in this draft.
• Printing. Join lambdas built by composition print as nested let _call0 = … in bindings.
• Wide heads. The trailing map lists every variable in order; with seven or nine variables that is positional again.
• One construction limit. A lookup key built with an open built-in constructor, such as Ok(x), Err(e) or None(), fails because corgi's inject needs the inactive lane's shape; only Some(x) works. A test records it.
• Nested []. An empty list inside a list literal needs its element shape from a sibling constant; [[]] alone is a clear DDIR error.

Not intended to land as is.

Implemented by GPT-6 Sol (Codex), with earlier rebase and test additions by Claude Code; reviewed, tested and opened by Claude Code.

🤖 Generated with Claude Code

https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL