[frankmcsherry]

flatmap now returns complete (key, val) rows, with no implicit positions, so a lambda that keeps zero or one row per input is a filter-and-map, and one that chooses a new key needs no map after it.
A draft, as a step toward smoother pattern matching in join fragments (#1085).

The change

• flatmap((k, v) -> l) takes l : List((key, val)). Each element is a whole output row, key included.
• It no longer keeps the input key and adds (pos, element) to the value. Positions came with the June port (interactive: port the data model from flat [i64] to a Value ADT + Term scalar language #760), so that flatmap and collect round-trip a list. A lambda that wants them writes ((0, l len) range, l) zip.
• With map (one row) and join_function (rows with times and diffs), the per-row linear operations now form a ladder of complete rows.

Run time

• Keys and values come straight from the list's element storage; only times and diffs are repeated.
• When every list holds at most one row, times and diffs are gathered by the surviving input rows.
• When every list holds exactly one row, the original time and diff columns are reused.

Conversion

• 33 .ddp files: 25 written programs and 8 generated snapshots. Of the 25 uses, 18 needed positions and now write range/zip; 7 choose their key in the lambda, removing a following map.
• Also: 10 Rust DDP sites, 2 Python plan generators (snb/rel.py, test_topk.py), explain's backward rules, shape and demand analysis, the language reference and README.
• Plan-pair snapshots change only for IS3, IS7, IC11 one-off and arrange_idem, each line a flatmap conversion.

Checks

• cargo test --release -p interactive passes, including a new tests/flatmap.rs: keeping and dropping, signed diffs, retractions, named variant arms, literal tests, rekeying, and a kernel that fails on dropped data, which shows dropped lanes are never built.
• All 41 authored LDBC plans pass check.py --fixture; the top-k checker passes at one and four workers; all 33 AoC parts and the example .expected outputs match.

Measured

On 3 M rows of (int, (Member, int)) through DDIR's linear-operator path (ns per input row):

	ns/row
demux, map | filter | map	6.17
demux, one flatmap with enlist unweave	4.90
plain filter	2.29
filter written as a flatmap	4.89

The demux is 21% faster as one flatmap; filter stays its own kind.

Implemented by GPT-6 Sol (Codex); reviewed, tested and opened by Claude Code.

🤖 Generated with Claude Code

https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL