Skip to content

fix(convert): refuse trees too deep to convert, and stop copying || operands - #86

Merged
kvlonge merged 2 commits into
mainfrom
fix/deep-expression-limits-landing
Nov 5, 2026
Merged

kvlonge merged 2 commits into
mainfrom
fix/deep-expression-limits-landing

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Why

A flat chain such as SELECT total + total + … AS x FROM orders passed check but crashed convert: past about 2,600 terms the CLI overflowed its stack and aborted, and a Python host segfaulted, which kills the interpreter. A || chain also took memory growing with the square of its length (about 1 GB at 800 terms), because each level copied its left operand.

Changes

  • Conversion measures the tree's depth iteratively before converting it, and refuses past 512 levels (128 on WebAssembly) with SQ0006 instead of crashing.
  • || converts its operands in place instead of cloning them, so its memory stays linear.
  • AND and OR chains are unaffected: the parser balances them, so they stay shallow.

Verification

  • CLI and Python end-to-end tests cover +, - and || chains past the limit (SQ0006, no crash), a || chain within it, and a long AND chain that converts. Both tests fail on the unfixed code.
  • Measured: a 200,000-term + chain returns SQ0006 at 354 MB peak; || refuses beyond about 509 terms; a 3,000-term AND chain converts.
  • D94 gate result follows as a comment.

…perands

A flat chain such as a + a + ... parsed and checked, but converting it
recursed once per operand and overflowed the stack: the CLI aborted and a
Python host segfaulted. A || chain also deep-copied its left operand at
every level, so memory grew with the square of its length.

- Conversion measures the tree's depth, iteratively, before serialising
  it, and refuses past 512 levels (128 on WebAssembly) with SQ0006.
- || converts its operands in place instead of cloning them.
- CLI and Python tests cover +, - and || chains past the limit, a ||
  chain within it, and a long AND chain, which the parser balances.

kvlonge commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

D94 gate on the exact head 861e0e1 (base main 7850e8d):

  • make check-ci RECORDER=0: exit 0
  • make test RECORDER=0: exit 0
  • Coverage report: byte-identical between base and head
  • sqlquack verify: 8,459 cases, 0 failed, on base and head
  • make verify-recorded: exits 2 identically on base and head. The translation-results step finds 5 stale entries in coverage/translation/results-baseline.json from catalogue r7 (three new forms and the dbt variant form rename). That is a defect in main, independent of this change, and is being fixed separately by re-recording the baseline.
  • Public-repository hygiene scan: clean
  • CI on 861e0e1: green

Landing by fast-forward to main.

@kvlonge
kvlonge merged commit 861e0e1 into main Nov 5, 2026
1 check passed
kvlonge referenced this pull request Nov 6, 2026
…cases

Five statements move from matching to declared_divergence. They are
corrections of main's silent tie conversion, not converter regressions:
main converted different values tied on every order key without
declaring a divergence, which is the wrong value D172 fixes.

- generated/hazards/array_agg__aggregate.toml#node_within_group_combination_0_integer_integer
- generated/hazards/array_agg__aggregate.toml#node_within_group_combination_0_text_text
- generated/hazards/listagg__aggregate.toml#node_within_group_1_combination_0_text_text
- generated/hazards/listagg__aggregate.toml#node_within_group_combination_0_text_literal_text_text
- generated/hazards/listagg__window.toml#node_window_function_2_combination_0_text_literal_text_text

The other 11 entries are additions from the D153c receipts and the new
group_mixed cases: nine statements, rlike.syntax syntax_only to
matching, and to-binary unmeasured to refused. Nothing else moves.
@kvlonge
kvlonge deleted the fix/deep-expression-limits-landing branch November 9, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant