Skip to content

fix(native): report deeply nested header values instead of crashing - #1015

Merged
kvlonge merged 2 commits into
mainfrom
fix/deep-header-nesting
Oct 8, 2026
Merged

kvlonge merged 2 commits into
mainfrom
fix/deep-header-nesting

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Why

A SQLBuild header whose value nests about 20,000 levels deep crashed discovery with a segmentation fault under every compiler engine: the native header value parser recursed once per level and overflowed the stack. Call syntax nested 231–256 levels also raised RecursionError in Python's projection of the parsed values (CHI-547).

Changes

  • The native header value parser refuses to nest deeper than 256 levels (the YAML loader's limit). Lists, sets, maps, typed constants and hook lists count one level; a plain call counts the two maps it builds, and hook entries count their arguments too, so recursion stays bounded on any thread.
  • Python projects native header values with an explicit stack instead of recursion, so depth no longer depends on the Python recursion limit with unchanged results.
  • Discovery reports the error under each statement's existing header syntax code, naming the file line of the container that is too deep, e.g. MODEL(...) in 'models/orders.sql:3' contains invalid SQLBuild header syntax: values nest deeper than 256 levels, with help flatten the value so it nests at most 256 levels deep; lint suggests the same help. Rust and Python wrappers produce identical text and help.
  • In-memory parse_*_text entry points run on the discovery worker pool.

Verification

  • cargo test --workspace --all-features: depths 128–257, 1k, 20k and 100k per container kind on a 2 MiB thread with a time bound.
  • Integration: engine parity at 1k/20k/100k; all container kinds at the limit discovered beneath 300 extra Python frames under all engines. E2E sqb compile for lists and for call syntax at 240/256/257/20k under all three engines.
  • Failure corpus cases for MODEL (D002) and CONSTANT calls (D013); make compiler-differential, make compiler-differential-shipped, tests/e2e/scripts/compiler_differential, make test, make check-ci (Rust 1.98.0) and the public-repository scan pass.

@kvlonge
kvlonge enabled auto-merge (squash) October 8, 2026 01:21
The native header value parser recursed once per nesting level, so a
header value nested about 20,000 levels deep overflowed the stack and
crashed discovery under every compiler engine.

The parser now refuses to open more than 256 nested containers, the
same limit the YAML loader uses. Discovery reports it under each
statement's existing header syntax code, naming the file line of the
container that is too deep, with help on how to fix it. The in-memory
parse entry points run on the discovery worker pool.
Python projected native header values recursively, spending about four
frames per call level, so call syntax nested 231 to 256 levels deep
raised RecursionError in every engine. The projection now walks values
with an explicit stack, and a plain call counts as the two levels it
builds (a hook's arguments count as one more), so the 256-level limit
measures the values discovery actually produces. Lint suggests the same
flatten help for a header nested too deeply.
@kvlonge
kvlonge force-pushed the fix/deep-header-nesting branch from c72c027 to ba15cf7 Compare October 8, 2026 01:58
@kvlonge
kvlonge merged commit eca2cec into main Oct 8, 2026
37 checks passed
@kvlonge
kvlonge deleted the fix/deep-header-nesting branch October 8, 2026 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant