Skip to content

perf(compile): read SQL-test relation markers from the parse tree - #932

Merged
kvlonge merged 1 commit into
mainfrom
perf/reuse-parses-test-planning
Oct 4, 2026
Merged

kvlonge merged 1 commit into
mainfrom
perf/reuse-parses-test-planning

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Why

SQL-test planning converted each parsed model to a streaming serialised form to find relation markers: about 19% of native planning CPU and ~810 lines of collector code. Reusing model-analysis parses instead isn't possible without changing output, because the two phases parse different SQL text (CHI-441 phase 2).

Changes

  • Read FROM/JOIN relation markers directly from the parse tree with a depth-first walk in declared field order, the same order as before. One serialised-but-not-walked field is visited explicitly.
  • Remove the serialisation-based collector and its unreachable error type.
  • Extend the JSON-walk oracle test with parenthesised joins, LATERAL, UPDATE/DELETE/INSERT shapes, and tsql/databricks.

Verification

  • Native planning CPU (base → head): dense 3k 4.27 → 3.57 s, dense 10k 13.62 → 11.48 s; full cold compile CPU at 10k 86.0 → 84.1 s.
  • Planning responses, compile JSON and target artifacts identical; corpus check of 76,315 statements in 7 dialects with 0 oracle mismatches.
  • One independent review; make test, make check-ci.

SQL-test planning found each model's FROM and JOIN marker calls by
streaming the parsed statement through a serializer that reproduced the
original JSON walk. The planner now reads those relations directly from
the parse tree in the same pre-order: each node's FROM markers, then its
JOIN markers, then its children in declared field order. The serializer
could never fail on a parsed statement, so its error path is removed.

The JSON-walk oracle still pins the order, with more relation shapes and
dialects. Native planning CPU drops by about 18 percent on dense
projects; plans and compile output are unchanged.
@kvlonge
kvlonge enabled auto-merge (squash) October 4, 2026 04:27
@kvlonge
kvlonge merged commit d80af81 into main Oct 4, 2026
34 checks passed
@kvlonge
kvlonge deleted the perf/reuse-parses-test-planning branch October 4, 2026 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant