Skip to content

feat(semantics): give each output column its expression's byte range - #170

Merged
kvlonge merged 5 commits into
mainfrom
feat/output-expression-range
Nov 11, 2026
Merged

kvlonge merged 5 commits into
mainfrom
feat/output-expression-range

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Why

Column lineage says which columns an output reads and how, but not where the output is defined in the SQL. A UI cannot highlight the expression behind a traced column. Stars also had no source range, so findings falling back to a star had none.

Changes

  • Star.span is a SourceSpan, as on other located nodes: located parses record it (not yet SQLite or SQL Server), equality ignores it, and serialised trees omit it when absent.
  • Scope findings that fall back to the first located column, name or star now point at that star instead of offset 0. Three expectations move: a star modifier naming an unknown column, a USING key one side lacks, and a duplicate CTE name in a query projecting a star.
  • OutputLineage::expression_range: Option<Range<usize>> is the UTF-8 byte range, in the analysed SQL, of the projection item that defines the column. A column a star brings points at the star; one a REPLACE gives a value points at that value, or at the star when the value has no range; outputs of any set operation (UNION, INTERSECT, EXCEPT, BY NAME) have none, and PIVOT pass-through columns carry no inner range (untested guard: no input reached it).
  • Spans count Unicode characters. In ASCII SQL they already are byte offsets; otherwise one pass over the statement converts them, stopping at the last offset needed.

Verification

  • New lineage cases slice the SQL by each range and compare the text: aliased and bare items after multi-byte text, a star over a CTE, a star with EXCLUDE and RENAME, a DuckDB REPLACE value, and UNION, EXCEPT and BY NAME outputs; the EXCEPT case fails without the third commit.
  • cargo test for every syntax crate, sqlquack-parse, sqlquack-lint and sqlquack-semantics passes.
  • make instruction-counts: analysis 241.3M against the 239.7M baseline (+0.7%), parse unchanged. Scanning every statement had cost +6.5%.
  • cargo fmt --check, both make rust-lint clippy passes on the touched crates, and fensu check pass.

A star's span was a plain Option<Span>, which a located parse never
fills, so `*`, `t.*` and a star with EXCLUDE, REPLACE or RENAME had no
range in any dialect. It is now a SourceSpan like other located nodes:
recorded only by a located parse, ignored by equality, and left out of
serialised trees when absent.

Scope findings that fall back to the first located column, name or star
in their scope now point at that star instead of the statement start:
a star modifier naming an unknown column, a USING key one side lacks,
and a duplicate CTE name in a query that projects a star.
OutputLineage gains expression_range, the UTF-8 byte range in the
analysed SQL of the projection item that defines the column, so a caller
can highlight it in the SQL it compiled. A column a star brings points
at the star, and one a REPLACE gives a value points at that value.
Columns merged by a set operation have no single defining item and get
none.

Spans count Unicode characters, so the ranges are converted to bytes in
one pass over the SQL per statement, reading only up to the last offset
the outputs need.
EXCEPT and MINUS outputs kept the left branch's projection range, and a
BY NAME column found in only one branch kept that branch's. None of them
has a single defining item, as UNION and INTERSECT outputs already do
not, so a set operation now clears the range of every output it returns.

Columns a PIVOT or UNPIVOT passes through, or groups by, no longer carry
the range of the item that defined them in their input.
Character offsets are byte offsets in ASCII text, so the per-character
scan now runs only for SQL holding multi-byte characters. This keeps the
analysis instruction count within its recorded baseline.
@kvlonge
kvlonge merged commit 8a2ea92 into main Nov 11, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant