Skip to content

fix: stop false errors on stars, comments, set operations, snapshots and table-view switches - #1158

Merged
kvlonge merged 8 commits into
mainfrom
fix/false-errors-and-stale-caches
Oct 19, 2026
Merged

kvlonge merged 8 commits into
mainfrom
fix/false-errors-and-stale-caches

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

Five bugs produced false errors or failed builds on valid projects: a star beside a cast constant lost its expanded columns, a quote inside a Snowflake // comment broke compile, a set operation after an unaliased parenthesised FROM query was parsed into that subquery, a snapshot's second build reported its own validity columns as removed, and switching a model between table and view failed. Separately, the compile performance budgets had 40–100% headroom over CI measurements, so a large regression could pass.

Changes

  • Compact analysis answers any query whose star it expanded, so a star beside CAST(1 AS INTEGER) keeps its columns and downstream shapes.
  • Snowflake model-analysis normalisation skips // line comments.
  • Polyglot pinned to the fork commit that keeps FROM (SELECT …) UNION … at the outer query (no false column-count errors, correct lineage and formatting).
  • Schema-change detection leaves out a snapshot's validity columns (default or renamed).
  • Table↔view changes: when replacing fails and the catalog shows the other relation type, the build drops it and retries (catalog read only after a failure).
  • Compile wall, memory and delta budgets set to about 1.2× the slowest recent CI measurement (for example cold 10k 31 → 25.5 s, 10k delta edit 9 → 8 s, delta CPU ratio 0.30 → 0.275). Same-runner ratio limits unchanged.

Verification

  • CLI E2E for each fix that fails without it (star beside a cast; Snowflake/BigQuery line comments; set operation after a FROM query on DuckDB, Snowflake, BigQuery and PostgreSQL; snapshot rebuild with default and renamed columns; table→view and view→table); Rust tests for the analysis and normalisation fixes.
  • Snapshot, planner schema, view promotion, delta and compile-reuse suites; compiler differential (no golden changes); Polyglot feature gates; same-machine instruction counts unchanged; make check-ci.

@kvlonge
kvlonge enabled auto-merge (squash) October 19, 2026 12:12
Re-analysing a model whose star waited on its inputs handed queries with a
cast constant such as CAST(1 AS INTEGER) to the legacy analysis, which never
expands stars, and then marked the star resolved. The model kept only its
explicit columns, so a unique_key naming a starred column was a false B300
and downstream shapes lost columns. The compact analysis now answers any
query whose star it expanded.
The Snowflake variant normalization in model analysis only recognised -- and
/* */ comments, so an apostrophe or double quote inside a // comment opened
a fake string and the compile failed with an internal analysis error. The
scan now skips // line comments. BigQuery # comments already worked and are
covered by the same end-to-end test.
A rebuilt snapshot compared its query's columns with the warehouse table,
which also holds the snapshot's own valid_from and valid_to columns, so every
second build planned a schema change listing them as removed. Schema change
detection now leaves a snapshot's validity columns, default or renamed, out
of the comparison.
…nd view

A model already built as a table and changed to a view planned "recreate
view", but CREATE OR REPLACE VIEW cannot replace a table, so the build
failed; the reverse change failed when promotion renamed the existing view.
When creating the view or promoting the table fails and the live catalog
shows the destination is the other relation type, the build now drops it
and retries. The extra catalog read happens only after such a failure.
…CI measurements

Each absolute budget is now about 1.2x the slowest of two CI runs of this
branch, and never below 1.15x the slowest recent CI run on the same code (runs
based on 0.158.2 or later). That covers the fresh-process, cache, varied-cache,
dense cold, warm and edit, and delta edit-loop wall budgets, the delta CPU
ratios and the peak RSS limits. A limit already below that floor is
unchanged. The fallback CPU limit, the 4 GiB cgroup and the same-runner ratio
limits are unchanged.
…r query

Adopt the Polyglot parser fix that stops a parenthesised query in FROM
from absorbing a following UNION, INTERSECT or EXCEPT. Such models no
longer raise false column-count errors, their star columns trace to every
set-operation branch, and the formatter no longer nests the set operation
inside the FROM query.

Add end-to-end compile coverage for DuckDB, Snowflake, BigQuery and
PostgreSQL, plus a DuckDB build case.
@kvlonge kvlonge changed the title fix: stop false errors on stars, Snowflake comments, snapshots and table-view switches fix: stop false errors on stars, comments, set operations, snapshots and table-view switches Oct 19, 2026
@kvlonge
kvlonge force-pushed the fix/false-errors-and-stale-caches branch from 54364e3 to b13811d Compare October 19, 2026 15:06
@kvlonge
kvlonge merged commit b21b0ff into main Oct 19, 2026
42 of 45 checks passed
@kvlonge
kvlonge deleted the fix/false-errors-and-stale-caches branch October 19, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant