Skip to content

fix(compile): linear missing-description locations and comment-tolerant expected fixtures - #924

Merged
kvlonge merged 2 commits into
mainfrom
fix/error-path-and-expected-comments
Oct 3, 2026
Merged

kvlonge merged 2 commits into
mainfrom
fix/error-path-and-expected-comments

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Why

  • Compiles with many missing descriptions slowed down quadratically. Each source or seed without a description re-read its whole YAML file to find its line, so a 10k-model project with 13,150 P010 errors spent about 200 s in attachment.
  • A comment before UNION ALL in an __expected__ SQL-test fixture raised a false P001 ("branch 2 does not match branch 1"), because the comment stayed attached to the last projection.

Changes

  • Index each YAML file's entry lines once, cached by contents, for P010 locations and the warehouse source-table lookup. Output is identical.
  • Trim comments around set-operation branches and projections in expected fixtures, on both the native and Python paths. The Python fallback splitter follows the adapter's lexical syntax, and Annotated branches are unwrapped.

Verification

  • Same-machine medians, wall (main → fix): 1k 8.3 → 5.4 s, 5k 68 → 20 s, 10k 232 → 41 s, with identical diagnostics JSON.
  • Scaling guard (500 vs 2,000 models without descriptions) fails on main and passes here; exact-output CLI test; entry-line unit tests and a randomised comparison with the old lookup.
  • DuckDB E2E with commented expected fixtures (line, block, nested), plus negative controls that still report P001.
  • One independent review; make test, make check-ci.

…cations

Locating each undescribed seed or source rescanned its whole YAML file with
a per-name regex, so error-heavy compiles grew quadratically with entries per
file. Index every entry name once per file and look names up from that index,
preserving the outermost-entry and pattern semantics exactly.
…fixtures

A comment before or after UNION ALL in an __expected__ fixture stayed
attached to the neighbouring branch, so the projection scanners read the
comment as an alias and reported a false P001 branch mismatch. Branch and
projection splits now drop surrounding comments, alias detection ignores
trailing comments, the Python fallback splitter follows the adapter's
lexical syntax, and Polyglot's annotated comment wrapper is unwrapped.
@kvlonge
kvlonge enabled auto-merge (squash) October 3, 2026 20:37
@kvlonge
kvlonge merged commit 2d8fe1d into main Oct 3, 2026
34 checks passed
@kvlonge
kvlonge deleted the fix/error-path-and-expected-comments branch October 3, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant