Skip to content

perf(compile): stop the speculative SQL lint when its result is unused - #919

Merged
kvlonge merged 1 commit into
mainfrom
perf/early-lint-wait
Oct 3, 2026
Merged

kvlonge merged 1 commit into
mainfrom
perf/early-lint-wait

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Why

Compiles start a speculative SQL lint in a background thread. When the compile has graph errors, the rules step never runs, but the compile still blocked for the lint to finish, about 10 s on a ~1,400-model project, outside every timing phase. The thread also competed for the GIL with model analysis. The lint itself rebuilt each file's newline table once per violation, character by character (CHI-479).

Changes

  • Build each file's newline table once per lint pass.
  • The early lint checks a stop flag between files and bodies. The compile stops it right after the graph phase when there are graph errors, and otherwise after rules, so a result that is consumed is never cut short.
  • New compile_timings.early_lint_wait_ms records any remaining wait.

Verification

  • Same-machine medians (CPU s, base 0.132.1 → head): ~1,400-model project with compile errors, cold 33.1 → 22.3, warm 24.2 → 13.5. Dense 3k with lint findings, cold 50.9 → 47.2 (built-in rules 25.3 → 15.9 s). Diagnostics and rule findings identical.
  • Unit tests for locations; integration test that the lint is stopped, not awaited, on graph errors; DuckDB E2E for error and clean projects (findings identical).
  • One independent review; make test, make check-ci.

… per file

Signal the speculative lint thread to stop as soon as the compile knows its
result will not be consumed (graph errors or end of analysis), checking the
stop between files and bodies instead of joining a full lint run. Report any
remaining join as compile_timings.early_lint_wait_ms.

Build each file's newline table once per native lint pass instead of once per
violation.
@kvlonge
kvlonge enabled auto-merge (squash) October 3, 2026 04:41
@kvlonge
kvlonge merged commit 9fae2ec into main Oct 3, 2026
55 of 57 checks passed
@kvlonge
kvlonge deleted the perf/early-lint-wait branch October 3, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant