Skip to content

perf: speed up warm compiles and upgrade Polyglot to 0.12.2 - #779

Merged
kvlonge merged 1 commit into
mainfrom
perf/warm-cache-and-polyglot
Sep 22, 2026
Merged

kvlonge merged 1 commit into
mainfrom
perf/warm-cache-and-polyglot

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Why

Warm compiles rebuilt facts that were already cached; the Polyglot fork needed upstream 0.12.1 set-operation type fixes.

Changes

  • Polyglot fork 0.12.2 (includes upstream 0.12.1). Early recursive-CTE anchor binding now requires WITH RECURSIVE. When native binding fully resolves lineage but an output type is unknown, the native lineage is kept and types are recovered through the legacy path.
  • SQL Rules cache unchanged non-model SQL files whose text has no macro or interpolation syntax. Files that need expansion are always re-linted.
  • Cached analysis entries decode lineage into lazy indexed facts, with an eager fallback. Per-model cache digests use byte-identical orjson encoding.
  • Macro modules load without deep-copying their analyzed AST.
  • Discovery uses a prefix-based project-relative path helper, CSafeLoader for schema YAML, and builds path-default error context only on failure.
  • The artifact writer compares stale paths as strings. SQL test artifact identities use orjson (fingerprint bumped).
  • New CI guard: every fully cached fresh-process compile must take at most 0.67x the cold time (0.113.3: ~0.72, this branch: ~0.61).

Verification

  • make test: 6922 passed. make check-ci passed.
  • New integration tests cover:
    • mixed-type UNION ALL / UNION ALL BY NAME executed on DuckDB;
    • preserved CTE lineage under type recovery;
    • warm SQL Rules after editing a test file and a test-scoped macro.
  • New unit tests cover cached lineage round-trips and digest stability.
  • Paired fresh-process benchmarks (0.113.3 vs this branch) on the repository's semantic cache projects. Semantic fingerprints were identical in every run.
Models Phase Before After
3000 warm 7.21s 5.78s
3000 leaf edit 5.89s 5.25s
3000 cold 9.96s 9.41s
5000 warm 11.33s 9.33s
5000 leaf edit 9.21s 8.51s
5000 cold 15.75s 14.98s
10000 warm 21.43s 16.77s

@kvlonge
kvlonge enabled auto-merge (squash) September 22, 2026 23:39
@kvlonge
kvlonge merged commit ca25236 into main Sep 22, 2026
28 checks passed
@kvlonge
kvlonge deleted the perf/warm-cache-and-polyglot branch September 22, 2026 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant