Repository navigation
perf(planner): read fingerprint state once and keep plan generation linear - #904
Merged
Merged
Conversation
Full-project plans and projects declaring migrations read the latest fingerprint per identity without a node filter, apply the selection in memory, and share the rows with migration inspection instead of reading the state table a second time. Narrow selections keep the filtered read.
Statement monitors without a query-ID provider now start their thread only once a statement outlives the heartbeat threshold, through one shared deferred-start daemon, instead of starting and joining a thread for every statement. Lifecycle events are validated once at construction rather than again on publish and on sequencing, JSON freezing builds error paths only on failure, and statements hash their SQL once. Recorded events, diagnostics, and -v output are unchanged.
Expected and write identity resolution copied the whole hash cache for every resolved node, which made identity context building and change reconciliation quadratic in project size. A memoized resolver now records each node once with the same resolution order, cycle handling, and results.
SQL-test fixture completion copied the project-wide expected-type map for every test, which made planning test-heavy projects quadratic. Each test now overlays only the models of its chain on the shared map.
The rules completion line reported only built-in and custom rule time, which run concurrently and are not phase wall time, so users could not see how long the rules phase took next to the compile line. It now leads with the phase wall time in the same form as other phase lines.
Plan the inspection benchmark at 1,000 and 3,000 models without the compile cache and fail when warehouse inspection or plan generation grows more than 1.5 times faster than the project, or when either phase exceeds a loose ceiling.
… starts A stopped monitor now cancels its pending deferred start and drops its callbacks and query-ID provider, so a finished statement and its SQL are no longer retained until the heartbeat deadline. A monitor thread is recorded only after it starts, so a refused start no longer makes stop() join an unstarted thread and fail an otherwise successful statement.
…imings The rules progress line reads the clock once before rules run, between the compile and planning windows. The phase-timing test now supplies that read, gives rules a measurable duration, and proves compile and planning stay disjoint from it, with and without progress reporting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Planning time grew faster than project size: at 10k models plan generation took 9-13 s and warehouse inspection about 16 s on a synthetic DuckDB project, and a real Snowflake plan read the fingerprint state table twice (CHI-460).
Changes
migrate_from) and shared with migration inspection; narrow selections keep the filtered read.Verification
-v,--jsonand lifecycle events identical apart from IDs and timings; retention test proves finished statements are collectable.make test,make check-ci; one independent review, findings fixed.