Skip to content

fix: speed up clone preparation and keep its console output clean - #1180

Merged
kvlonge merged 7 commits into
mainfrom
fix/clone-and-console-output
Oct 20, 2026
Merged

kvlonge merged 7 commits into
mainfrom
fix/clone-and-console-output

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Why

sqb clone spent most of its time preparing (two to four full compiles with SQL analysis, every source inspected twice), its finished spinner redrew over clone output, and library logs (INFO lines, warning tracebacks) printed amid normal output.

Changes

  • Prepared ... messages end their phase, so clone and --defer-clone-from spinners stop before items print; failed preparation stops its spinner at once. Failed ... planning messages (cursor watermarks) end their phase as an error line.
  • The clone completion line shows its time once: (7.83s), not ((7.83s)).
  • Clone compiles each target it reads once, structurally (no SQL validation or column inference); a defer_sources_to target reuses a compiled one. Clone's no-op --no-sql-analysis is accepted but hidden and documented as such.
  • Clone resolves source columns once, only for sources recreated views and functions read.
  • Without --debug, installed-library records reach the console only at ERROR (full) or WARNING (one line); INFO is hidden. Records logged from project files (hooks, loaders, rules) keep full output. --debug is unchanged.
  • Stored diagnostics (compute logs, Recall) keep library tracebacks.
  • Console log writes pause and redraw the live progress line, reusing its spinner thread.

Verification

  • DuckDB E2E comparing clone items, relations and view SQL across selection, exclude, defer_sources_to, seeds, sources, incrementals, functions and a missing origin, against the previous release.
  • Terminal E2E: no spinner after preparation completes or fails. Unit tests: phase-ending messages, completion line, hidden clone help, one spinner thread across 200 log records.
  • Recall integration test: a project hook logs in full while a library is summarised; both keep tracebacks in Recall.
  • 1,000 models: preparation 3.73 s to 1.00 s wall, 58.3 to 18.9 billion instructions (median of 3 alternating runs); 10,000 models: 35.8 s to 11.5 s. Results identical.
  • make check-ci.

@kvlonge
kvlonge enabled auto-merge (squash) October 20, 2026 15:35
Records logged from outside the project now count as library output, so the user family in
the stdout-purity test comes from a module inside the test project.
@kvlonge
kvlonge merged commit f21d774 into main Oct 20, 2026
86 of 88 checks passed
@kvlonge
kvlonge deleted the fix/clone-and-console-output branch October 20, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant