Skip to content

fix(lineage): expand each shared node once when rendering lineage trees - #870

Merged
kvlonge merged 1 commit into
mainfrom
fix/lineage-tree-shared-subtrees
Sep 28, 2026
Merged

kvlonge merged 1 commit into
mainfrom
fix/lineage-tree-shared-subtrees

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Why

sqb lineage <model> text output could run for minutes and grow past 20 GB of memory on projects with shared dependencies. The tree renderer only tracked the current path, so every shared model was expanded again on every path reaching it, and output grew exponentially with the number of diamonds.

Changes

  • Relation and column-trace tree renderers track shown nodes across the whole render: each node is expanded once, later occurrences print (already shown).
  • Both renderers are iterative, so deep graphs cannot hit the recursion limit.

Verification

  • New DuckDB E2E over a 14-layer diamond project (downstream, upstream and column trace) asserts each node is expanded at most once and output stays linear; the relation cases fail on the old renderer.
  • On a private ~2,000-model project the same command went from >135 s / 21.8 GB (killed) to 2.4 s / 79 MB.
  • make test, make check-ci and the hygiene scan pass; one independent review.

@kvlonge
kvlonge enabled auto-merge (squash) September 28, 2026 18:56
@kvlonge
kvlonge merged commit 78b4626 into main Sep 28, 2026
29 checks passed
@kvlonge
kvlonge deleted the fix/lineage-tree-shared-subtrees branch September 28, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant