Skip to content

fix: run selected Python nodes against upstreams outside the run - #864

Merged
kvlonge merged 1 commit into
mainfrom
fix/chi-417-python-node-selection
Sep 28, 2026
Merged

kvlonge merged 1 commit into
mainfrom
fix/chi-417-python-node-selection

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Why

A selected Python task or asset only ran after every declared SQL dependency reported a result in the same build. A dependency outside the run never reports one, so sqb build --select task:x skipped the task, and a task on an unmanaged source was skipped on every build. SQL models already build against existing upstream relations when those upstreams are not selected.

Changes

  • The read-side Python tracker gates a node only on SQL resources scheduled in this run (models, seeds and source loads in the plan's execution order, plus pre-SQL source loads). Dependencies outside the run are read as they exist.
  • A skipped upstream model now skips its Python dependents, matching how the build blocks downstream models; seeds and source loads still gate on failure only.
  • sqb check no longer fakes success results for dependencies.
  • Python nodes overview documents the selection rule.

Verification

  • New DuckDB E2E test_python_node_selection.py (11 cases; 7 fail on the old code) covering tasks and assets selected alone, unmanaged sources, waiting on and skipping after a selected upstream, and a skipped upstream model.
  • The deferral E2E now asserts the task-only run reads the deferred source.
  • make test, make check-ci and the hygiene scan pass; one independent review.

Read-side tasks and assets now wait only for SQL resources scheduled in the current build; unselected upstreams and unmanaged sources are read as they exist, like unselected upstreams of a selected SQL model. A skipped upstream model now skips its dependent Python nodes, as it already blocks dependent models.
@kvlonge
kvlonge enabled auto-merge (squash) September 28, 2026 10:45
@kvlonge
kvlonge merged commit 0d1a8b4 into main Sep 28, 2026
31 checks passed
@kvlonge
kvlonge deleted the fix/chi-417-python-node-selection branch September 28, 2026 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant