Repository navigation
Commit cbaa787
authored
feat!: merge Python node folders into python/ (#850)
* feat!: merge Python node folders into python/
Discover loaders, tasks, assets, checks, factories, and audit factories
anywhere under a single python/ root. The decorator defines each node
kind, so folder-based kind validation is gone and undecorated helper
modules under python/ are importable without becoming nodes.
Remove the tasks/, assets/, checks/, loaders/, factories/, libs/,
dagster/, and rivers_pipeline/ Python roots and the root definitions.py
allowance. The Dagster and Rivers playgrounds now generate an
orchestrator directory with definitions.py next to a nested waffle_shop/
SQLBuild project. Path selectors for Python nodes use the python/ root.
BREAKING CHANGE: Python nodes and their helper modules must live under
python/ (for example python/tasks/orders.py, imported as
python.tasks.orders). Python under tasks/, assets/, checks/, loaders/,
factories/, libs/, dagster/, rivers_pipeline/, or a root definitions.py
now fails compilation with D016. Python path selectors use python/...
instead of tasks/, assets/, checks/, or loaders/.
* docs: document the python/ node root and external orchestrator layout
Update the Python nodes, project Python, selector, init, playground, quickstart, dlt, Dagster, and Rivers pages for the single python/ root and for orchestrator definitions that live outside the SQLBuild project, then regenerate the bundled skill references.
* docs: regenerate bundled skill references
* fix: import python/ modules under their package names
Discovery imports every python/ module as python.<dotted.path> through the normal import system, so helper packages can use relative imports and each module initializes once per discovery pass. python.* entries are evicted at the start of each pass so sequential projects and edited files are imported afresh.
* fix: register factory-returned module nodes once
A factory may return a decorated node that direct discovery already registers from its own python/ module. Skip that identical callable in factory output while keeping the duplicate-name error for distinct nodes.
* fix: install wrapper playground skills in the SQLBuild project
Dagster and Rivers playgrounds nest the SQLBuild project under waffle_shop/. Install the scaffolded agent skills there, where sqb skills and stale-skill maintenance resolve them, and document the location.
* test: align python/ discovery regressions with fensu rules
Check node decorators directly when de-duplicating factory output, split the factory E2E into success and conflict tests, and use expected_ fields and the shared repo-file fixture in the new regressions.
* fix: bind python/ to the project and de-duplicate factory nodes by identity
Discovery installs an explicit python package whose __path__ is the
project's python/ folder, so project imports never resolve to an
unrelated python package elsewhere on sys.path. Unrelated python.*
modules displaced during discovery are restored afterwards.
Discovery now imports every python/ module and registers direct nodes
before running factories, then skips factory results that are the same
callable direct discovery registered. A factory that binds a generated
node to a module global registers it once instead of dropping it.
* test: resolve the project root from python/ node fixtures1 parent ffe0dcf commit cbaa787
123 files changed
Lines changed: 2634 additions & 1187 deletions
File tree
- src/sqlbuild
- .agents/skills/sqlbuild/references/docs
- cli
- concepts
- python-nodes
- integrations
- cli/commands
- _helpers
- init
- playground
- templates
- loader_waffle_shop/python/loaders
- waffle_shop/python/loaders
- main/workspace
- compiler
- discovery
- _helpers/filesystem
- planner
- python_nodes
- tests
- e2e
- fixtures
- source_loader_strategies/python/loaders
- waffle_shop/python/loaders
- src/sqlbuild
- cli/commands/main
- bigquery
- build
- no_tests_no_audits
- check
- clone
- compile
- databricks
- freshness
- init
- load
- plan
- playground
- postgres
- providers
- snowflake
- sqlserver
- integrations
- dagster
- rivers
- integration/src/sqlbuild
- cli/commands/main
- compiler/pipeline
- executor/build/concurrent
- unit/src/sqlbuild
- cli
- commands
- _helpers/load_selection
- main
- dag
- init
- playground
- output/main
- plan_json
- plan
- compiler
- compile/_helpers
- dag/main
- discovery
- _helpers
- pipeline/_helpers
- planner/_helpers
- python_nodes/_helpers
- executor
- build/_helpers
- load
- python_nodes/_helpers
- integrations/dagster
- virtual/state/main
- website
- examples/waffle-shop/python/loaders
- src/content/docs/docs
- cli
- concepts
- python-nodes
- integrations
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
50 | 52 | | |
51 | 53 | | |
52 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
| 33 | + | |
37 | 34 | | |
38 | 35 | | |
39 | 36 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | | - | |
| 24 | + | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
41 | | - | |
| 42 | + | |
42 | 43 | | |
43 | | - | |
44 | | - | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
45 | 52 | | |
46 | 53 | | |
47 | 54 | | |
48 | | - | |
| 55 | + | |
49 | 56 | | |
50 | 57 | | |
51 | 58 | | |
| |||
61 | 68 | | |
62 | 69 | | |
63 | 70 | | |
64 | | - | |
| 71 | + | |
65 | 72 | | |
66 | 73 | | |
67 | 74 | | |
68 | 75 | | |
69 | 76 | | |
70 | | - | |
| 77 | + | |
71 | 78 | | |
72 | 79 | | |
73 | 80 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
86 | 93 | | |
87 | 94 | | |
88 | 95 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
531 | 531 | | |
532 | 532 | | |
533 | 533 | | |
534 | | - | |
535 | | - | |
536 | | - | |
537 | | - | |
538 | | - | |
539 | | - | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
540 | 555 | | |
541 | 556 | | |
542 | | - | |
543 | | - | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
544 | 560 | | |
545 | 561 | | |
546 | 562 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
| 60 | + | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
124 | | - | |
| 124 | + | |
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
| 39 | + | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
| 43 | + | |
64 | 44 | | |
65 | 45 | | |
66 | 46 | | |
67 | | - | |
| 47 | + | |
68 | 48 | | |
69 | 49 | | |
70 | | - | |
| 50 | + | |
71 | 51 | | |
72 | 52 | | |
73 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| |||
357 | 357 | | |
358 | 358 | | |
359 | 359 | | |
360 | | - | |
361 | | - | |
362 | | - | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
363 | 364 | | |
364 | 365 | | |
365 | 366 | | |
366 | 367 | | |
367 | 368 | | |
368 | 369 | | |
369 | 370 | | |
370 | | - | |
| 371 | + | |
371 | 372 | | |
372 | 373 | | |
373 | 374 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
| |||
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
23 | | - | |
| 24 | + | |
24 | 25 | | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
| |||
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
43 | 64 | | |
44 | 65 | | |
45 | 66 | | |
| |||
0 commit comments