Skip to content

Commit 896c452

Browse files
authored
feat(test): select unit tests by name and require unique test names (#989)
* feat(test): require globally unique unit-test and scenario names Unit-test names (TEST name or file stem) and scenario names now join the global resource-name rule with models, sources, seeds, functions, Python nodes, providers and hooks. A clash is a compile error naming both kinds and files; cases of one parameterized test share its name. Fixtures that named a test or scenario after its model are renamed. * feat(test): select unit tests by name sqb test and sqb build accept a unit-test name, bare or as test:<name>, with glob patterns. Test selectors union with resource selectors, --case still narrows, and selecting only tests runs them without building models. Every other command rejects unit-test selectors with S013, and unknown names suggest the closest selectable names.
1 parent 65cc87e commit 896c452

74 files changed

Lines changed: 1390 additions & 139 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ own option. `--version` works only directly after `sqb`, as in `sqb --version`:
7272
| `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](../concepts/planning/selection-and-staleness.md) |
7373
| `--manifest` | Generate `target/manifest.json` with plan-aware project metadata |
7474
| `SELECTOR ...` | Positional selectors, the same as `--select` |
75-
| `--select`, `-s` | Select specific models |
75+
| `--select`, `-s` | Select specific models, or unit tests by name (`test:<name>`); selecting only unit tests runs them without building models |
7676
| `--exclude` | Exclude specific models |
7777
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |
7878

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,12 +23,19 @@ Positional `SELECTOR` arguments work like `--select`, and both forms can be comb
2323
|------|-------------|
2424
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
2525
| `SELECTOR ...` | Positional selectors, the same as `--select` |
26-
| `--select`, `-s` | Select tests targeting specific models |
27-
| `--exclude` | Exclude tests targeting specific models |
26+
| `--select`, `-s` | Select tests by name (`order_status_rules` or `test:order_status_rules`), or the tests targeting specific models |
27+
| `--exclude` | Exclude tests by name, or the tests targeting specific models |
2828
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |
2929

30-
Parameterized cases use parent-level selection. Selecting a target model includes every case in
31-
each matching `TEST` template; case names are not model selectors.
30+
Test names are globally unique, so a bare test name selects exactly that test. `test:<name>` is the
31+
explicit form and accepts the same glob patterns as names, such as `test:orders_*`. A bare pattern
32+
selects tests only when it matches no other resource. Test selectors are unioned with model
33+
selectors, so `sqb test test:order_status_rules stg_orders` runs that test and every test targeting
34+
`stg_orders`. Test selectors cannot use `+` graph expansion or `,` intersection. Only `sqb test` and
35+
`sqb build` accept test selectors; other commands reject them with `S013`.
36+
37+
Parameterized cases use parent-level selection. Selecting a test or a target model includes every
38+
case of each matching `TEST` template, and `--case` narrows the run to one case name.
3239

3340
Text output identifies each case as `<parent> [<case>]` and includes its source path and safe typed
3441
parameters. Structured JSON emits one check per case with stable source/block/case identity,
@@ -44,4 +51,10 @@ sqb test
4451

4552
# Run tests for a specific model
4653
sqb test stg_orders
54+
55+
# Run one test by name
56+
sqb test order_status_rules
57+
58+
# Run one case of a parameterized test
59+
sqb test test:order_status_rules --case open_case
4760
```

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/python-nodes/overview.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,7 @@ sqb build --select tag:exports # by tag
162162
sqb build --select +orders_export # with upstreams
163163
```
164164

165-
Names are globally unique across models, sources, seeds, functions, loaders, tasks, assets, checks, providers, and hooks.
165+
Names are globally unique across models, sources, seeds, functions, loaders, tasks, assets, checks, providers, hooks, unit tests, and scenarios.
166166

167167
## Lifecycle: run, build, check
168168

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/scenarios.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ tests/
4444
fulfillment__late_shipment.sql
4545
```
4646

47-
Folders are organizational only. The scenario name is the filename stem and must be globally unique across all discovered scenario files.
47+
Folders are organizational only. The scenario name is the filename stem. Like model, source, seed, and unit-test names, it must be globally unique across the project.
4848

4949
### Scenario format
5050

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/selectors.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,19 @@ sqb build --select seed:waffle_types
8080
sqb build --select source:raw__orders
8181
```
8282

83+
### Unit test
84+
85+
`sqb test` and `sqb build` select SQL unit tests by name. Test names are globally unique, so a bare
86+
name works; `test:` is the explicit form and accepts glob patterns:
87+
88+
```bash
89+
sqb test order_status_rules
90+
sqb test "test:orders_*" stg_customers
91+
```
92+
93+
Test selectors are unioned with other selectors and cannot use `+` or `,`. Other commands reject
94+
them with `S013`.
95+
8396
## Graph expansion
8497

8598
### Upstream
@@ -163,6 +176,14 @@ error[S007]: unknown selector name 'missing_*'
163176
error[S009]: no models found under path 'models/nonexistent'.
164177
error[S000]: no SQL resources or Python nodes found with tag 'nonexistent_tag'
165178
error[S003]: path selector 'fact_orders~' requires names on both sides of '~'
179+
error[S013]: selector 'order_status_rules' selects a unit test; only `sqb test` and `sqb build` accept unit-test selectors
180+
```
181+
182+
Unknown names suggest the closest selectable names, including unit tests:
183+
184+
```
185+
error[S007]: unknown selector name 'order_status_rule'
186+
= help: did you mean 'order_status_rules'?
166187
```
167188

168189
If a path selector omits the root directory, SQLBuild asks for the explicit form:

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/testing.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -548,7 +548,11 @@ CTE prefixes from other modes are not allowed. For example, `__source__` in a ma
548548

549549
## Multiple tests per file
550550

551-
A single test file can contain multiple `TEST()` blocks. Each block must have a unique `name`:
551+
A single test file can contain multiple `TEST()` blocks. A test's name is its `name`, or the file
552+
stem when `name` is omitted. Test names are globally unique: a test cannot share a name with another
553+
test, a scenario, a model, a source, a seed, a function, or a Python node. Name the file or the block
554+
after the behavior it checks, such as `orders_status_rules`, rather than after the model. The cases
555+
of one parameterized test share its name. Each block in a file must have its own `name`:
552556

553557
```sql
554558
TEST (name "completed_orders_only");

‎src/sqlbuild/.agents/skills/sqlbuild/references/testing.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@
2323

2424
Files live under `tests/unit/` (any subfolder). Each file is a `TEST();` header, CTEs, and a
2525
required closing `SELECT 1`.
26+
A test's name is its `name` or the file stem. Test names are globally unique across tests,
27+
scenarios, models, sources, seeds, functions and Python nodes, so never name a test file after its
28+
model (`test_stg_orders.sql`, not `stg_orders.sql`). Run one test with `sqb test <test name>` or
29+
`sqb test test:<name>`; only `sqb test` and `sqb build` accept test selectors.
2630

2731
```sql
2832
-- tests/unit/test_stg_orders.sql

‎src/sqlbuild/cli/commands/_helpers/build_planning/defer_clone.py‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
3333
from sqlbuild.compiler.pipeline.main.clone_with_options import run_clone_pipeline_with_options
3434
from sqlbuild.compiler.pipeline.main.compiled_project import build_compiled_project
35+
from sqlbuild.compiler.pipeline.main.unit_test_selection import split_unit_test_selectors
3536
from sqlbuild.compiler.pipeline.models import (
3637
ClonePipelineConnection,
3738
ClonePipelineOptions,
@@ -44,7 +45,7 @@
4445
resolve_skipped_view_chain,
4546
)
4647
from sqlbuild.compiler.planner.main.selection.scope import build_planner_scope
47-
from sqlbuild.compiler.planner.models import PlannerScope
48+
from sqlbuild.compiler.planner.models import PlannerScope, UnitTestSelectorSplit
4849
from sqlbuild.compiler.planner.types import MaterializationType
4950
from sqlbuild.executor.clone.main.build_destination_transient_models import (
5051
build_destination_transient_models,
@@ -173,6 +174,17 @@ def run_defer_clone_boundary_prephase(
173174
) -> DeferClonePrephaseOutcome:
174175
"""Resolve defer-clone boundaries and clone them before build planning."""
175176

177+
split: UnitTestSelectorSplit = split_unit_test_selectors(
178+
select=request.select,
179+
exclude=request.exclude,
180+
discovered_inputs=invocation.discovered_inputs,
181+
)
182+
if not split.sql_test_selection.models_selected:
183+
return DeferClonePrephaseOutcome(
184+
destination_target_name=invocation.effective_target_name,
185+
boundary_selectors=(),
186+
view_chain_selectors=(),
187+
)
176188
cloned_project: CompiledProject
177189
boundary_selectors: tuple[str, ...]
178190
view_chain_selectors: tuple[str, ...]
@@ -187,8 +199,8 @@ def run_defer_clone_boundary_prephase(
187199
selected_target=request.selected_target,
188200
no_sql_validation=request.no_sql_validation,
189201
no_cache=request.no_cache,
190-
select=request.select,
191-
exclude=request.exclude,
202+
select=split.select,
203+
exclude=split.exclude,
192204
cli_vars=request.cli_vars,
193205
project_dir=invocation.effective_project_dir,
194206
auto_load_sources=invocation.should_load_sources,
@@ -208,7 +220,7 @@ def run_defer_clone_boundary_prephase(
208220
no_sql_validation=request.no_sql_validation,
209221
no_cache=request.no_cache,
210222
select=(*boundary_selectors, *view_chain_selectors),
211-
caused_by_names=request.select,
223+
caused_by_names=split.select,
212224
cli_vars=request.cli_vars,
213225
connection_config=invocation.connection_config,
214226
project_dir=invocation.effective_project_dir,

‎src/sqlbuild/cli/commands/_helpers/build_planning/planning.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ def _run_build_compile_pipeline(
5959
max_microbatches=request.max_microbatches,
6060
selection_diagnostics=request.selection_diagnostics,
6161
plan_sql_tests=request.run_tests,
62+
accepts_unit_test_selectors=True,
6263
),
6364
hooks=ConnectionHooks(
6465
on_progress=invocation.planning_progress.on_progress,

‎src/sqlbuild/cli/commands/_helpers/check/execution.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
)
2525
from sqlbuild.compiler.discovery.models import DiscoveredCheckFunction
2626
from sqlbuild.compiler.pipeline.main.project_relation_targets import build_project_relation_targets
27+
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
2728
from sqlbuild.compiler.pipeline.models import CompilePipelineResult
2829
from sqlbuild.compiler.python_nodes.main.graph import build_discovered_python_node_graph
2930
from sqlbuild.compiler.python_nodes.main.run_lifecycle import build_python_sql_run_lifecycle
@@ -57,6 +58,11 @@ def prepare_check_execution(
5758
python_graph: PythonNodeGraph = build_discovered_python_node_graph(
5859
discovered_inputs=invocation.discovered_inputs
5960
)
61+
reject_unit_test_selectors(
62+
select=request.select,
63+
exclude=request.exclude,
64+
discovered_inputs=invocation.discovered_inputs,
65+
)
6066
check_names: frozenset[str] = resolve_selected_check_names(
6167
graph=python_graph,
6268
select=request.select,

0 commit comments

Comments
 (0)