Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ own option. `--version` works only directly after `sqb`, as in `sqb --version`:
| `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](../concepts/planning/selection-and-staleness.md) |
| `--manifest` | Generate `target/manifest.json` with plan-aware project metadata |
| `SELECTOR ...` | Positional selectors, the same as `--select` |
| `--select`, `-s` | Select specific models |
| `--select`, `-s` | Select specific models, or unit tests by name (`test:<name>`); selecting only unit tests runs them without building models |
| `--exclude` | Exclude specific models |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |

Expand Down
21 changes: 17 additions & 4 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,19 @@ Positional `SELECTOR` arguments work like `--select`, and both forms can be comb
|------|-------------|
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `SELECTOR ...` | Positional selectors, the same as `--select` |
| `--select`, `-s` | Select tests targeting specific models |
| `--exclude` | Exclude tests targeting specific models |
| `--select`, `-s` | Select tests by name (`order_status_rules` or `test:order_status_rules`), or the tests targeting specific models |
| `--exclude` | Exclude tests by name, or the tests targeting specific models |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |

Parameterized cases use parent-level selection. Selecting a target model includes every case in
each matching `TEST` template; case names are not model selectors.
Test names are globally unique, so a bare test name selects exactly that test. `test:<name>` is the
explicit form and accepts the same glob patterns as names, such as `test:orders_*`. A bare pattern
selects tests only when it matches no other resource. Test selectors are unioned with model
selectors, so `sqb test test:order_status_rules stg_orders` runs that test and every test targeting
`stg_orders`. Test selectors cannot use `+` graph expansion or `,` intersection. Only `sqb test` and
`sqb build` accept test selectors; other commands reject them with `S013`.

Parameterized cases use parent-level selection. Selecting a test or a target model includes every
case of each matching `TEST` template, and `--case` narrows the run to one case name.

Text output identifies each case as `<parent> [<case>]` and includes its source path and safe typed
parameters. Structured JSON emits one check per case with stable source/block/case identity,
Expand All @@ -44,4 +51,10 @@ sqb test

# Run tests for a specific model
sqb test stg_orders

# Run one test by name
sqb test order_status_rules

# Run one case of a parameterized test
sqb test test:order_status_rules --case open_case
```
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ sqb build --select tag:exports # by tag
sqb build --select +orders_export # with upstreams
```

Names are globally unique across models, sources, seeds, functions, loaders, tasks, assets, checks, providers, and hooks.
Names are globally unique across models, sources, seeds, functions, loaders, tasks, assets, checks, providers, hooks, unit tests, and scenarios.

## Lifecycle: run, build, check

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ tests/
fulfillment__late_shipment.sql
```

Folders are organizational only. The scenario name is the filename stem and must be globally unique across all discovered scenario files.
Folders are organizational only. The scenario name is the filename stem. Like model, source, seed, and unit-test names, it must be globally unique across the project.

### Scenario format

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,19 @@ sqb build --select seed:waffle_types
sqb build --select source:raw__orders
```

### Unit test

`sqb test` and `sqb build` select SQL unit tests by name. Test names are globally unique, so a bare
name works; `test:` is the explicit form and accepts glob patterns:

```bash
sqb test order_status_rules
sqb test "test:orders_*" stg_customers
```

Test selectors are unioned with other selectors and cannot use `+` or `,`. Other commands reject
them with `S013`.

## Graph expansion

### Upstream
Expand Down Expand Up @@ -163,6 +176,14 @@ error[S007]: unknown selector name 'missing_*'
error[S009]: no models found under path 'models/nonexistent'.
error[S000]: no SQL resources or Python nodes found with tag 'nonexistent_tag'
error[S003]: path selector 'fact_orders~' requires names on both sides of '~'
error[S013]: selector 'order_status_rules' selects a unit test; only `sqb test` and `sqb build` accept unit-test selectors
```

Unknown names suggest the closest selectable names, including unit tests:

```
error[S007]: unknown selector name 'order_status_rule'
= help: did you mean 'order_status_rules'?
```

If a path selector omits the root directory, SQLBuild asks for the explicit form:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -548,7 +548,11 @@ CTE prefixes from other modes are not allowed. For example, `__source__` in a ma

## Multiple tests per file

A single test file can contain multiple `TEST()` blocks. Each block must have a unique `name`:
A single test file can contain multiple `TEST()` blocks. A test's name is its `name`, or the file
stem when `name` is omitted. Test names are globally unique: a test cannot share a name with another
test, a scenario, a model, a source, a seed, a function, or a Python node. Name the file or the block
after the behavior it checks, such as `orders_status_rules`, rather than after the model. The cases
of one parameterized test share its name. Each block in a file must have its own `name`:

```sql
TEST (name "completed_orders_only");
Expand Down
4 changes: 4 additions & 0 deletions src/sqlbuild/.agents/skills/sqlbuild/references/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@

Files live under `tests/unit/` (any subfolder). Each file is a `TEST();` header, CTEs, and a
required closing `SELECT 1`.
A test's name is its `name` or the file stem. Test names are globally unique across tests,
scenarios, models, sources, seeds, functions and Python nodes, so never name a test file after its
model (`test_stg_orders.sql`, not `stg_orders.sql`). Run one test with `sqb test <test name>` or
`sqb test test:<name>`; only `sqb test` and `sqb build` accept test selectors.

```sql
-- tests/unit/test_stg_orders.sql
Expand Down
20 changes: 16 additions & 4 deletions src/sqlbuild/cli/commands/_helpers/build_planning/defer_clone.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
from sqlbuild.compiler.pipeline.main.clone_with_options import run_clone_pipeline_with_options
from sqlbuild.compiler.pipeline.main.compiled_project import build_compiled_project
from sqlbuild.compiler.pipeline.main.unit_test_selection import split_unit_test_selectors
from sqlbuild.compiler.pipeline.models import (
ClonePipelineConnection,
ClonePipelineOptions,
Expand All @@ -44,7 +45,7 @@
resolve_skipped_view_chain,
)
from sqlbuild.compiler.planner.main.selection.scope import build_planner_scope
from sqlbuild.compiler.planner.models import PlannerScope
from sqlbuild.compiler.planner.models import PlannerScope, UnitTestSelectorSplit
from sqlbuild.compiler.planner.types import MaterializationType
from sqlbuild.executor.clone.main.build_destination_transient_models import (
build_destination_transient_models,
Expand Down Expand Up @@ -173,6 +174,17 @@ def run_defer_clone_boundary_prephase(
) -> DeferClonePrephaseOutcome:
"""Resolve defer-clone boundaries and clone them before build planning."""

split: UnitTestSelectorSplit = split_unit_test_selectors(
select=request.select,
exclude=request.exclude,
discovered_inputs=invocation.discovered_inputs,
)
if not split.sql_test_selection.models_selected:
return DeferClonePrephaseOutcome(
destination_target_name=invocation.effective_target_name,
boundary_selectors=(),
view_chain_selectors=(),
)
cloned_project: CompiledProject
boundary_selectors: tuple[str, ...]
view_chain_selectors: tuple[str, ...]
Expand All @@ -187,8 +199,8 @@ def run_defer_clone_boundary_prephase(
selected_target=request.selected_target,
no_sql_validation=request.no_sql_validation,
no_cache=request.no_cache,
select=request.select,
exclude=request.exclude,
select=split.select,
exclude=split.exclude,
cli_vars=request.cli_vars,
project_dir=invocation.effective_project_dir,
auto_load_sources=invocation.should_load_sources,
Expand All @@ -208,7 +220,7 @@ def run_defer_clone_boundary_prephase(
no_sql_validation=request.no_sql_validation,
no_cache=request.no_cache,
select=(*boundary_selectors, *view_chain_selectors),
caused_by_names=request.select,
caused_by_names=split.select,
cli_vars=request.cli_vars,
connection_config=invocation.connection_config,
project_dir=invocation.effective_project_dir,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ def _run_build_compile_pipeline(
max_microbatches=request.max_microbatches,
selection_diagnostics=request.selection_diagnostics,
plan_sql_tests=request.run_tests,
accepts_unit_test_selectors=True,
),
hooks=ConnectionHooks(
on_progress=invocation.planning_progress.on_progress,
Expand Down
6 changes: 6 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/check/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
)
from sqlbuild.compiler.discovery.models import DiscoveredCheckFunction
from sqlbuild.compiler.pipeline.main.project_relation_targets import build_project_relation_targets
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.compiler.pipeline.models import CompilePipelineResult
from sqlbuild.compiler.python_nodes.main.graph import build_discovered_python_node_graph
from sqlbuild.compiler.python_nodes.main.run_lifecycle import build_python_sql_run_lifecycle
Expand Down Expand Up @@ -57,6 +58,11 @@ def prepare_check_execution(
python_graph: PythonNodeGraph = build_discovered_python_node_graph(
discovered_inputs=invocation.discovered_inputs
)
reject_unit_test_selectors(
select=request.select,
exclude=request.exclude,
discovered_inputs=invocation.discovered_inputs,
)
check_names: frozenset[str] = resolve_selected_check_names(
graph=python_graph,
select=request.select,
Expand Down
6 changes: 6 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/check/planning.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from sqlbuild.compiler.compile.models import CompiledObjectKey
from sqlbuild.compiler.compile.types import CompiledResourceType
from sqlbuild.compiler.graph.main.sql_ref_key import sql_ref_key
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.compiler.pipeline.main.static_command import compile_static_command_context
from sqlbuild.compiler.pipeline.main.static_result import build_static_pipeline_result
from sqlbuild.compiler.pipeline.models import (
Expand All @@ -35,6 +36,11 @@ def compile_check_plan(
python_graph: PythonNodeGraph = build_discovered_python_node_graph(
discovered_inputs=invocation.discovered_inputs
)
reject_unit_test_selectors(
select=request.select,
exclude=request.exclude,
discovered_inputs=invocation.discovered_inputs,
)
check_names: frozenset[str] = resolve_selected_check_names(
graph=python_graph,
select=request.select,
Expand Down
4 changes: 4 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/clone/invocation.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from sqlbuild.cli.progress.classes.planning_progress_reporter import PlanningProgressReporter
from sqlbuild.compiler.discovery.main.discover import discover_project_inputs
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.presentation.main.supports_color import supports_color
from sqlbuild.spec.contracts.main.resolve_effective_adapter_name import (
resolve_effective_adapter_name,
Expand All @@ -38,6 +39,9 @@ def resolve_clone_invocation(*, request: CloneCommandRequest) -> CloneInvocation
project_dir=effective_project_dir
)
progress.complete("Discovered project.")
reject_unit_test_selectors(
select=request.select, exclude=request.exclude, discovered_inputs=discovered_inputs
)
destination_target_name: str | None = request.destination_target_name
if destination_target_name is None:
destination_target_name = resolve_target_name(
Expand Down
2 changes: 2 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/compile/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
from sqlbuild.compiler.discovery.main.discover import discover_project_inputs
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs, DiscoveryCacheRequest
from sqlbuild.compiler.lineage.models import ProjectColumnLineage
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.compiler.pipeline.main.selected_graph import (
build_project_graph_with_analysis_selection,
)
Expand Down Expand Up @@ -126,6 +127,7 @@ def _analyze_compile_project(
),
declaration_reuse=COMPILE_RENDER_REUSE.claim_discovery(),
)
reject_unit_test_selectors(select=select, exclude=exclude, discovered_inputs=discovered_inputs)
discover_ms: int = elapsed_ms(discover_start)
_ = complete_compile_phase(
status=status, message=f"Discovered project. ({discover_ms / 1000:.2f}s)"
Expand Down
4 changes: 4 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/diff/invocation.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
discover_project_configuration,
)
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors


def resolve_diff_invocation(*, request: DiffCommandRequest) -> DiffInvocation:
Expand All @@ -34,6 +35,9 @@ def resolve_diff_invocation(*, request: DiffCommandRequest) -> DiffInvocation:
if is_query_diff_request(request=request)
else discover_project_inputs(project_dir=effective_project_dir)
)
reject_unit_test_selectors(
select=request.select, exclude=request.exclude, discovered_inputs=discovered_inputs
)
if not request.select and not is_query_diff_request(request=request):
raise CliUserError("diff requires --select in v1", code="C204")
return DiffInvocation(
Expand Down
35 changes: 28 additions & 7 deletions src/sqlbuild/cli/commands/_helpers/lineage/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,10 @@
from sqlbuild.compiler.lineage.main.build_semantic_uses import build_direct_semantic_uses
from sqlbuild.compiler.lineage.models import DirectSemanticColumnUse
from sqlbuild.compiler.pipeline.main.graph import build_project_graph
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.compiler.pipeline.models import ProjectGraph
from sqlbuild.compiler.planner.constants import UNKNOWN_SELECTOR_ERROR_CODE
from sqlbuild.compiler.planner.exceptions import PlannerInputError
from sqlbuild.compiler.planner.main.selection.selection import resolve_project_selectors
from sqlbuild.presentation.main.supports_color import supports_color
from sqlbuild.spec.contracts.main.resolve_effective_adapter_name import (
Expand Down Expand Up @@ -113,6 +116,9 @@ def _prepare_graph(
sql_analysis_enabled_override=(None if requires_compiled_graph else False),
extract_output_column_locations=False,
)
reject_unit_test_selectors(
select=request.select, exclude=request.exclude, discovered_inputs=discovered
)
adapter: BaseAdapter = resolve_adapter(
adapter_name=resolve_effective_adapter_name(
project_config=discovered.project_config,
Expand Down Expand Up @@ -197,13 +203,28 @@ def _render_lineage(
depth=parsed_depth,
)
else:
lineage_graph = select_selector_lineage(
graph=graph,
select=request.select,
exclude=request.exclude,
depth=parsed_depth,
direction=request.direction,
)
try:
lineage_graph = select_selector_lineage(
graph=graph,
select=request.select,
exclude=request.exclude,
depth=parsed_depth,
direction=request.direction,
)
except PlannerInputError as error:
if error.code == UNKNOWN_SELECTOR_ERROR_CODE and isinstance(
graph, RelationLineageIndex
):
reject_unit_test_selectors(
select=request.select,
exclude=request.exclude,
discovered_inputs=discover_project_inputs(
project_dir=request.project_dir or Path.cwd(),
sql_analysis_enabled_override=False,
extract_output_column_locations=False,
),
)
raise
return _format_graph(graph=lineage_graph, output_format=request.output_format)


Expand Down
2 changes: 2 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/lint/selection.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from sqlbuild.compiler.discovery.main.discover import discover_project_inputs
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs, DiscoveredSqlModelFile
from sqlbuild.compiler.pipeline.main.graph import build_project_graph
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.compiler.pipeline.models import ProjectGraph
from sqlbuild.compiler.planner.exceptions import PlannerInputError
from sqlbuild.compiler.planner.main.selection.selection import resolve_project_selectors
Expand Down Expand Up @@ -61,6 +62,7 @@ def resolve_lint_inputs(
sql_analysis_enabled_override=False,
extract_output_column_locations=False,
)
reject_unit_test_selectors(select=select, exclude=exclude, discovered_inputs=discovered)
adapter = _resolve_discovered_adapter(
project_dir=project_dir,
discovered=discovered,
Expand Down
4 changes: 4 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/load/invocation.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from sqlbuild.compiler.compile.main.effective_target import build_effective_target_config
from sqlbuild.compiler.discovery.main.discover import discover_project_inputs
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.presentation.main.supports_color import supports_color
from sqlbuild.spec.contracts.main.resolve_effective_adapter_name import (
resolve_effective_adapter_name,
Expand All @@ -33,6 +34,9 @@ def resolve_load_invocation(*, request: LoadCommandRequest) -> LoadInvocation:
discovered_inputs: DiscoveredProjectInputs = discover_project_inputs(
project_dir=effective_project_dir
)
reject_unit_test_selectors(
select=request.select, exclude=request.exclude, discovered_inputs=discovered_inputs
)
target_config: TargetConfig | None = build_effective_target_config(
discovered_inputs=discovered_inputs,
selected_target=request.selected_target,
Expand Down
4 changes: 4 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/rules/command.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from sqlbuild.compiler.discovery.main.discover import discover_project_inputs
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
from sqlbuild.compiler.pipeline.main.graph import build_project_graph
from sqlbuild.compiler.pipeline.main.reject_unit_test_selectors import reject_unit_test_selectors
from sqlbuild.compiler.pipeline.models import ProjectGraph
from sqlbuild.compiler.planner.main.selection.selection import resolve_project_selectors
from sqlbuild.rule_engine.main.build_catalogue import build_catalogue
Expand Down Expand Up @@ -127,6 +128,9 @@ def _run_rule_selection(
return 2
print(f"Evaluating rule selection {selector}...", file=sys.stderr)
discovered: DiscoveredProjectInputs = discover_project_inputs(project_dir=project_dir)
reject_unit_test_selectors(
select=request.select, exclude=request.exclude, discovered_inputs=discovered
)
adapter: BaseAdapter = resolve_adapter(
adapter_name=resolve_effective_adapter_name(
project_config=discovered.project_config,
Expand Down
Loading
Loading