Skip to content

fix(config): use the query warehouse for standalone test, scenario and janitor #942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kvlonge merged 1 commit into from
Oct 4, 2026
Merged

fix(config): use the query warehouse for standalone test, scenario and janitor #942

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/janitor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ sqb --project-dir <path> janitor [flags]
| `--retention-days` | Override the configured retention period (days) before a stale relation is archived |
| `--direct-state-history-versions` | Override how many state-history versions are kept per identity |
| `--drop-old-name-view <name>` | Drop the [compatibility view](../concepts/models/migrations.md#old-names) at a renamed model's old name before it expires. Repeat for several views. Fails with `C503` if no compatibility view is recorded at the name |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |

## Targets

Expand Down
4 changes: 2 additions & 2 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scenario.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ sqb scenario test [flags]
| `--max-snapshot-bytes` | Override per-relation byte limit for capture |
| `--max-snapshot-total-bytes` | Override total byte limit for capture |
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |

### Selectors

Expand Down Expand Up @@ -131,7 +131,7 @@ sqb scenario capture [flags]
| `--max-snapshot-bytes` | Override per-relation byte limit |
| `--max-snapshot-total-bytes` | Override total byte limit |
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |

### Examples

Expand Down
2 changes: 1 addition & 1 deletion src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ sqb --project-dir <path> test [flags]
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--select`, `-s` | Select tests targeting specific models |
| `--exclude` | Exclude tests targeting specific models |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse |

Parameterized cases use parent-level selection. Selecting a target model includes every case in
each matching `TEST` template; case names are not model selectors.
Expand Down
6 changes: 4 additions & 2 deletions ...build/.agents/skills/sqlbuild/references/docs/concepts/project-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,11 +262,13 @@ temporary comparison tables on the `query` warehouse. The exception is `sqb dbt`

| Group | Commands |
|-------|----------|
| `build` | `audit`, `build`, `check`, `clone`, `dbt` (SQLBuild-owned work), `janitor`, `load`, `plan`, `scenario`, `seed`, `test` |
| `query` | `contract`, `cost`, `debug`, `diff`, `freshness`, `query` |
| `build` | `audit`, `build`, `check`, `clone`, `dbt` (SQLBuild-owned work), `load`, `plan`, `seed` |
| `query` | `contract`, `cost`, `debug`, `diff`, `freshness`, `janitor`, `query`, `scenario`, `test` |

Other commands, such as `compile`, `lineage`, and `format`, never connect to a warehouse.
Integrations that run builds, such as Dagster, invoke `sqb build` and use the `build` group.
Standalone `sqb test`, `sqb scenario` and `sqb janitor` use the `query` group because they work on
fixture data or catalogue metadata; tests and audits that run inside `sqb build` use the `build` group.

Each developer can point ad-hoc commands at their own warehouse without editing shared
configuration:
Expand Down
6 changes: 3 additions & 3 deletions src/sqlbuild/cli/commands/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,18 +151,18 @@
CliCommand.CHECK: WarehouseGroup.BUILD,
CliCommand.CLONE: WarehouseGroup.BUILD,
CliCommand.DBT: WarehouseGroup.BUILD,
CliCommand.JANITOR: WarehouseGroup.BUILD,
CliCommand.LOAD: WarehouseGroup.BUILD,
CliCommand.PLAN: WarehouseGroup.BUILD,
CliCommand.SCENARIO: WarehouseGroup.BUILD,
CliCommand.SEED: WarehouseGroup.BUILD,
CliCommand.TEST: WarehouseGroup.BUILD,
CliCommand.CONTRACT: WarehouseGroup.QUERY,
CliCommand.COST: WarehouseGroup.QUERY,
CliCommand.DEBUG: WarehouseGroup.QUERY,
CliCommand.DIFF: WarehouseGroup.QUERY,
CliCommand.FRESHNESS: WarehouseGroup.QUERY,
CliCommand.QUERY: WarehouseGroup.QUERY,
CliCommand.JANITOR: WarehouseGroup.QUERY,
CliCommand.SCENARIO: WarehouseGroup.QUERY,
CliCommand.TEST: WarehouseGroup.QUERY,
CliCommand.CLEAN: None,
CliCommand.COMPILE: None,
CliCommand.DAG: None,
Expand Down
8 changes: 8 additions & 0 deletions tests/unit/src/sqlbuild/cli/commands/_helpers/runtime/_test_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from sqlbuild.cli.commands.types import CliCommand
from sqlbuild.spec.contracts.models import LocalTargetConfig, TargetWarehousesConfig
from sqlbuild.spec.contracts.types import WarehouseGroup


@dataclass(frozen=True)
Expand Down Expand Up @@ -88,3 +89,10 @@ class CommandWarehouseClassificationTestCase:
description: str
command: CliCommand
expected_classified: bool = True


@dataclass(frozen=True)
class CommandWarehouseGroupTestCase:
description: str
command: CliCommand
expected_group: WarehouseGroup
44 changes: 44 additions & 0 deletions tests/unit/src/sqlbuild/cli/commands/_helpers/runtime/test_warehouses.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,11 @@
from sqlbuild.cli.commands.types import CliCommand
from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs
from sqlbuild.spec.contracts.models import LocalTargetConfig, TargetWarehousesConfig
from sqlbuild.spec.contracts.types import WarehouseGroup
from tests.unit.src.sqlbuild.cli.commands._helpers.runtime._test_types import (
CommandWarehouseClassificationTestCase,
CommandWarehouseErrorTestCase,
CommandWarehouseGroupTestCase,
CommandWarehouseResolutionTestCase,
WarehouseFlagParseTestCase,
WarehouseFlagRejectedTestCase,
Expand Down Expand Up @@ -339,5 +341,47 @@ def test_given_command_without_group_when_parsing_warehouse_flag_then_it_is_reje
assert test_case.expected_error_fragment in capsys.readouterr().err


@pytest.mark.parametrize(
"test_case",
[
CommandWarehouseGroupTestCase(
description="standalone SQL tests use fixture data",
command=CliCommand.TEST,
expected_group=WarehouseGroup.QUERY,
),
CommandWarehouseGroupTestCase(
description="scenarios build against fixture data",
command=CliCommand.SCENARIO,
expected_group=WarehouseGroup.QUERY,
),
CommandWarehouseGroupTestCase(
description="janitor works on catalogue metadata",
command=CliCommand.JANITOR,
expected_group=WarehouseGroup.QUERY,
),
CommandWarehouseGroupTestCase(
description="audits scan built tables",
command=CliCommand.AUDIT,
expected_group=WarehouseGroup.BUILD,
),
CommandWarehouseGroupTestCase(
description="Python checks read real outputs",
command=CliCommand.CHECK,
expected_group=WarehouseGroup.BUILD,
),
CommandWarehouseGroupTestCase(
description="builds execute project resources",
command=CliCommand.BUILD,
expected_group=WarehouseGroup.BUILD,
),
],
ids=lambda case: case.description,
)
def test_given_cli_command_when_resolving_group_then_matches_its_workload(
test_case: CommandWarehouseGroupTestCase,
) -> None:
assert COMMAND_WAREHOUSE_GROUPS[test_case.command] is test_case.expected_group


if __name__ == "__main__":
pytest.main([__file__, "-vv"])
2 changes: 1 addition & 1 deletion website/src/content/docs/docs/cli/janitor.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ sqb --project-dir <path> janitor [flags]
| `--retention-days` | Override the configured retention period (days) before a stale relation is archived |
| `--direct-state-history-versions` | Override how many state-history versions are kept per identity |
| `--drop-old-name-view <name>` | Drop the [compatibility view](/docs/concepts/models/migrations#old-names) at a renamed model's old name before it expires. Repeat for several views. Fails with `C503` if no compatibility view is recorded at the name |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |

## Targets

Expand Down
4 changes: 2 additions & 2 deletions website/src/content/docs/docs/cli/scenario.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ sqb scenario test [flags]
| `--max-snapshot-bytes` | Override per-relation byte limit for capture |
| `--max-snapshot-total-bytes` | Override total byte limit for capture |
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |

### Selectors

Expand Down Expand Up @@ -124,7 +124,7 @@ sqb scenario capture [flags]
| `--max-snapshot-bytes` | Override per-relation byte limit |
| `--max-snapshot-total-bytes` | Override total byte limit |
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |

### Examples

Expand Down
2 changes: 1 addition & 1 deletion website/src/content/docs/docs/cli/test.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ sqb --project-dir <path> test [flags]
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--select`, `-s` | Select tests targeting specific models |
| `--exclude` | Exclude tests targeting specific models |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |
| `--warehouse <name>` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse |

Parameterized cases use parent-level selection. Selecting a target model includes every case in
each matching `TEST` template; case names are not model selectors.
Expand Down
6 changes: 4 additions & 2 deletions website/src/content/docs/docs/concepts/project-configuration.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,11 +239,13 @@ temporary comparison tables on the `query` warehouse. The exception is `sqb dbt`

| Group | Commands |
|-------|----------|
| `build` | `audit`, `build`, `check`, `clone`, `dbt` (SQLBuild-owned work), `janitor`, `load`, `plan`, `scenario`, `seed`, `test` |
| `query` | `contract`, `cost`, `debug`, `diff`, `freshness`, `query` |
| `build` | `audit`, `build`, `check`, `clone`, `dbt` (SQLBuild-owned work), `load`, `plan`, `seed` |
| `query` | `contract`, `cost`, `debug`, `diff`, `freshness`, `janitor`, `query`, `scenario`, `test` |

Other commands, such as `compile`, `lineage`, and `format`, never connect to a warehouse.
Integrations that run builds, such as Dagster, invoke `sqb build` and use the `build` group.
Standalone `sqb test`, `sqb scenario` and `sqb janitor` use the `query` group because they work on
fixture data or catalogue metadata; tests and audits that run inside `sqb build` use the `build` group.

Each developer can point ad-hoc commands at their own warehouse without editing shared
configuration:
Expand Down
Loading