Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,8 @@ A test mocks the sources and asserts on the model, resolving every model in betw
SQL:

```sql
TEST();

WITH
__source__raw__orders AS (
SELECT
Expand Down
9 changes: 4 additions & 5 deletions src/sqlbuild/.agents/skills/sqlbuild/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,11 +125,10 @@ do not hand-write that filter. Unlisted inputs are read in full. Use `__cursor_s
window into batches, each with its own audits. Always `sqb plan` an incremental change before building. Read
[references/incremental-models.md](references/incremental-models.md).

**Tests.** SQL unit tests live under `tests/unit/`. A test file mocks inputs with
`__source__<name>`, `__ref__<name>`, `__seed__<name>` CTEs, states expected output with
`__expected__<model>` (only the listed columns are compared, matched by name; unlisted columns are
ignored), and adds zero-row checks with `__assert__<name>`. A single-test file needs no `TEST();`
header and no closing `SELECT 1`.
**Tests.** SQL unit tests live under `tests/unit/`. A test file starts with a `TEST();` header,
mocks inputs with `__source__<name>`, `__ref__<name>`, `__seed__<name>` CTEs, states expected output
with `__expected__<model>` (only the listed columns are compared, matched by name; unlisted columns
are ignored), and adds zero-row checks with `__assert__<name>`. The closing `SELECT 1` is optional.
One test can span many models: mock the sources, assert on the final model, and every intermediate
model runs from its real SQL. Never write a test that only proves empty inputs produce no rows; it
cannot fail, Rules reject it, and it does not count toward `min_tests_per_model`. Mock real rows
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ A test first sees declarations available from its own folder tree. It may also u
macros, enums, and constants available to a model for which it defines expected output.

```sql
TEST();

WITH
__expected__orders AS (
SELECT
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ def mock_orders(count: int = 1) -> str:
```

```sql
TEST();

WITH
__source__raw__orders AS (
@mock_orders(3)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ A test file defines mock inputs and expected outputs using CTEs. SQLBuild substi

```sql
-- tests/unit/test_stg_orders.sql
TEST();

WITH
__source__raw__orders AS (
SELECT
Expand Down Expand Up @@ -81,11 +83,12 @@ that read them. Statically provable collection-versus-scalar type conflicts iden
column and suggest explicit `CAST`, `ARRAY_CONSTRUCT`, or `PARSE_JSON` expressions. SQLBuild never
invents missing values.

A test file needs no header and no closing statement: a file that holds one test can be just its
CTEs. Add a `TEST (...)` header to name the test, set a cursor window, declare parameters, or turn
off SQL analysis, and on every block of a file that holds several tests. The `TEST();` header and
the trailing `SELECT 1` that earlier releases required are still accepted, so this file is
equivalent to the first example:
Every test file starts with a `TEST()` header. All of its fields are optional: `TEST();` is a
complete header, and a test without a `name` is named after its file. Add fields to name the test,
set a cursor window, declare parameters, or turn off SQL analysis.

A test ends after its last CTE. The trailing `SELECT 1` that earlier releases required is still
accepted but no longer needed, so this file is equivalent to the first example:

```sql
TEST();
Expand Down Expand Up @@ -140,6 +143,8 @@ expected rows can therefore live in one helper that both an `__expected__` CTE a
read:

```sql
TEST();

WITH
__source__raw__orders AS (
SELECT 1 AS id, 3 AS quantity UNION ALL SELECT 2 AS id, 5 AS quantity
Expand Down Expand Up @@ -167,6 +172,8 @@ Tests can span multiple models in a single file. Mock your sources, define an ex
```sql
-- Mock two sources, assert on the final mart.
-- stg_orders and stg_payments resolve automatically from their real SQL.
TEST();

WITH
__source__raw__orders AS (
SELECT 1 AS id, 100 AS customer_id, 2 AS waffle_type_id, 3 AS quantity,
Expand Down Expand Up @@ -200,6 +207,8 @@ dependency errors.
You can mock models directly with `__ref__<name>` and seeds with `__seed__<name>`, not just sources. This skips the model's real SQL (or the seed's real CSV data) and provides controlled data instead:

```sql
TEST();

WITH
__ref__stg_orders AS (
SELECT
Expand Down Expand Up @@ -270,6 +279,8 @@ expressions, with the fixture relation. The mocked function is not deployed as a
```sql
-- models/customer_order_totals.sql reads:
-- __table_fn("customer_orders")(42)
TEST();

WITH
fixture_orders AS (
SELECT 101 AS order_id, 42 AS customer_id, 2500 AS amount_cents
Expand All @@ -295,6 +306,8 @@ compares the real function.
A single test can assert on multiple models. SQLBuild resolves and compares each one independently:

```sql
TEST();

WITH
__source__raw__orders AS (
SELECT 1 AS id, 100 AS customer_id, 2 AS waffle_type_id, 3 AS quantity,
Expand Down Expand Up @@ -330,6 +343,8 @@ Only explicit expected CTEs create grants. A matching test filename, `__ref__` m
Because unit tests are written in SQL, they support macro calls. This lets you write reusable mock generators instead of copy-pasting mock data across test files:

```sql
TEST();

WITH
__source__raw__orders AS (
@mock_orders()
Expand Down Expand Up @@ -361,6 +376,8 @@ with `D013`.
When a model uses macros that you want to control in tests (e.g. target-specific logic, dynamic SQL generation), you can override their output with `__macro__<name>` CTEs:

```sql
TEST();

WITH
__macro__country_filter AS (
SELECT 'country_code = ''US'''
Expand All @@ -387,6 +404,8 @@ This is useful for:
Unit tests can include `__assert__<name>` CTEs for property-based checks. An assertion passes if the query returns zero rows - any returned rows are failing examples.

```sql
TEST();

WITH
__ref__stg_orders AS (
SELECT 1 AS order_id, 100 AS customer_id, 3 AS quantity,
Expand Down Expand Up @@ -454,8 +473,9 @@ project file; see [Rule options](rules/configuration-and-selection.md#rule-optio
Model tests mock sources and refs and compare model outputs. Three more modes test reusable logic
directly, without a model chain. SQLBuild infers the mode from the test's CTEs: a test that defines
`__macro_actual__` is a macro test, `__udf_actual__` a UDF test, `__table_fn_actual__` a table
function test, and any other test a model test. `TEST (mode ...)` states the mode explicitly; an
explicit mode that contradicts the CTEs is a compile error.
function test, and any other test a model test, so the header's `mode` field is optional.
`TEST (mode ...)` states the mode explicitly; an explicit mode that contradicts the CTEs is a
compile error.

### Macro tests

Expand Down Expand Up @@ -544,11 +564,10 @@ CTE prefixes from other modes are not allowed. For example, `__source__` in a ma
## Multiple tests per file

A single test file can contain multiple `TEST()` blocks. A test's name is its `name`, or the file
stem when `name` or the whole header is omitted. Test names are globally unique: a test cannot share
a name with another test, a scenario, a model, a source, a seed, a function, or a Python node. Name
the file or the block after the behavior it checks, such as `orders_status_rules`, rather than after
the model. The cases of one parameterized test share its name. Every block in a multi-test file needs
its own `TEST (...)` header with a unique `name`:
stem when `name` is omitted. Test names are globally unique: a test cannot share a name with another
test, a scenario, a model, a source, a seed, a function, or a Python node. Name the file or the block
after the behavior it checks, such as `orders_status_rules`, rather than after the model. The cases
of one parameterized test share its name. Each block in a file must have its own `name`:

```sql
TEST (name "completed_orders_only");
Expand All @@ -572,8 +591,7 @@ __expected__stg_orders AS (
)
```

A file with a single test can omit the `name` field, or the whole header. Files with multiple tests
require a header with a name on every block.
A file with a single test can omit the `name` field. Files with multiple tests require names on every block.

## Repeating test logic

Expand Down Expand Up @@ -669,6 +687,8 @@ capture and replay identity remains explicit.
When independent status is unnecessary, a normal SQL case table remains concise:

```sql
TEST();

WITH cases(case_name, source_status, expected_status) AS (
VALUES
('completed', 'completed', 'completed'),
Expand Down
12 changes: 7 additions & 5 deletions src/sqlbuild/.agents/skills/sqlbuild/references/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,19 @@

## Writing a unit test

Files live under `tests/unit/` (any subfolder). A file with one test is just its CTEs; add a
`TEST (...)` header only to set a name, cursor window, parameters or `sql_analysis false`, and on
every block of a file with several tests. The older `TEST();` header and trailing `SELECT 1` are
still accepted but not needed.
Files live under `tests/unit/` (any subfolder). Every file starts with a `TEST();` header followed
by CTEs. Header fields are optional; add them to set a name, cursor window, parameters or
`sql_analysis false`, and give every block of a file with several tests its own `name`. The
trailing `SELECT 1` is optional.
A test's name is its `name` or the file stem. Test names are globally unique across tests,
scenarios, models, sources, seeds, functions and Python nodes, so never name a test file after its
model (`test_stg_orders.sql`, not `stg_orders.sql`). Run one test with `sqb test <test name>` or
`sqb test test:<name>`; only `sqb test` and `sqb build` accept test selectors.

```sql
-- tests/unit/test_stg_orders.sql
TEST();

WITH
__source__raw__orders AS (
SELECT 1 AS id, 100 AS customer_id, 'completed' AS status,
Expand Down Expand Up @@ -67,7 +69,7 @@ shared expected rows can live in one helper that both an `__expected__` CTE and

Macros work inside tests, so reusable mock generators such as `@mock_orders(count=5)` are normal.
Tests can also target a macro, UDF or table function directly: defining `__macro_actual__`,
`__udf_actual__` or `__table_fn_actual__` selects that mode without a `TEST (mode ...)` header; see
`__udf_actual__` or `__table_fn_actual__` selects that mode without a `mode` header field; see
[docs/concepts/testing.md](docs/concepts/testing.md) for those modes.

## Empty-input tests
Expand Down
25 changes: 5 additions & 20 deletions src/sqlbuild/compiler/discovery/_helpers/sql/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@
r"^\s*TEST\s*\(" + STATEMENT_HEADER_BODY_PATTERN + r"\)\s*;\s*",
re.DOTALL | re.MULTILINE,
)
_TEST_HEADER_START_PATTERN: re.Pattern[str] = re.compile(r"\s*TEST\s*\(")
_MODE_BY_ACTUAL_CTE_NAME: dict[str, SqlTestMode] = {
MACRO_ACTUAL_TEST_CTE_NAME: SqlTestMode.MACRO,
UDF_ACTUAL_TEST_CTE_NAME: SqlTestMode.UDF,
Expand Down Expand Up @@ -123,7 +122,10 @@ def parse_sql_test_file(*, contents: str, file_path: Path) -> tuple[DiscoveredSq
file_path=file_path, contents=contents
)
if not raw_test_blocks:
return (_parse_headerless_sql_test_file(file_path=file_path, contents=contents),)
raise SqlTestParseError(
f"SQL test '{file_path}' must start with a TEST() header as the first "
"non-whitespace content"
)

discovered_blocks: list[DiscoveredSqlTestBlock] = []
test_index: int
Expand Down Expand Up @@ -220,23 +222,6 @@ def _code_segment_bounds(
return (start + leading if first is None else first), start + len(stripped) - 1


def _parse_headerless_sql_test_file(*, file_path: Path, contents: str) -> DiscoveredSqlTestBlock:
if _TEST_HEADER_START_PATTERN.match(contents) is not None:
raise SqlTestParseError(
f"SQL test '{file_path}' starts with a TEST header that is not a complete "
"TEST(...); statement; close the header with `);` or omit it"
)
sql_body: str = cleandoc(contents)
if not sql_body.strip():
raise SqlTestParseError(f"SQL test '{file_path}' must define SQL")
return DiscoveredSqlTestBlock(
test_index=1,
header_values={},
sql_body=sql_body,
mode=_infer_sql_test_mode(sql_body=sql_body, file_path=file_path),
)


def _infer_sql_test_mode(*, sql_body: str, file_path: Path) -> SqlTestMode:
"""Infer a TEST block's mode from the direct-logic actual CTE it defines, if any."""

Expand Down Expand Up @@ -268,7 +253,7 @@ def _split_sql_test_blocks(*, file_path: Path, contents: str) -> tuple[tuple[str
if contents[: matches[0].start()].strip():
raise SqlTestParseError(
f"SQL test '{file_path}' must start with a TEST() header as the first "
"non-whitespace content; only a file with a single test may omit its header"
"non-whitespace content"
)

raw_blocks: list[tuple[str, int]] = []
Expand Down
2 changes: 1 addition & 1 deletion src/sqlbuild/lint/_helpers/expansion.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ def expand_file_bodies(
external_identifiers: tuple[str, ...] = external_identifiers_for_headers(
contents=contents, headers=headers
)
allows_ceremonial_select: bool = role.in_sql_test_directory or any(
allows_ceremonial_select: bool = any(
header.kind in {HEADER_KIND_TEST, HEADER_KIND_SCENARIO} for header in headers
)
allows_empty_fixture_star: bool = allows_ceremonial_select
Expand Down
2 changes: 0 additions & 2 deletions src/sqlbuild/lint/_helpers/headers.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
from pathlib import Path, PurePath

from sqlbuild.compiler.compile.constants import HOOK_DIRECTORY_NAME
from sqlbuild.compiler.discovery.constants import SQL_TESTS_OWNERSHIP_ROOT
from sqlbuild.compiler.discovery.main.named_declaration_role_kind import (
named_declaration_role_kind,
)
Expand Down Expand Up @@ -45,7 +44,6 @@ def lint_file_role(
declaration_kind=named_declaration_role_kind(relative_path=relative_path),
in_project=True,
in_hook_directory=relative_path.is_relative_to(HOOK_DIRECTORY_NAME),
in_sql_test_directory=relative_path.is_relative_to(SQL_TESTS_OWNERSHIP_ROOT),
)


Expand Down
14 changes: 3 additions & 11 deletions src/sqlbuild/lint/_helpers/native_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
)
from sqlbuild.compiler.sql_analysis.constants import POLYGLOT_MAX_FUNCTION_CALL_DEPTH
from sqlbuild.compiler.sql_analysis.models import SqlLexicalSyntax
from sqlbuild.lint._helpers.headers import lint_body_ranges, lint_file_role, scan_headers
from sqlbuild.lint._helpers.headers import lint_body_ranges, scan_headers
from sqlbuild.lint._helpers.sqlbuild_tokens import neutralize_interpolation, restore_interpolation
from sqlbuild.lint.constants import (
CARRIAGE_RETURN_LINE_FEED,
Expand All @@ -27,13 +27,7 @@
VIOLATION_SEVERITY_FAULT,
)
from sqlbuild.lint.exceptions import InterpolationRestorationError, NativeLintError
from sqlbuild.lint.models import (
HeaderSpan,
InterpolationSite,
LintConfig,
LintFileRole,
LintViolation,
)
from sqlbuild.lint.models import HeaderSpan, InterpolationSite, LintConfig, LintViolation

_NATIVE_FORMAT_API_VERSION: int = 1
_FIXTURE_HEADER_KINDS: frozenset[str] = frozenset({HEADER_KIND_TEST, HEADER_KIND_SCENARIO})
Expand Down Expand Up @@ -75,15 +69,13 @@ def format_native_sql_bodies(
requests_by_key: dict[tuple[str, str, str], dict[str, object]] = {}
for file_path, contents in sorted(files.items()):
headers: tuple[HeaderSpan, ...] = scan_headers(contents=contents)
role: LintFileRole = lint_file_role(file_path=file_path, project_dir=project_dir)
body_ranges: tuple[tuple[int, int], ...] = lint_body_ranges(
contents=contents,
headers=headers,
file_path=file_path,
project_dir=project_dir,
role=role,
)
allows_omitted_select: bool = role.in_sql_test_directory or any(
allows_omitted_select: bool = any(
header.kind in _FIXTURE_HEADER_KINDS for header in headers
)
prepared_bodies: list[_PreparedBody] = []
Expand Down
1 change: 0 additions & 1 deletion src/sqlbuild/lint/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ class LintFileRole:
declaration_kind: DeclarationKind | None = None
in_project: bool = False
in_hook_directory: bool = False
in_sql_test_directory: bool = False


@dataclass(frozen=True)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ class AuthoredSqlParityTestCase:

@dataclass(frozen=True)
class OptionalTestCeremonyE2ETestCase:
"""SQL test files written with and without their optional header, mode, and SELECT 1."""
"""SQL test files written with and without their optional mode and SELECT 1."""

description: str
test_files: dict[str, str]
Expand Down
Loading
Loading