diff --git a/README.md b/README.md index 289ae98aaf..3c0f57583b 100644 --- a/README.md +++ b/README.md @@ -175,6 +175,8 @@ A test mocks the sources and asserts on the model, resolving every model in betw SQL: ```sql +TEST(); + WITH __source__raw__orders AS ( SELECT diff --git a/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md b/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md index b36ae29cbc..40dff43ac0 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md @@ -125,11 +125,10 @@ do not hand-write that filter. Unlisted inputs are read in full. Use `__cursor_s window into batches, each with its own audits. Always `sqb plan` an incremental change before building. Read [references/incremental-models.md](references/incremental-models.md). -**Tests.** SQL unit tests live under `tests/unit/`. A test file mocks inputs with -`__source__`, `__ref__`, `__seed__` CTEs, states expected output with -`__expected__` (only the listed columns are compared, matched by name; unlisted columns are -ignored), and adds zero-row checks with `__assert__`. A single-test file needs no `TEST();` -header and no closing `SELECT 1`. +**Tests.** SQL unit tests live under `tests/unit/`. A test file starts with a `TEST();` header, +mocks inputs with `__source__`, `__ref__`, `__seed__` CTEs, states expected output +with `__expected__` (only the listed columns are compared, matched by name; unlisted columns +are ignored), and adds zero-row checks with `__assert__`. The closing `SELECT 1` is optional. One test can span many models: mock the sources, assert on the final model, and every intermediate model runs from its real SQL. Never write a test that only proves empty inputs produce no rows; it cannot fail, Rules reject it, and it does not count toward `min_tests_per_model`. Mock real rows diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/visibility.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/visibility.md index 04b5c46328..35846fac07 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/visibility.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/declaration-scopes/visibility.md @@ -110,6 +110,8 @@ A test first sees declarations available from its own folder tree. It may also u macros, enums, and constants available to a model for which it defines expected output. ```sql +TEST(); + WITH __expected__orders AS ( SELECT diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/macros.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/macros.md index 7e741b05d9..7c28c84d6a 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/macros.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/macros.md @@ -177,6 +177,8 @@ def mock_orders(count: int = 1) -> str: ``` ```sql +TEST(); + WITH __source__raw__orders AS ( @mock_orders(3) diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/testing.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/testing.md index 8943459fe7..0e912fe427 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/testing.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/testing.md @@ -37,6 +37,8 @@ A test file defines mock inputs and expected outputs using CTEs. SQLBuild substi ```sql -- tests/unit/test_stg_orders.sql +TEST(); + WITH __source__raw__orders AS ( SELECT @@ -81,11 +83,12 @@ that read them. Statically provable collection-versus-scalar type conflicts iden column and suggest explicit `CAST`, `ARRAY_CONSTRUCT`, or `PARSE_JSON` expressions. SQLBuild never invents missing values. -A test file needs no header and no closing statement: a file that holds one test can be just its -CTEs. Add a `TEST (...)` header to name the test, set a cursor window, declare parameters, or turn -off SQL analysis, and on every block of a file that holds several tests. The `TEST();` header and -the trailing `SELECT 1` that earlier releases required are still accepted, so this file is -equivalent to the first example: +Every test file starts with a `TEST()` header. All of its fields are optional: `TEST();` is a +complete header, and a test without a `name` is named after its file. Add fields to name the test, +set a cursor window, declare parameters, or turn off SQL analysis. + +A test ends after its last CTE. The trailing `SELECT 1` that earlier releases required is still +accepted but no longer needed, so this file is equivalent to the first example: ```sql TEST(); @@ -140,6 +143,8 @@ expected rows can therefore live in one helper that both an `__expected__` CTE a read: ```sql +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 3 AS quantity UNION ALL SELECT 2 AS id, 5 AS quantity @@ -167,6 +172,8 @@ Tests can span multiple models in a single file. Mock your sources, define an ex ```sql -- Mock two sources, assert on the final mart. -- stg_orders and stg_payments resolve automatically from their real SQL. +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 100 AS customer_id, 2 AS waffle_type_id, 3 AS quantity, @@ -200,6 +207,8 @@ dependency errors. You can mock models directly with `__ref__` and seeds with `__seed__`, not just sources. This skips the model's real SQL (or the seed's real CSV data) and provides controlled data instead: ```sql +TEST(); + WITH __ref__stg_orders AS ( SELECT @@ -270,6 +279,8 @@ expressions, with the fixture relation. The mocked function is not deployed as a ```sql -- models/customer_order_totals.sql reads: -- __table_fn("customer_orders")(42) +TEST(); + WITH fixture_orders AS ( SELECT 101 AS order_id, 42 AS customer_id, 2500 AS amount_cents @@ -295,6 +306,8 @@ compares the real function. A single test can assert on multiple models. SQLBuild resolves and compares each one independently: ```sql +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 100 AS customer_id, 2 AS waffle_type_id, 3 AS quantity, @@ -330,6 +343,8 @@ Only explicit expected CTEs create grants. A matching test filename, `__ref__` m Because unit tests are written in SQL, they support macro calls. This lets you write reusable mock generators instead of copy-pasting mock data across test files: ```sql +TEST(); + WITH __source__raw__orders AS ( @mock_orders() @@ -361,6 +376,8 @@ with `D013`. When a model uses macros that you want to control in tests (e.g. target-specific logic, dynamic SQL generation), you can override their output with `__macro__` CTEs: ```sql +TEST(); + WITH __macro__country_filter AS ( SELECT 'country_code = ''US''' @@ -387,6 +404,8 @@ This is useful for: Unit tests can include `__assert__` CTEs for property-based checks. An assertion passes if the query returns zero rows - any returned rows are failing examples. ```sql +TEST(); + WITH __ref__stg_orders AS ( SELECT 1 AS order_id, 100 AS customer_id, 3 AS quantity, @@ -454,8 +473,9 @@ project file; see [Rule options](rules/configuration-and-selection.md#rule-optio Model tests mock sources and refs and compare model outputs. Three more modes test reusable logic directly, without a model chain. SQLBuild infers the mode from the test's CTEs: a test that defines `__macro_actual__` is a macro test, `__udf_actual__` a UDF test, `__table_fn_actual__` a table -function test, and any other test a model test. `TEST (mode ...)` states the mode explicitly; an -explicit mode that contradicts the CTEs is a compile error. +function test, and any other test a model test, so the header's `mode` field is optional. +`TEST (mode ...)` states the mode explicitly; an explicit mode that contradicts the CTEs is a +compile error. ### Macro tests @@ -544,11 +564,10 @@ CTE prefixes from other modes are not allowed. For example, `__source__` in a ma ## Multiple tests per file A single test file can contain multiple `TEST()` blocks. A test's name is its `name`, or the file -stem when `name` or the whole header is omitted. Test names are globally unique: a test cannot share -a name with another test, a scenario, a model, a source, a seed, a function, or a Python node. Name -the file or the block after the behavior it checks, such as `orders_status_rules`, rather than after -the model. The cases of one parameterized test share its name. Every block in a multi-test file needs -its own `TEST (...)` header with a unique `name`: +stem when `name` is omitted. Test names are globally unique: a test cannot share a name with another +test, a scenario, a model, a source, a seed, a function, or a Python node. Name the file or the block +after the behavior it checks, such as `orders_status_rules`, rather than after the model. The cases +of one parameterized test share its name. Each block in a file must have its own `name`: ```sql TEST (name "completed_orders_only"); @@ -572,8 +591,7 @@ __expected__stg_orders AS ( ) ``` -A file with a single test can omit the `name` field, or the whole header. Files with multiple tests -require a header with a name on every block. +A file with a single test can omit the `name` field. Files with multiple tests require names on every block. ## Repeating test logic @@ -669,6 +687,8 @@ capture and replay identity remains explicit. When independent status is unnecessary, a normal SQL case table remains concise: ```sql +TEST(); + WITH cases(case_name, source_status, expected_status) AS ( VALUES ('completed', 'completed', 'completed'), diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/testing.md b/src/sqlbuild/.agents/skills/sqlbuild/references/testing.md index 20291ec9dc..a0a9b46cc9 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/testing.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/testing.md @@ -21,10 +21,10 @@ ## Writing a unit test -Files live under `tests/unit/` (any subfolder). A file with one test is just its CTEs; add a -`TEST (...)` header only to set a name, cursor window, parameters or `sql_analysis false`, and on -every block of a file with several tests. The older `TEST();` header and trailing `SELECT 1` are -still accepted but not needed. +Files live under `tests/unit/` (any subfolder). Every file starts with a `TEST();` header followed +by CTEs. Header fields are optional; add them to set a name, cursor window, parameters or +`sql_analysis false`, and give every block of a file with several tests its own `name`. The +trailing `SELECT 1` is optional. A test's name is its `name` or the file stem. Test names are globally unique across tests, scenarios, models, sources, seeds, functions and Python nodes, so never name a test file after its model (`test_stg_orders.sql`, not `stg_orders.sql`). Run one test with `sqb test ` or @@ -32,6 +32,8 @@ model (`test_stg_orders.sql`, not `stg_orders.sql`). Run one test with `sqb test ```sql -- tests/unit/test_stg_orders.sql +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 100 AS customer_id, 'completed' AS status, @@ -67,7 +69,7 @@ shared expected rows can live in one helper that both an `__expected__` CTE and Macros work inside tests, so reusable mock generators such as `@mock_orders(count=5)` are normal. Tests can also target a macro, UDF or table function directly: defining `__macro_actual__`, -`__udf_actual__` or `__table_fn_actual__` selects that mode without a `TEST (mode ...)` header; see +`__udf_actual__` or `__table_fn_actual__` selects that mode without a `mode` header field; see [docs/concepts/testing.md](docs/concepts/testing.md) for those modes. ## Empty-input tests diff --git a/src/sqlbuild/compiler/discovery/_helpers/sql/tests.py b/src/sqlbuild/compiler/discovery/_helpers/sql/tests.py index bd97f583cc..0fc1599718 100644 --- a/src/sqlbuild/compiler/discovery/_helpers/sql/tests.py +++ b/src/sqlbuild/compiler/discovery/_helpers/sql/tests.py @@ -52,7 +52,6 @@ r"^\s*TEST\s*\(" + STATEMENT_HEADER_BODY_PATTERN + r"\)\s*;\s*", re.DOTALL | re.MULTILINE, ) -_TEST_HEADER_START_PATTERN: re.Pattern[str] = re.compile(r"\s*TEST\s*\(") _MODE_BY_ACTUAL_CTE_NAME: dict[str, SqlTestMode] = { MACRO_ACTUAL_TEST_CTE_NAME: SqlTestMode.MACRO, UDF_ACTUAL_TEST_CTE_NAME: SqlTestMode.UDF, @@ -123,7 +122,10 @@ def parse_sql_test_file(*, contents: str, file_path: Path) -> tuple[DiscoveredSq file_path=file_path, contents=contents ) if not raw_test_blocks: - return (_parse_headerless_sql_test_file(file_path=file_path, contents=contents),) + raise SqlTestParseError( + f"SQL test '{file_path}' must start with a TEST() header as the first " + "non-whitespace content" + ) discovered_blocks: list[DiscoveredSqlTestBlock] = [] test_index: int @@ -220,23 +222,6 @@ def _code_segment_bounds( return (start + leading if first is None else first), start + len(stripped) - 1 -def _parse_headerless_sql_test_file(*, file_path: Path, contents: str) -> DiscoveredSqlTestBlock: - if _TEST_HEADER_START_PATTERN.match(contents) is not None: - raise SqlTestParseError( - f"SQL test '{file_path}' starts with a TEST header that is not a complete " - "TEST(...); statement; close the header with `);` or omit it" - ) - sql_body: str = cleandoc(contents) - if not sql_body.strip(): - raise SqlTestParseError(f"SQL test '{file_path}' must define SQL") - return DiscoveredSqlTestBlock( - test_index=1, - header_values={}, - sql_body=sql_body, - mode=_infer_sql_test_mode(sql_body=sql_body, file_path=file_path), - ) - - def _infer_sql_test_mode(*, sql_body: str, file_path: Path) -> SqlTestMode: """Infer a TEST block's mode from the direct-logic actual CTE it defines, if any.""" @@ -268,7 +253,7 @@ def _split_sql_test_blocks(*, file_path: Path, contents: str) -> tuple[tuple[str if contents[: matches[0].start()].strip(): raise SqlTestParseError( f"SQL test '{file_path}' must start with a TEST() header as the first " - "non-whitespace content; only a file with a single test may omit its header" + "non-whitespace content" ) raw_blocks: list[tuple[str, int]] = [] diff --git a/src/sqlbuild/lint/_helpers/expansion.py b/src/sqlbuild/lint/_helpers/expansion.py index ca7026ac9d..d0235ec799 100644 --- a/src/sqlbuild/lint/_helpers/expansion.py +++ b/src/sqlbuild/lint/_helpers/expansion.py @@ -361,7 +361,7 @@ def expand_file_bodies( external_identifiers: tuple[str, ...] = external_identifiers_for_headers( contents=contents, headers=headers ) - allows_ceremonial_select: bool = role.in_sql_test_directory or any( + allows_ceremonial_select: bool = any( header.kind in {HEADER_KIND_TEST, HEADER_KIND_SCENARIO} for header in headers ) allows_empty_fixture_star: bool = allows_ceremonial_select diff --git a/src/sqlbuild/lint/_helpers/headers.py b/src/sqlbuild/lint/_helpers/headers.py index 47d1d323fd..9dc2ccb033 100644 --- a/src/sqlbuild/lint/_helpers/headers.py +++ b/src/sqlbuild/lint/_helpers/headers.py @@ -7,7 +7,6 @@ from pathlib import Path, PurePath from sqlbuild.compiler.compile.constants import HOOK_DIRECTORY_NAME -from sqlbuild.compiler.discovery.constants import SQL_TESTS_OWNERSHIP_ROOT from sqlbuild.compiler.discovery.main.named_declaration_role_kind import ( named_declaration_role_kind, ) @@ -45,7 +44,6 @@ def lint_file_role( declaration_kind=named_declaration_role_kind(relative_path=relative_path), in_project=True, in_hook_directory=relative_path.is_relative_to(HOOK_DIRECTORY_NAME), - in_sql_test_directory=relative_path.is_relative_to(SQL_TESTS_OWNERSHIP_ROOT), ) diff --git a/src/sqlbuild/lint/_helpers/native_format.py b/src/sqlbuild/lint/_helpers/native_format.py index 3d4b6edba5..48911aa3e8 100644 --- a/src/sqlbuild/lint/_helpers/native_format.py +++ b/src/sqlbuild/lint/_helpers/native_format.py @@ -15,7 +15,7 @@ ) from sqlbuild.compiler.sql_analysis.constants import POLYGLOT_MAX_FUNCTION_CALL_DEPTH from sqlbuild.compiler.sql_analysis.models import SqlLexicalSyntax -from sqlbuild.lint._helpers.headers import lint_body_ranges, lint_file_role, scan_headers +from sqlbuild.lint._helpers.headers import lint_body_ranges, scan_headers from sqlbuild.lint._helpers.sqlbuild_tokens import neutralize_interpolation, restore_interpolation from sqlbuild.lint.constants import ( CARRIAGE_RETURN_LINE_FEED, @@ -27,13 +27,7 @@ VIOLATION_SEVERITY_FAULT, ) from sqlbuild.lint.exceptions import InterpolationRestorationError, NativeLintError -from sqlbuild.lint.models import ( - HeaderSpan, - InterpolationSite, - LintConfig, - LintFileRole, - LintViolation, -) +from sqlbuild.lint.models import HeaderSpan, InterpolationSite, LintConfig, LintViolation _NATIVE_FORMAT_API_VERSION: int = 1 _FIXTURE_HEADER_KINDS: frozenset[str] = frozenset({HEADER_KIND_TEST, HEADER_KIND_SCENARIO}) @@ -75,15 +69,13 @@ def format_native_sql_bodies( requests_by_key: dict[tuple[str, str, str], dict[str, object]] = {} for file_path, contents in sorted(files.items()): headers: tuple[HeaderSpan, ...] = scan_headers(contents=contents) - role: LintFileRole = lint_file_role(file_path=file_path, project_dir=project_dir) body_ranges: tuple[tuple[int, int], ...] = lint_body_ranges( contents=contents, headers=headers, file_path=file_path, project_dir=project_dir, - role=role, ) - allows_omitted_select: bool = role.in_sql_test_directory or any( + allows_omitted_select: bool = any( header.kind in _FIXTURE_HEADER_KINDS for header in headers ) prepared_bodies: list[_PreparedBody] = [] diff --git a/src/sqlbuild/lint/models.py b/src/sqlbuild/lint/models.py index e9f32d7945..30539080cd 100644 --- a/src/sqlbuild/lint/models.py +++ b/src/sqlbuild/lint/models.py @@ -40,7 +40,6 @@ class LintFileRole: declaration_kind: DeclarationKind | None = None in_project: bool = False in_hook_directory: bool = False - in_sql_test_directory: bool = False @dataclass(frozen=True) diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py index a23a1a051e..36b8fea57b 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/test/_test_types.py @@ -226,7 +226,7 @@ class AuthoredSqlParityTestCase: @dataclass(frozen=True) class OptionalTestCeremonyE2ETestCase: - """SQL test files written with and without their optional header, mode, and SELECT 1.""" + """SQL test files written with and without their optional mode and SELECT 1.""" description: str test_files: dict[str, str] diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/test/test_optional_test_ceremony.py b/tests/e2e/src/sqlbuild/cli/commands/main/test/test_optional_test_ceremony.py index 9ee7760fbd..4675da2a0d 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/test/test_optional_test_ceremony.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/test/test_optional_test_ceremony.py @@ -1,4 +1,4 @@ -"""E2E coverage for SQL tests without a TEST header, mode, or trailing SELECT 1.""" +"""E2E coverage for SQL tests without an explicit mode or trailing SELECT 1.""" from __future__ import annotations @@ -15,7 +15,7 @@ ) from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import run_sqb -_HEADERLESS_MODEL_TEST: str = ( +_MODEL_TEST_CTES: str = ( "WITH\n" "__source__raw_orders AS (\n" " SELECT 1 AS order_id, ' PAID ' AS status\n" @@ -24,7 +24,7 @@ " SELECT 1 AS order_id, 'paid' AS status\n" ")\n" ) -_HEADERLESS_MACRO_TEST: str = ( +_MACRO_TEST_CTES: str = ( "WITH\n" "input_values AS (\n" " SELECT ' PAID ' AS raw_status\n" @@ -37,17 +37,17 @@ " SELECT 'paid' AS status\n" ")\n" ) +_SHORT_MODEL_TEST: str = "TEST();\n\n" + _MODEL_TEST_CTES +_SHORT_MACRO_TEST: str = "TEST();\n\n" + _MACRO_TEST_CTES _EXPLICIT_MACRO_TEST: str = ( - 'TEST (mode macro, name "normalizes_status_explicitly");\n\n' - + _HEADERLESS_MACRO_TEST - + "SELECT 1\n" + 'TEST (mode macro, name "normalizes_status_explicitly");\n\n' + _MACRO_TEST_CTES + "SELECT 1\n" ) -_EXPLICIT_MODEL_TEST: str = "TEST();\n\n" + _HEADERLESS_MODEL_TEST + "SELECT 1\n" +_EXPLICIT_MODEL_TEST: str = "TEST();\n\n" + _MODEL_TEST_CTES + "SELECT 1\n" _NAMED_BLOCKS_TEST: str = ( 'TEST (name "orders_keep_paid");\n\n' - + _HEADERLESS_MODEL_TEST + + _MODEL_TEST_CTES + '\nTEST (name "orders_keep_void");\n\n' - + _HEADERLESS_MODEL_TEST.replace("' PAID '", "'void'").replace("'paid'", "'void'") + + _MODEL_TEST_CTES.replace("' PAID '", "'void'").replace("'paid'", "'void'") ) @@ -55,10 +55,10 @@ "test_case", [ OptionalTestCeremonyE2ETestCase( - description="headerless and explicit test forms run, lint, and stay formatted", + description="short and explicit test forms run, lint, and stay formatted", test_files={ - "tests/unit/test_orders.sql": _HEADERLESS_MODEL_TEST, - "tests/unit/test_normalize_status.sql": _HEADERLESS_MACRO_TEST, + "tests/unit/test_orders.sql": _SHORT_MODEL_TEST, + "tests/unit/test_normalize_status.sql": _SHORT_MACRO_TEST, "tests/unit/test_orders_explicit.sql": _EXPLICIT_MODEL_TEST, "tests/unit/test_normalize_status_explicit.sql": _EXPLICIT_MACRO_TEST, "tests/unit/test_orders_named_blocks.sql": _NAMED_BLOCKS_TEST, @@ -115,7 +115,7 @@ def test_given_optional_test_ceremony_when_testing_then_short_and_explicit_forms description="an explicit mode that contradicts the ctes is rejected", test_files={ "tests/unit/test_normalize_status.sql": ( - "TEST (mode model);\n\n" + _HEADERLESS_MACRO_TEST + "TEST (mode model);\n\n" + _MACRO_TEST_CTES ), }, expected_exit_code=1, @@ -145,5 +145,38 @@ def test_given_contradicting_explicit_mode_when_testing_then_compile_rejects_it( assert fragment in output, output +@pytest.mark.parametrize( + "test_case", + [ + OptionalTestCeremonyE2ETestCase( + description="a test file without a TEST header is rejected", + test_files={"tests/unit/test_orders.sql": _MODEL_TEST_CTES}, + expected_exit_code=1, + expected_output_fragments=( + "D003", + "must start with a TEST() header as the first non-whitespace content", + ), + ), + ], + ids=lambda case: case.description, +) +def test_given_test_file_without_header_when_testing_then_discovery_rejects_it( + test_case: OptionalTestCeremonyE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_optional_ceremony_project( + tmp_path=tmp_path, test_files=test_case.test_files + ) + + tested: subprocess.CompletedProcess[str] = run_sqb( + command=("--no-color", "test"), project_dir=project_dir + ) + + output: str = tested.stdout + tested.stderr + assert tested.returncode == test_case.expected_exit_code, output + for fragment in test_case.expected_output_fragments: + assert fragment in output, output + + if __name__ == "__main__": pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_sql_tests.py b/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_sql_tests.py index 11481b30cb..66b3b7e59e 100644 --- a/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_sql_tests.py +++ b/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_sql_tests.py @@ -122,9 +122,7 @@ def test_given_batch_preparation_failure_when_discovering_tolerantly_then_report ) -> None: tests_dir: Path = tmp_path / "tests" / "unit" tests_dir.mkdir(parents=True) - (tests_dir / "a_invalid.sql").write_text( - 'TEST (name "unterminated")\nSELECT 1', encoding="utf-8" - ) + (tests_dir / "a_invalid.sql").write_text("SELECT 1", encoding="utf-8") (tests_dir / "b_valid.sql").write_text('TEST (name "valid_order");\nSELECT 1', encoding="utf-8") preparation: Mock = Mock(side_effect=ValueError("invalid batch projection")) monkeypatch.setattr(test_file_discovery, "prepare_sql_test_file_headers", preparation) @@ -165,23 +163,6 @@ def test_given_batch_preparation_failure_when_discovering_tolerantly_then_report expected_test_indexes=(1,), expected_header_values=({},), ), - ParseSqlTestFileTestCase( - description="treats a headerless file as one unnamed test block", - contents=""" - -- Orders keep their identifiers. - WITH - __source__orders AS ( - SELECT 1 AS order_id - ) - """, - expected_names=(None,), - expected_sql_bodies=( - "-- Orders keep their identifiers.\nWITH\n__source__orders AS (\n" - " SELECT 1 AS order_id\n)", - ), - expected_test_indexes=(1,), - expected_header_values=({},), - ), ParseSqlTestFileTestCase( description="discovers multiple named test blocks from one file", contents=""" @@ -340,23 +321,37 @@ def test_given_sql_test_file_variants_when_parsing_then_it_returns_expected_raw_ "test_case", [ ParseSqlTestFileErrorTestCase( - description="raises when a headerless body precedes another test block", + description="raises when the file does not start with a test header", + contents="SELECT 1\n", + expected_error_fragment="must start with a TEST", + ), + ParseSqlTestFileErrorTestCase( + description="raises when a single-test file of ctes omits its test header", + contents=( + "WITH __source__orders AS (SELECT 1 AS id),\n" + "__expected__orders AS (SELECT 1 AS id)\n" + ), + expected_error_fragment=r"must start with a TEST\(\) header", + ), + ParseSqlTestFileErrorTestCase( + description="raises when a body precedes another test block", contents='SELECT 1;\n\nTEST (name "second");\n\nSELECT 1\n', - expected_error_fragment="only a file with a single test may omit its header", + expected_error_fragment="must start with a TEST", ), ParseSqlTestFileErrorTestCase( description="raises when a test header is not terminated", contents='TEST (name "orders")\n\nWITH __source__orders AS (SELECT 1 AS id)\n', - expected_error_fragment=r"not a complete TEST\(\.\.\.\); statement", + expected_error_fragment="must start with a TEST", ), ParseSqlTestFileErrorTestCase( - description="raises when a headerless file is empty", + description="raises when the file is empty", contents="\n \n", - expected_error_fragment="must define SQL", + expected_error_fragment="must start with a TEST", ), ParseSqlTestFileErrorTestCase( description="raises when one block defines actual CTEs for several modes", contents=( + "TEST();\n" "WITH __macro_actual__ AS (SELECT 1 AS a),\n__udf_actual__ AS (SELECT 1 AS a)\n" ), expected_error_fragment=r"actual CTEs for several test modes \(macro, udf\)", @@ -536,13 +531,14 @@ def test_given_invalid_sql_test_file_contents_when_parsing_then_it_raises_clear_ [ ParseSqlTestModeTestCase( description="defaults to model mode without direct-logic actual CTEs", - contents="WITH __source__orders AS (SELECT 1 AS id),\n__expected__orders AS (SELECT 1 AS id)", + contents="TEST();\nWITH __source__orders AS (SELECT 1 AS id),\n__expected__orders AS (SELECT 1 AS id)", expected_modes=("model",), expected_header_values=({},), ), ParseSqlTestModeTestCase( - description="infers macro mode in a headerless file", + description="infers macro mode under an empty header", contents=( + "TEST();\n" "WITH __macro_actual__ AS (SELECT @double(1) AS value),\n" "__macro_expected__ AS (SELECT 2 AS value)" ), @@ -563,6 +559,7 @@ def test_given_invalid_sql_test_file_contents_when_parsing_then_it_raises_clear_ ParseSqlTestModeTestCase( description="infers table function mode from a lower-case as keyword", contents=( + "TEST();\n" "WITH __table_fn_actual__ as (SELECT 1 AS value),\n" "__table_fn_expected__ as (SELECT 1 AS value)" ), @@ -585,6 +582,7 @@ def test_given_invalid_sql_test_file_contents_when_parsing_then_it_raises_clear_ ParseSqlTestModeTestCase( description="ignores actual CTE names in comments, strings and nested queries", contents=( + "TEST();\n" "-- __macro_actual__ AS (\n" "WITH __source__orders AS (\n" " WITH __udf_actual__ AS (SELECT 1 AS id) SELECT id FROM __udf_actual__\n" diff --git a/tests/unit/src/sqlbuild/lint/_test_types.py b/tests/unit/src/sqlbuild/lint/_test_types.py index 5f4c66122b..e532fb69c7 100644 --- a/tests/unit/src/sqlbuild/lint/_test_types.py +++ b/tests/unit/src/sqlbuild/lint/_test_types.py @@ -49,7 +49,7 @@ class FormatDescriptionResolutionTestCase: @dataclass(frozen=True) class FixtureCeremonyFormatTestCase: - """A test or scenario file formatted with or without its optional TEST header and SELECT 1.""" + """A test or scenario file formatted with or without its optional trailing SELECT 1.""" description: str relative_path: str diff --git a/tests/unit/src/sqlbuild/lint/test_run_lint.py b/tests/unit/src/sqlbuild/lint/test_run_lint.py index e714c105cb..5cd6a93000 100644 --- a/tests/unit/src/sqlbuild/lint/test_run_lint.py +++ b/tests/unit/src/sqlbuild/lint/test_run_lint.py @@ -345,16 +345,16 @@ def test_given_commented_body_when_formatting_then_comment_is_preserved_in_canon "test_case", [ FixtureCeremonyFormatTestCase( - description="headerless test without a trailing select stays without one", + description="test without a trailing select stays without one", relative_path="tests/unit/test_orders.sql", - contents=f"{_FIXTURE_CTES}\n", - expected_contents=f"{_FORMATTED_FIXTURE_CTES}\n", + contents=f"TEST();\n{_FIXTURE_CTES}\n", + expected_contents=f"TEST();\n{_FORMATTED_FIXTURE_CTES}\n", ), FixtureCeremonyFormatTestCase( description="semicolon and trailing comment after the last cte are kept", relative_path="tests/unit/test_orders.sql", - contents=f"{_FIXTURE_CTES}; -- done\n", - expected_contents=f"{_FORMATTED_FIXTURE_CTES}; -- done\n", + contents=f"TEST();\n{_FIXTURE_CTES}; -- done\n", + expected_contents=f"TEST();\n{_FORMATTED_FIXTURE_CTES}; -- done\n", ), FixtureCeremonyFormatTestCase( description="explicit header and trailing select are kept", diff --git a/website/examples/waffle-shop/tests/unit/test_fact_orders.sql b/website/examples/waffle-shop/tests/unit/test_fact_orders.sql index ff2bfd0ddd..83ba4e4fa2 100644 --- a/website/examples/waffle-shop/tests/unit/test_fact_orders.sql +++ b/website/examples/waffle-shop/tests/unit/test_fact_orders.sql @@ -1,3 +1,5 @@ +TEST(); + WITH __ref__stg_orders AS ( SELECT diff --git a/website/examples/waffle-shop/tests/unit/test_stg_orders.sql b/website/examples/waffle-shop/tests/unit/test_stg_orders.sql index 127563fcb1..01eef8980d 100644 --- a/website/examples/waffle-shop/tests/unit/test_stg_orders.sql +++ b/website/examples/waffle-shop/tests/unit/test_stg_orders.sql @@ -1,3 +1,5 @@ +TEST(); + WITH __source__raw__orders AS ( SELECT diff --git a/website/src/content/docs/docs/concepts/declaration-scopes/visibility.mdx b/website/src/content/docs/docs/concepts/declaration-scopes/visibility.mdx index 62cf7a140c..39ebe51613 100644 --- a/website/src/content/docs/docs/concepts/declaration-scopes/visibility.mdx +++ b/website/src/content/docs/docs/concepts/declaration-scopes/visibility.mdx @@ -100,6 +100,8 @@ A test first sees declarations available from its own folder tree. It may also u macros, enums, and constants available to a model for which it defines expected output. ```sql +TEST(); + WITH __expected__orders AS ( SELECT diff --git a/website/src/content/docs/docs/concepts/macros.mdx b/website/src/content/docs/docs/concepts/macros.mdx index 37f8683e46..b6f389f70d 100644 --- a/website/src/content/docs/docs/concepts/macros.mdx +++ b/website/src/content/docs/docs/concepts/macros.mdx @@ -168,6 +168,8 @@ def mock_orders(count: int = 1) -> str: ``` ```sql +TEST(); + WITH __source__raw__orders AS ( @mock_orders(3) diff --git a/website/src/content/docs/docs/concepts/testing.mdx b/website/src/content/docs/docs/concepts/testing.mdx index 1170054feb..0c64348c3a 100644 --- a/website/src/content/docs/docs/concepts/testing.mdx +++ b/website/src/content/docs/docs/concepts/testing.mdx @@ -13,6 +13,8 @@ A test file defines mock inputs and expected outputs using CTEs. SQLBuild substi ```sql -- tests/unit/test_stg_orders.sql +TEST(); + WITH __source__raw__orders AS ( SELECT @@ -57,11 +59,12 @@ that read them. Statically provable collection-versus-scalar type conflicts iden column and suggest explicit `CAST`, `ARRAY_CONSTRUCT`, or `PARSE_JSON` expressions. SQLBuild never invents missing values. -A test file needs no header and no closing statement: a file that holds one test can be just its -CTEs. Add a `TEST (...)` header to name the test, set a cursor window, declare parameters, or turn -off SQL analysis, and on every block of a file that holds several tests. The `TEST();` header and -the trailing `SELECT 1` that earlier releases required are still accepted, so this file is -equivalent to the first example: +Every test file starts with a `TEST()` header. All of its fields are optional: `TEST();` is a +complete header, and a test without a `name` is named after its file. Add fields to name the test, +set a cursor window, declare parameters, or turn off SQL analysis. + +A test ends after its last CTE. The trailing `SELECT 1` that earlier releases required is still +accepted but no longer needed, so this file is equivalent to the first example: ```sql TEST(); @@ -116,6 +119,8 @@ expected rows can therefore live in one helper that both an `__expected__` CTE a read: ```sql +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 3 AS quantity UNION ALL SELECT 2 AS id, 5 AS quantity @@ -143,6 +148,8 @@ Tests can span multiple models in a single file. Mock your sources, define an ex ```sql -- Mock two sources, assert on the final mart. -- stg_orders and stg_payments resolve automatically from their real SQL. +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 100 AS customer_id, 2 AS waffle_type_id, 3 AS quantity, @@ -176,6 +183,8 @@ dependency errors. You can mock models directly with `__ref__` and seeds with `__seed__`, not just sources. This skips the model's real SQL (or the seed's real CSV data) and provides controlled data instead: ```sql +TEST(); + WITH __ref__stg_orders AS ( SELECT @@ -246,6 +255,8 @@ expressions, with the fixture relation. The mocked function is not deployed as a ```sql -- models/customer_order_totals.sql reads: -- __table_fn("customer_orders")(42) +TEST(); + WITH fixture_orders AS ( SELECT 101 AS order_id, 42 AS customer_id, 2500 AS amount_cents @@ -271,6 +282,8 @@ compares the real function. A single test can assert on multiple models. SQLBuild resolves and compares each one independently: ```sql +TEST(); + WITH __source__raw__orders AS ( SELECT 1 AS id, 100 AS customer_id, 2 AS waffle_type_id, 3 AS quantity, @@ -306,6 +319,8 @@ Only explicit expected CTEs create grants. A matching test filename, `__ref__` m Because unit tests are written in SQL, they support macro calls. This lets you write reusable mock generators instead of copy-pasting mock data across test files: ```sql +TEST(); + WITH __source__raw__orders AS ( @mock_orders() @@ -337,6 +352,8 @@ with `D013`. When a model uses macros that you want to control in tests (e.g. target-specific logic, dynamic SQL generation), you can override their output with `__macro__` CTEs: ```sql +TEST(); + WITH __macro__country_filter AS ( SELECT 'country_code = ''US''' @@ -363,6 +380,8 @@ This is useful for: Unit tests can include `__assert__` CTEs for property-based checks. An assertion passes if the query returns zero rows - any returned rows are failing examples. ```sql +TEST(); + WITH __ref__stg_orders AS ( SELECT 1 AS order_id, 100 AS customer_id, 3 AS quantity, @@ -430,8 +449,9 @@ project file; see [Rule options](/docs/concepts/rules/configuration-and-selectio Model tests mock sources and refs and compare model outputs. Three more modes test reusable logic directly, without a model chain. SQLBuild infers the mode from the test's CTEs: a test that defines `__macro_actual__` is a macro test, `__udf_actual__` a UDF test, `__table_fn_actual__` a table -function test, and any other test a model test. `TEST (mode ...)` states the mode explicitly; an -explicit mode that contradicts the CTEs is a compile error. +function test, and any other test a model test, so the header's `mode` field is optional. +`TEST (mode ...)` states the mode explicitly; an explicit mode that contradicts the CTEs is a +compile error. ### Macro tests @@ -520,11 +540,10 @@ CTE prefixes from other modes are not allowed. For example, `__source__` in a ma ## Multiple tests per file A single test file can contain multiple `TEST()` blocks. A test's name is its `name`, or the file -stem when `name` or the whole header is omitted. Test names are globally unique: a test cannot share -a name with another test, a scenario, a model, a source, a seed, a function, or a Python node. Name -the file or the block after the behavior it checks, such as `orders_status_rules`, rather than after -the model. The cases of one parameterized test share its name. Every block in a multi-test file needs -its own `TEST (...)` header with a unique `name`: +stem when `name` is omitted. Test names are globally unique: a test cannot share a name with another +test, a scenario, a model, a source, a seed, a function, or a Python node. Name the file or the block +after the behavior it checks, such as `orders_status_rules`, rather than after the model. The cases +of one parameterized test share its name. Each block in a file must have its own `name`: ```sql TEST (name "completed_orders_only"); @@ -548,8 +567,7 @@ __expected__stg_orders AS ( ) ``` -A file with a single test can omit the `name` field, or the whole header. Files with multiple tests -require a header with a name on every block. +A file with a single test can omit the `name` field. Files with multiple tests require names on every block. ## Repeating test logic @@ -645,6 +663,8 @@ capture and replay identity remains explicit. When independent status is unnecessary, a normal SQL case table remains concise: ```sql +TEST(); + WITH cases(case_name, source_status, expected_status) AS ( VALUES ('completed', 'completed', 'completed'),