diff --git a/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md b/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md index a35c616793..fcbb1c481b 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md @@ -58,11 +58,11 @@ JOIN __ref("stg_payments") p USING (order_id) than guessing how pieces connect. 2. **Look at the data.** `sqb query "SELECT ..." --limit 20` runs SQL on the active target. Profile inputs (counts, nulls, key uniqueness) before writing joins. -3. **Compile.** `sqb compile --select ` after every edit. It is offline and fast. -4. **Plan.** `sqb plan --select ` shows exactly what a build will do and why. Read it before +3. **Compile.** `sqb compile ` after every edit. It is offline and fast. +4. **Plan.** `sqb plan ` shows exactly what a build will do and why. Read it before building, especially for incremental models, full refreshes and anything destructive. -5. **Test.** `sqb test --select ` runs SQL unit tests without building. -6. **Build.** `sqb build --select ` runs tests, builds, and runs audits in DAG order. +5. **Test.** `sqb test ` runs SQL unit tests without building. +6. **Build.** `sqb build ` runs tests, builds, and runs audits in DAG order. Use `--defer-to ` to read unselected upstream models from another target instead of rebuilding them. 7. **Verify.** Prove the result is right with `sqb diff` (below), not only that it built. @@ -79,13 +79,13 @@ the trigger applies. |---|---| | Check a refactor did not change results, or compare dev with prod | `sqb diff prod:dev --full --select ` | | Compare two arbitrary queries (old vs new logic, before vs after a fix, two tables) | `sqb diff --left-query "..." --right-query "..." --key ` | -| Know what a model can see: which macros, enums and constants are visible, used, or unavailable | `sqb scope model:` | -| Fix "unknown macro/enum/constant", or decide where a declaration should live | `sqb scope model: --explain macro:` | -| Preview whether moving a file breaks declaration visibility | `sqb scope model: --as-path ` | +| Know what a model can see: which macros, enums and constants are visible, used, or unavailable | `sqb scope ` | +| Fix "unknown macro/enum/constant", or decide where a declaration should live | `sqb scope --explain macro:` | +| Preview whether moving a file breaks declaration visibility | `sqb scope --as-path ` | | Rename a model, move a model file, or rename a column | `sqb rename `, `sqb mv `, `sqb rename . ` (`model:`/`column:` prefixes optional) | | Find what a model depends on, or what breaks if it changes | `sqb lineage --direction both` | | Trace where a column comes from, or who consumes it before renaming or dropping it | `sqb lineage . --direction downstream` | -| See which intermediate models a SQL test will really execute | `sqb test --select --inspect` | +| See which intermediate models a SQL test will really execute | `sqb test --inspect` | | Inspect the full graph as data | `sqb dag --json`, `sqb plan --json`, `sqb compile --json` | | Adopt an existing warehouse table's columns into a contract | `sqb contract generate --from --select ` | | Try syntax safely in a throwaway DuckDB project | `sqb playground /tmp/sqb-sandbox` | @@ -167,7 +167,7 @@ SQLBuild models downstream of them. ## Selectors -`--select a b` unions, `--exclude` subtracts. `+model` includes upstream, `model+` downstream. +`sqb build a b` (or `--select a b`) unions, `--exclude` subtracts. `+model` includes upstream, `model+` downstream. `tag:`, `path:models/marts` (or any value containing `/`), `seed:`, `source:`, and quoted name globs such as `"stg_*"`. They work with plan, build, test, audit, seed, clone and diff. diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/audit.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/audit.md index 964152ccd2..d859ce83aa 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/audit.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/audit.md @@ -11,15 +11,18 @@ Runs all attached audits without rebuilding models. Useful for verifying data qu ## Usage ```bash -sqb --project-dir audit [flags] +sqb audit [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | | `--defer-to` | Resolve model references against another target | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select audits attached to specific models | | `--select-file` | Read selectors from a file | | `--exclude` | Exclude audits attached to specific models | @@ -62,7 +65,7 @@ guarantee immediate cancellation of a query already running in the warehouse. sqb audit # Run audits for marts only -sqb audit --select path:models/marts +sqb audit path:models/marts # Run up to 8 audit queries concurrently sqb audit --concurrency 8 diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md index 6290b31341..11886b7e12 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md @@ -25,13 +25,17 @@ Compiles, plans, and executes the selected build lifecycle, running the full sel ## Usage ```bash -sqb --project-dir build [flags] +sqb build [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. + ## Global options -These options apply to every `sqb` command and go before the command name, as in -`sqb --debug build`: +These options apply to every `sqb` command. `--project-dir`, `--no-color`, and `--debug` go +before or after the command name, as in `sqb --debug build` or `sqb build --debug`. Under +`sqb dbt`, put `--project-dir` before `dbt`: after it, `--project-dir` is passed to dbt as dbt's +own option. `--version` works only directly after `sqb`, as in `sqb --version`: | Option | Description | |--------|-------------| @@ -56,6 +60,8 @@ These options apply to every `sqb` command and go before the command name, as in | `--fail-fast` | Stop on first failure and skip remaining nodes | | `--concurrency` | Number of worker connections (default: 1) | | `--verbose`, `-v` | Show lifecycle SQL inline after each model | +| `--start-cursor` | Override the start cursor, typed from the selected incremental models' cursor type; fails with `S303` if the selection mixes timestamp and integer cursors. With no cursor model selected, an integer value is read as an integer cursor and anything else as an ISO timestamp | +| `--end-cursor` | Override the end cursor, typed the same way as `--start-cursor` | | `--start-cursor-ts` | Override start cursor for timestamp incremental models (ISO format) | | `--end-cursor-ts` | Override end cursor for timestamp incremental models (ISO format) | | `--start-cursor-int` | Override start cursor for integer incremental models | @@ -65,6 +71,7 @@ These options apply to every `sqb` command and go before the command name, as in | `--reload` | Reload managed sources (passes `is_reload=True` to loaders) | | `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](../concepts/planning/selection-and-staleness.md) | | `--manifest` | Generate `target/manifest.json` with plan-aware project metadata | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific models | | `--exclude` | Exclude specific models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | @@ -132,7 +139,7 @@ statement or step still prints its `FAIL` row. Use `--defer-to` to resolve unselected model references against another target. This lets you build a subset of models in dev while referencing production tables for everything else: ```bash -sqb build --select fact_orders --defer-to prod +sqb build fact_orders --defer-to prod ``` No `manifest.json` is required. `--defer-to` selects only the namespace used for unselected diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/check.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/check.md index befd47345e..967cadc701 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/check.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/check.md @@ -11,13 +11,16 @@ Runs Python [checks](../concepts/python-nodes/checks.md) in isolation. Checks va ## Usage ```bash -sqb --project-dir check [flags] +sqb check [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select checks to run (by name, `check:`, `tag:`, or graph expansion) | | `--exclude` | Exclude checks from the selection | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | @@ -35,10 +38,10 @@ Selecting a non-check node is rejected; use [`sqb build`](build.md) to run tasks sqb check # Run one check and its required dependencies -sqb check --select +check_orders_exported +sqb check +check_orders_exported # Run checks by tag -sqb check --select tag:exports +sqb check tag:exports # Print results as JSON sqb check --json diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/compile.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/compile.md index 966b45dbd6..7c89acd0ad 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/compile.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/compile.md @@ -23,9 +23,11 @@ Discovers the complete project, resolves references, expands macros, validates S ## Usage ```bash -sqb --project-dir compile [flags] +sqb compile [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | @@ -36,6 +38,7 @@ sqb --project-dir compile [flags] | `--json` | Output the full compile report as JSON | | `--manifest` | Generate `target/manifest.json` with project metadata | | `--lineage-mode` | Column lineage mode: `fast` (default), `rich` (slower, more detail), or `none` | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Analyze and report selected resources | | `--select-file` | Read selectors from a file, one selector per line | | `--exclude` | Remove resources from the selected scope | @@ -58,7 +61,7 @@ Use normal selectors to limit expensive analysis and reporting while retaining c discovery and reference integrity: ```bash -sqb compile --select fact_orders daily_revenue +sqb compile fact_orders daily_revenue sqb compile --select-file selected-models.txt sqb compile --exclude deprecated_model ``` @@ -284,7 +287,7 @@ sqb compile --lineage-mode none sqb compile --manifest # Compile only selected models -sqb compile --select fact_orders daily_revenue +sqb compile fact_orders daily_revenue # Disable SQL analysis sqb compile --no-sql-analysis diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/freshness.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/freshness.md index 97ae4958ac..b8a2788104 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/freshness.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/freshness.md @@ -22,9 +22,11 @@ Observes the current data version of each source and reports whether the data ha ## Usage ```bash -sqb --project-dir freshness [flags] +sqb freshness [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | @@ -35,6 +37,7 @@ sqb --project-dir freshness [flags] | `--json` | Output as JSON instead of human-readable text | | `--json-output` | Write JSON output to a file path (also prints text to stdout unless `--json` is set) | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific sources or models (sources upstream of selected models are included) | | `--exclude` | Exclude specific sources or models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | @@ -48,10 +51,10 @@ Without `--select`, all sources in the project are observed. When `--select` is sqb freshness # Observe a specific source -sqb freshness --select raw_orders +sqb freshness raw_orders # Observe sources upstream of a model -sqb freshness --select fact_orders +sqb freshness fact_orders ``` ## Observation without state diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/lineage.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/lineage.md index 5efed973e4..a36f972e1b 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/lineage.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/lineage.md @@ -41,6 +41,7 @@ Exactly one of a positional target or `--select` is required. | `--direction` | `upstream` (default), `downstream`, or `both`. `both` is only available for model lineage. | | `--depth` | How many hops to traverse. An integer or `all` (default: `all`). | | `--format` | Output format: `tree` (default), `list`, or `json`. | +| `--json` | Shorthand for `--format json`. | | `--mode` | Column lineage analysis mode: `rich` (default) or `fast`. | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias). | | `--select`, `-s` | Select resources using the same selector syntax as `sqb compile`, `sqb plan` and `sqb build`, for example `path:models/marts`, `tag:finance` or `stg_*`. | @@ -101,7 +102,7 @@ source:raw__payments -> model:stg_payments Structured output with nodes, edges, and metadata: ```bash -sqb lineage fact_orders --format json +sqb lineage fact_orders --json ``` ```json @@ -173,7 +174,7 @@ raw__payments.amount_cents -> stg_payments.amount_cents direct ### JSON ```bash -sqb lineage fact_orders.total_cents --format json +sqb lineage fact_orders.total_cents --json ``` ```json @@ -240,7 +241,7 @@ sqb lineage --select "stg_orders~daily_activity_rollup" sqb lineage --select "+fact_orders" # JSON output for programmatic consumption -sqb lineage fact_orders --format json --direction upstream +sqb lineage fact_orders --json --direction upstream ``` ## Depth limiting diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/load.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/load.md index 4f7f514996..a991f7df01 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/load.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/load.md @@ -22,6 +22,8 @@ sqb --project-dir load [flags] | `--exclude` | Exclude specific sources or loaders | | `--reload` | Pass `is_reload=True` to loader functions | | `--concurrency` | Number of worker connections (default: from settings or 1) | +| `--start-cursor` | Override the start cursor; an integer value sets `start_cursor_int`, anything else is read as an ISO timestamp and sets `start_cursor_ts` | +| `--end-cursor` | Override the end cursor, typed the same way as `--start-cursor` | | `--start-cursor-ts` | Override start cursor for timestamp-based loaders (ISO format) | | `--end-cursor-ts` | Override end cursor for timestamp-based loaders (ISO format) | | `--start-cursor-int` | Override start cursor for integer-based loaders | @@ -47,7 +49,7 @@ sqb load --concurrency 4 sqb load --reload # Override cursor bounds -sqb load --start-cursor-ts "2026-05-01T00:00:00" +sqb load --start-cursor "2026-05-01T00:00:00" ``` ## Execution order diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/plan.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/plan.md index 47640cc24e..808e85b2f5 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/plan.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/plan.md @@ -19,9 +19,11 @@ Shows the execution plan without making any changes. Useful for inspecting chang ## Usage ```bash -sqb --project-dir plan [flags] +sqb plan [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | @@ -34,10 +36,13 @@ sqb --project-dir plan [flags] | `--json` | Output the plan as JSON | | `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](../concepts/planning/selection-and-staleness.md) | | `--full-refresh` | Plan a full rebuild for selected models unless a model sets `full_refresh false`; `full_refresh true` forces a model without this flag | +| `--start-cursor` | Override the start cursor, typed from the selected incremental models' cursor type; fails with `S303` if the selection mixes timestamp and integer cursors. With no cursor model selected, an integer value is read as an integer cursor and anything else as an ISO timestamp | +| `--end-cursor` | Override the end cursor, typed the same way as `--start-cursor` | | `--start-cursor-ts` | Override start cursor for timestamp incremental models (ISO format) | | `--end-cursor-ts` | Override end cursor for timestamp incremental models (ISO format) | | `--start-cursor-int` | Override start cursor for integer incremental models | | `--end-cursor-int` | Override end cursor for integer incremental models | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific models | | `--exclude` | Exclude specific models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/query.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/query.md index ce42aee3e7..6b3b907b0d 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/query.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/query.md @@ -6,6 +6,14 @@ Online: https://sqlbuild.com/docs/cli/query/ +## Contents + +- Usage +- Flags +- Output formats +- Errors +- Examples + Execute ad hoc SQL against the active project connection. Useful for inspecting data, debugging models, or running one-off queries without leaving the SQLBuild CLI. ## Usage @@ -29,6 +37,7 @@ File paths are resolved from the current working directory. | `sql` | SQL to execute (positional argument) | | `--file` | Read SQL from a file instead of the command line | | `--format` | Output format: `long` (default), `table`, `json`, or `csv` | +| `--json` | Shorthand for `--format json` | | `--limit` | Maximum rows to return (default: 20) | | `--no-limit` | Disable the row limit | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | @@ -101,7 +110,7 @@ sqb query "SELECT * FROM dev.fact_orders" sqb query "SELECT * FROM dev.dim_customers" --format table --limit 50 # Export to JSON -sqb query "SELECT * FROM dev.daily_revenue" --format json --no-limit +sqb query "SELECT * FROM dev.daily_revenue" --json --no-limit # Run SQL from a file sqb query --file debug_query.sql diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md index 762d5b6339..36907ac377 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/scope.md @@ -43,9 +43,10 @@ sqb scope --at PATH [flags] sqb scope TARGET --as-path PATH [flags] ``` -`TARGET` is either a kind-qualified identity or an exact project-relative resource path: +`TARGET` is a model, source, seed, or function name, a kind-qualified identity, or an exact project-relative resource path: ```bash +sqb scope stg_orders sqb scope model:stg_orders sqb scope test:stg_orders__excludes_cancelled sqb scope scenario:daily_revenue @@ -61,7 +62,7 @@ sqb scope constant:minimum_order_value sqb scope models/staging/orders/stg_orders.sql ``` -Bare names such as `stg_orders` and `order_status` are rejected even if they are currently unique. Qualification keeps commands stable when names collide across resource or declaration kinds. Public declaration identities are `macro:`, `enum:`, and `constant:`. +Models, sources, seeds, and functions share one project-wide namespace, so their bare names, such as `stg_orders`, are accepted and resolve to the qualified identity. Macros, enums, constants, tests, and audits can share names with each other and with resources, so they always need their prefix: a bare name such as `order_status` that names one of them fails with `C960` and shows the qualified form to use, and a bare name that matches both a resource and one of these declarations fails and lists every match. Public declaration identities are `macro:`, `enum:`, and `constant:`. Model-private constants and enums include their owner in the introspection identity: diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/seed.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/seed.md index b06457fd5a..9ba3c4fae3 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/seed.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/seed.md @@ -11,13 +11,16 @@ Loads seed CSV files into the warehouse as tables. Seeds are fully replaced on e ## Usage ```bash -sqb --project-dir seed [flags] +sqb seed [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific seeds by name | | `--exclude` | Exclude specific seeds | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | @@ -29,5 +32,5 @@ sqb --project-dir seed [flags] sqb seed # Load a specific seed -sqb seed --select seed:waffle_types +sqb seed seed:waffle_types ``` diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md index a4a2b7d79f..909ceb1c74 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/test.md @@ -12,14 +12,17 @@ building models. Useful for validating test logic independently. ## Usage ```bash -sqb --project-dir test [flags] +sqb test [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select tests targeting specific models | | `--exclude` | Exclude tests targeting specific models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | @@ -39,5 +42,5 @@ are strings in JSON so their scale is preserved. sqb test # Run tests for a specific model -sqb test --select stg_orders +sqb test stg_orders ``` diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/selectors.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/selectors.md index e96f34019b..0b1373b684 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/selectors.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/selectors.md @@ -21,12 +21,13 @@ Selectors let you scope commands to specific subsets of your project. They work ## Basic usage ```bash +sqb build daily_revenue +sqb build daily_revenue customer_status_snapshot sqb build --select daily_revenue -sqb build --select daily_revenue customer_status_snapshot sqb build --exclude stg_customers ``` -`--select` (or `-s` for short) accepts one or more names. Multiple values are unioned. Space-separated names within one `--select` are also unioned. `--exclude` subtracts from the selected set. +Selectors can be written as positional arguments on `compile`, `plan`, `build`, `test`, `seed`, `freshness`, `audit`, and `check`, or with `--select` (or `-s` for short) on any command that selects. Both forms accept one or more names, can be combined, and are unioned. `--exclude` subtracts from the selected set. When no `--select` is provided, all models are selected. diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md index 5b7f08be63..827a81ca1a 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md @@ -198,10 +198,10 @@ The models and functions are no longer listed as first run, and the seed is curr ```bash # Rebuild a specific model -sqb build --select daily_revenue +sqb build daily_revenue # Rebuild models under a path -sqb build --select path:models/marts +sqb build path:models/marts # Full refresh of everything sqb build --full-refresh diff --git a/src/sqlbuild/cli/commands/_helpers/build_execution/execution.py b/src/sqlbuild/cli/commands/_helpers/build_execution/execution.py index f0f64cda94..b1fde65ad6 100644 --- a/src/sqlbuild/cli/commands/_helpers/build_execution/execution.py +++ b/src/sqlbuild/cli/commands/_helpers/build_execution/execution.py @@ -32,6 +32,9 @@ from sqlbuild.cli.progress.main._write_execution_header import write_execution_header from sqlbuild.compiler.pipeline.main.project_relation_targets import build_project_relation_targets from sqlbuild.compiler.pipeline.models import CompilePipelineResult +from sqlbuild.compiler.planner.main.cursor_window.typed_cursor_overrides import ( + typed_cursor_overrides, +) from sqlbuild.compiler.planner.models import CursorOverrides from sqlbuild.executor.build.models import ( BuildCallbacks, @@ -97,7 +100,11 @@ def prepare_build_execution( blank_line_after_complete=True, use_color=invocation.use_color, ) - cursor_overrides: CursorOverrides = request.cursor_overrides or CursorOverrides() + cursor_overrides: CursorOverrides = ( + pipeline_result.plan_output.cursor_overrides + or typed_cursor_overrides(cursor_overrides=request.cursor_overrides) + or CursorOverrides() + ) preparation: BuildExecutionPreparation = BuildExecutionPreparation( callbacks=callbacks, effective_concurrency=effective_concurrency, diff --git a/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py b/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py index 8e8698cf27..4cff7ca407 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py @@ -131,6 +131,8 @@ def _dispatch_cli_command(*, args: CliNamespace, handlers: CliEntrypointHandlers end_ts=args.end_cursor_ts, start_int=args.start_cursor_int, end_int=args.end_cursor_int, + start=args.start_cursor, + end=args.end_cursor, ), json_output=args.json, full_refresh=args.full_refresh, @@ -169,6 +171,8 @@ def _dispatch_cli_command(*, args: CliNamespace, handlers: CliEntrypointHandlers end_ts=args.end_cursor_ts, start_int=args.start_cursor_int, end_int=args.end_cursor_int, + start=args.start_cursor, + end=args.end_cursor, ), no_color=args.no_color, fail_fast=args.fail_fast, @@ -279,6 +283,8 @@ def _dispatch_cli_command(*, args: CliNamespace, handlers: CliEntrypointHandlers end_ts=args.end_cursor_ts, start_int=args.start_cursor_int, end_int=args.end_cursor_int, + start=args.start_cursor, + end=args.end_cursor, ), cli_vars=args.vars, json_output=args.json, diff --git a/src/sqlbuild/cli/commands/_helpers/entry/parser.py b/src/sqlbuild/cli/commands/_helpers/entry/parser.py index 80a6e37819..3360e7ccd7 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/parser.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/parser.py @@ -14,6 +14,7 @@ add_execution_args, add_execution_json_output_arg, add_microbatch_limit_override_arg, + add_positional_select_arg, add_scenario_snapshot_safety_args, add_select_args, add_vars_args, @@ -23,6 +24,7 @@ COLUMN_LINEAGE_MODE_VALUES, COMMAND_WAREHOUSE_GROUPS, COMPILE_LINEAGE_MODE_VALUES, + JSON_OUTPUT_FORMAT, PLAYGROUND_TEMPLATE_VALUES, ) from sqlbuild.cli.commands.types import CliCommand, CompileLineageMode @@ -51,14 +53,7 @@ def build_cli_parser(*, use_color: bool = False) -> argparse.ArgumentParser: action="version", version=f"sqb {_installed_version()}", ) - parser.add_argument("--project-dir", "--sqb-project-dir", dest="project_dir", default=None) - parser.add_argument("--no-color", action="store_true", default=False) - parser.add_argument( - "--debug", - action="store_true", - default=False, - help="enable verbose output and mirror internal diagnostics to stderr", - ) + _add_global_args(parser=parser, suppress_defaults=False) subparsers: argparse._SubParsersAction[argparse.ArgumentParser] = parser.add_subparsers( dest="command", @@ -77,9 +72,58 @@ def build_cli_parser(*, use_color: bool = False) -> argparse.ArgumentParser: _add_skills_parsers(subparsers) _add_rules_parser(subparsers) _add_warehouse_args(subparsers) + _add_global_args_to_subcommands(subparsers) return parser +def _add_global_args(*, parser: argparse.ArgumentParser, suppress_defaults: bool) -> None: + """Add flags accepted before or after the subcommand; subcommand copies suppress defaults.""" + + parser.add_argument( + "--project-dir", + "--sqb-project-dir", + dest="project_dir", + default=argparse.SUPPRESS if suppress_defaults else None, + help="SQLBuild project directory (default: the current directory)", + ) + parser.add_argument( + "--no-color", + action="store_true", + default=argparse.SUPPRESS if suppress_defaults else False, + help="disable colored output", + ) + parser.add_argument( + "--debug", + action="store_true", + default=argparse.SUPPRESS if suppress_defaults else False, + help="enable verbose output and mirror internal diagnostics to stderr", + ) + + +def _add_global_args_to_subcommands( + subparsers: argparse._SubParsersAction[argparse.ArgumentParser], +) -> None: + command_name: str + command_parser: argparse.ArgumentParser + for command_name, command_parser in subparsers.choices.items(): + if command_name == CliCommand.DBT: + continue + _add_global_args(parser=command_parser, suppress_defaults=True) + nested: argparse._SubParsersAction[argparse.ArgumentParser] + for nested in _nested_subparsers(command_parser): + _add_global_args_to_subcommands(nested) + + +def _nested_subparsers( + parser: argparse.ArgumentParser, +) -> list[argparse._SubParsersAction[argparse.ArgumentParser]]: + return [ + cast("argparse._SubParsersAction[argparse.ArgumentParser]", action) + for action in parser._actions + if isinstance(action, argparse._SubParsersAction) + ] + + def _add_warehouse_args( subparsers: argparse._SubParsersAction[argparse.ArgumentParser], ) -> None: @@ -94,11 +138,7 @@ def _add_warehouse_args( def _add_warehouse_arg_to_leaves(parser: argparse.ArgumentParser) -> None: - nested: list[argparse._SubParsersAction[argparse.ArgumentParser]] = [ - cast("argparse._SubParsersAction[argparse.ArgumentParser]", action) - for action in parser._actions - if isinstance(action, argparse._SubParsersAction) - ] + nested: list[argparse._SubParsersAction[argparse.ArgumentParser]] = _nested_subparsers(parser) if not nested: parser.add_argument( "--warehouse", @@ -239,6 +279,7 @@ def _add_compile_and_dag_parsers( help="Diagnostic: skip writing target/compiled artifacts", ) add_select_args(compile_parser) + _ = add_positional_select_arg(compile_parser) _ = add_vars_args(compile_parser) _ = add_dbt_config_args(parser=compile_parser) @@ -290,6 +331,7 @@ def _add_plan_and_build_parsers( _ = add_cursor_override_args(plan_parser) _ = add_microbatch_limit_override_arg(plan_parser) _ = add_select_args(plan_parser) + _ = add_positional_select_arg(plan_parser) _ = add_vars_args(plan_parser) _ = add_dbt_config_args(parser=plan_parser) @@ -328,6 +370,7 @@ def _add_plan_and_build_parsers( build_load_group.add_argument("--reload", dest="reload", action="store_true", default=False) _ = add_execution_args(build_parser) _ = add_select_args(build_parser) + _ = add_positional_select_arg(build_parser) _ = add_vars_args(build_parser) _ = add_dbt_config_args(parser=build_parser) @@ -344,6 +387,7 @@ def _add_quality_parsers( freshness_parser.add_argument("--fail-on-stale", action="store_true", default=False) _ = add_execution_json_output_arg(freshness_parser) _ = add_select_args(freshness_parser) + _ = add_positional_select_arg(freshness_parser) _ = add_vars_args(freshness_parser) _ = add_dbt_config_args(parser=freshness_parser) @@ -358,6 +402,7 @@ def _add_quality_parsers( ) _ = add_execution_json_output_arg(test_parser) _ = add_select_args(test_parser) + _ = add_positional_select_arg(test_parser) _ = add_vars_args(test_parser) _ = add_dbt_config_args(parser=test_parser) @@ -367,6 +412,7 @@ def _add_quality_parsers( check_parser.add_argument("--target", default=None) _ = add_execution_json_output_arg(check_parser) _ = add_select_args(check_parser) + _ = add_positional_select_arg(check_parser) _ = add_vars_args(check_parser) _ = add_dbt_config_args(parser=check_parser) @@ -378,6 +424,7 @@ def _add_quality_parsers( audit_parser.add_argument("--json", action="store_true", default=False) _ = add_execution_json_output_arg(audit_parser) _ = add_select_args(audit_parser) + _ = add_positional_select_arg(audit_parser) _ = add_vars_args(audit_parser) _ = add_dbt_config_args(parser=audit_parser) @@ -426,6 +473,7 @@ def _add_data_parsers( seed_parser.add_argument("--concurrency", type=int, default=None) _ = add_execution_json_output_arg(seed_parser) _ = add_select_args(seed_parser) + _ = add_positional_select_arg(seed_parser) _ = add_vars_args(seed_parser) clone_parser: argparse.ArgumentParser = subparsers.add_parser(CliCommand.CLONE) @@ -620,12 +668,16 @@ def _add_inspection_parsers( query_parser.add_argument("query_sql", nargs="?", metavar="sql") query_parser.add_argument("--file", dest="query_file", default=None) query_parser.add_argument("--target", default=None) - query_parser.add_argument( + query_format_group: argparse._MutuallyExclusiveGroup = ( + query_parser.add_mutually_exclusive_group() + ) + query_format_group.add_argument( "--format", dest="query_format", choices=("long", "table", "json", "csv"), default="long", ) + _add_json_format_alias(group=query_format_group, dest="query_format") query_parser.add_argument("--limit", dest="query_limit", type=int, default=20) query_parser.add_argument( "--no-limit", dest="query_no_limit", action="store_true", default=False @@ -641,12 +693,16 @@ def _add_inspection_parsers( "or one model.column", ) _add_sql_analysis_override(lineage_parser) - lineage_parser.add_argument( + lineage_format_group: argparse._MutuallyExclusiveGroup = ( + lineage_parser.add_mutually_exclusive_group() + ) + lineage_format_group.add_argument( "--format", dest="lineage_format", choices=("tree", "json", "list"), default="tree", ) + _add_json_format_alias(group=lineage_format_group, dest="lineage_format") lineage_parser.add_argument( "--include-uses", dest="lineage_include_uses", @@ -672,6 +728,16 @@ def _add_inspection_parsers( _ = add_vars_args(lineage_parser) +def _add_json_format_alias(*, group: argparse._MutuallyExclusiveGroup, dest: str) -> None: + group.add_argument( + "--json", + dest=dest, + action="store_const", + const=JSON_OUTPUT_FORMAT, + help="shorthand for --format json", + ) + + def _add_maintenance_parsers( subparsers: argparse._SubParsersAction[argparse.ArgumentParser], ) -> None: diff --git a/src/sqlbuild/cli/commands/_helpers/entry/parser_arguments.py b/src/sqlbuild/cli/commands/_helpers/entry/parser_arguments.py index bbf8b91063..0ac183d1bd 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/parser_arguments.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/parser_arguments.py @@ -36,10 +36,22 @@ def add_vars_args(parser: argparse.ArgumentParser) -> None: def add_cursor_override_args(parser: argparse.ArgumentParser) -> None: """Add cursor override flags to a subparser.""" - parser.add_argument("--start-cursor-ts", default=None) - parser.add_argument("--end-cursor-ts", default=None) - parser.add_argument("--start-cursor-int", default=None) - parser.add_argument("--end-cursor-int", default=None) + parser.add_argument( + "--start-cursor", + default=None, + metavar="VALUE", + help="override the start cursor, typed from the selected models' cursor type", + ) + parser.add_argument( + "--end-cursor", + default=None, + metavar="VALUE", + help="override the end cursor, typed from the selected models' cursor type", + ) + parser.add_argument("--start-cursor-ts", default=None, help="timestamp start cursor override") + parser.add_argument("--end-cursor-ts", default=None, help="timestamp end cursor override") + parser.add_argument("--start-cursor-int", default=None, help="integer start cursor override") + parser.add_argument("--end-cursor-int", default=None, help="integer end cursor override") def add_microbatch_limit_override_arg(parser: argparse.ArgumentParser) -> None: @@ -83,6 +95,18 @@ def add_select_args(parser: argparse.ArgumentParser) -> None: parser.add_argument("--exclude", nargs="+", action="extend", default=[]) +def add_positional_select_arg(parser: argparse.ArgumentParser) -> None: + """Accept selectors as positional arguments; they are combined with --select.""" + + parser.add_argument( + "positional_select", + nargs="*", + default=[], + metavar="SELECTOR", + help="selectors to include, the same as --select; combined with any --select values", + ) + + def add_dbt_config_args(*, parser: argparse.ArgumentParser, prefix: str = "dbt") -> None: """Add dbt configuration flags to a subparser.""" diff --git a/src/sqlbuild/cli/commands/_helpers/entry/parsing.py b/src/sqlbuild/cli/commands/_helpers/entry/parsing.py index 19053f1c49..56bc7f1221 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/parsing.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/parsing.py @@ -14,6 +14,7 @@ DEBUG_OPTION, EMPTY_ENV_VALUE, NO_COLOR_OPTION, + POSITIONAL_SELECT_COMMANDS, SCOPE_DEFAULT_PAGE_SIZE, SCOPE_GLOBAL_SUMMARY, SQLBUILD_CONCURRENCY_ENV_VAR, @@ -104,8 +105,15 @@ def parse_cli_invocation( continue dbt_passthrough_args.append(dbt_arg) args.dbt_args = dbt_passthrough_args - elif unknown_args: - parser.error(f"unrecognized arguments: {' '.join(unknown_args)}") + else: + if args.command in POSITIONAL_SELECT_COMMANDS: + args.select, unknown_args = _merge_positional_selectors( + positional=args.positional_select, + select=args.select, + unknown_args=unknown_args, + ) + if unknown_args: + parser.error(f"unrecognized arguments: {' '.join(unknown_args)}") args.verbose = args.verbose or args.debug if ( args.debug @@ -125,6 +133,8 @@ def parse_cli_invocation( args.concurrency = resolve_env_default_concurrency(args.concurrency) except argparse.ArgumentTypeError as error: parser.error(str(error)) + if args.command in {CliCommand.BUILD, CliCommand.LOAD, CliCommand.PLAN}: + _validate_cursor_override_args(args=args, parser=parser) if args.command == CliCommand.SCOPE: _validate_scope_args(args=args, parser=parser) if args.command == CliCommand.TEST: @@ -139,6 +149,33 @@ def parse_cli_invocation( return ParsedCliInvocation(args=args, exit_code=None) +def _merge_positional_selectors( + *, positional: list[str], select: list[str], unknown_args: list[str] +) -> tuple[list[str], list[str]]: + """Combine positional, trailing and --select selectors; return what is still unrecognised.""" + + if unknown_args and not any(arg.startswith("-") for arg in unknown_args): + return [*positional, *unknown_args, *select], [] + return [*positional, *select], unknown_args + + +def _validate_cursor_override_args(*, args: CliNamespace, parser: argparse.ArgumentParser) -> None: + has_untyped: bool = args.start_cursor is not None or args.end_cursor is not None + has_typed: bool = any( + value is not None + for value in ( + args.start_cursor_ts, + args.end_cursor_ts, + args.start_cursor_int, + args.end_cursor_int, + ) + ) + if has_untyped and has_typed: + parser.error( + "--start-cursor/--end-cursor cannot be combined with the -ts or -int cursor flags" + ) + + def _validate_scope_args(*, args: CliNamespace, parser: argparse.ArgumentParser) -> None: has_target: bool = args.scope_target is not None has_at: bool = args.scope_at is not None diff --git a/src/sqlbuild/cli/commands/_helpers/load/planning.py b/src/sqlbuild/cli/commands/_helpers/load/planning.py index a7ff6a2802..0fdae04c59 100644 --- a/src/sqlbuild/cli/commands/_helpers/load/planning.py +++ b/src/sqlbuild/cli/commands/_helpers/load/planning.py @@ -15,6 +15,9 @@ ) from sqlbuild.compiler.compile.main.effective_runtime import build_effective_runtime_config from sqlbuild.compiler.compile.main.effective_settings import build_effective_settings_config +from sqlbuild.compiler.planner.main.cursor_window.typed_cursor_overrides import ( + typed_cursor_overrides, +) from sqlbuild.compiler.planner.models import CursorOverrides from sqlbuild.compiler.references.main.render_source_relation import render_source_relation from sqlbuild.provider.main.session import build_provider_session @@ -74,7 +77,9 @@ def prepare_load_execution( target_name=target_name, effective_vars=effective_vars, run_id=run_id, - effective_cursor_overrides=request.cursor_overrides or CursorOverrides(), + effective_cursor_overrides=( + typed_cursor_overrides(cursor_overrides=request.cursor_overrides) or CursorOverrides() + ), effective_concurrency=effective_concurrency, provider_session=build_provider_session( discovered_providers=invocation.discovered_inputs.providers diff --git a/src/sqlbuild/cli/commands/_helpers/playground/templates/waffle_shop/README.md b/src/sqlbuild/cli/commands/_helpers/playground/templates/waffle_shop/README.md index a538e3f997..a0246bea43 100644 --- a/src/sqlbuild/cli/commands/_helpers/playground/templates/waffle_shop/README.md +++ b/src/sqlbuild/cli/commands/_helpers/playground/templates/waffle_shop/README.md @@ -16,6 +16,12 @@ Build the project locally with DuckDB: sqb build ``` +Build one model and everything downstream of it: + +```bash +sqb build stg_orders+ +``` + After the build creates the local DuckDB objects, run SQL unit tests: ```bash diff --git a/src/sqlbuild/cli/commands/_helpers/scope/command.py b/src/sqlbuild/cli/commands/_helpers/scope/command.py index 57223c228d..0e991874e2 100644 --- a/src/sqlbuild/cli/commands/_helpers/scope/command.py +++ b/src/sqlbuild/cli/commands/_helpers/scope/command.py @@ -8,6 +8,7 @@ from typing import TextIO from sqlbuild.cli.commands._helpers.scope.output import render_scope_result +from sqlbuild.cli.commands._helpers.scope.target import qualify_scope_target from sqlbuild.cli.commands.models import ScopeCommandRequest from sqlbuild.cli.commands.types import ScopeIndexLoader from sqlbuild.compiler.scopes.main.browse_scope_folders import browse_scope_folders @@ -42,6 +43,7 @@ def run_scope_command( project_dir: Path = request.project_dir if request.project_dir is not None else Path.cwd() index: ScopeIndex = load_scope_index(project_dir=project_dir, no_cache=request.no_cache) lookup: ScopeLookup = build_scope_lookup(index=index) + request = replace(request, target=qualify_scope_target(lookup=lookup, target=request.target)) filters: ScopeReportFilters = ScopeReportFilters( include_nearby=request.include_nearby, defined_under=request.defined_under, diff --git a/src/sqlbuild/cli/commands/_helpers/scope/target.py b/src/sqlbuild/cli/commands/_helpers/scope/target.py new file mode 100644 index 0000000000..502a77fb5f --- /dev/null +++ b/src/sqlbuild/cli/commands/_helpers/scope/target.py @@ -0,0 +1,84 @@ +"""Read a bare `sqb scope` target name the way `sqb lineage` and `sqb rename` read targets.""" + +from __future__ import annotations + +from sqlbuild.cli.commands._helpers.lineage.selection import resolve_lineage_target +from sqlbuild.cli.commands.exceptions import CliUserError +from sqlbuild.cli.commands.models import LineageTarget +from sqlbuild.compiler.compile.models import CompiledObjectKey +from sqlbuild.compiler.compile.types import CompiledResourceType +from sqlbuild.compiler.scopes.models import ResourceIdentity, ScopeLookup +from sqlbuild.compiler.scopes.types import ResourceKind + +_KIND_SEPARATOR: str = ":" +_PATH_MARKERS: tuple[str, ...] = ("/", "\\", ".") +_BARE_RESOURCE_TYPES: dict[ResourceKind, CompiledResourceType] = { + ResourceKind.MODEL: CompiledResourceType.MODEL, + ResourceKind.SOURCE: CompiledResourceType.SOURCE, + ResourceKind.SEED: CompiledResourceType.SEED, + ResourceKind.FUNCTION: CompiledResourceType.UDF, +} +_BARE_USAGE: str = ( + "bare names work for models, sources, seeds and functions; write macros, enums, constants, " + "tests and audits with their prefix, for example macro:" +) + + +def qualify_scope_target(*, lookup: ScopeLookup, target: str | None) -> str | None: + """Return `target` with its kind prefix when it is a bare model, source, seed or function.""" + + if target is None or _KIND_SEPARATOR in target or any(m in target for m in _PATH_MARKERS): + return target + bare_matches: dict[str, ResourceIdentity] = {} + identity: ResourceIdentity + for identity in lookup.resources: + if identity.kind in _BARE_RESOURCE_TYPES and identity.name == target: + bare_matches[str(identity.kind)] = identity + all_keys: dict[str, CompiledObjectKey] = { + identity.name: CompiledObjectKey( + resource_type=_BARE_RESOURCE_TYPES[identity.kind], name=identity.name + ) + for identity in bare_matches.values() + } + resolved: LineageTarget = resolve_lineage_target(all_keys=all_keys, target=target) + prefixed_matches: tuple[str, ...] = _prefix_required_matches(lookup=lookup, name=target) + if resolved.key is None: + if prefixed_matches: + raise CliUserError( + f"scope target '{target}' needs its prefix: {_or_list(prefixed_matches)}", + code="C960", + help=_BARE_USAGE, + ) + raise CliUserError(f"unknown scope target '{target}'", code="C960", help=_BARE_USAGE) + candidates: tuple[str, ...] = ( + *(f"{kind}:{target}" for kind in sorted(bare_matches)), + *prefixed_matches, + ) + if len(candidates) > 1: + raise CliUserError( + f"scope target '{target}' matches more than one resource or declaration; " + f"write {_or_list(candidates)}", + code="C960", + help=_BARE_USAGE, + ) + return candidates[0] + + +def _prefix_required_matches(*, lookup: ScopeLookup, name: str) -> tuple[str, ...]: + resources: set[str] = { + f"{identity.kind}:{name}" + for identity in lookup.resources + if identity.kind not in _BARE_RESOURCE_TYPES and identity.name == name + } + declarations: set[str] = { + f"{identity.kind}:{name}" + for identity in lookup.declarations + if identity.owner is None and identity.name == name + } + return tuple(sorted(resources | declarations)) + + +def _or_list(values: tuple[str, ...]) -> str: + if len(values) == 1: + return values[0] + return ", ".join(values[:-1]) + f" or {values[-1]}" diff --git a/src/sqlbuild/cli/commands/classes/cli_namespace.py b/src/sqlbuild/cli/commands/classes/cli_namespace.py index 929137fb5a..b07cb18c1c 100644 --- a/src/sqlbuild/cli/commands/classes/cli_namespace.py +++ b/src/sqlbuild/cli/commands/classes/cli_namespace.py @@ -40,6 +40,8 @@ "profile_skip_column_inference": False, "profile_skip_contracts": False, "profile_skip_write": False, + "start_cursor": None, + "end_cursor": None, "start_cursor_ts": None, "max_microbatches": None, "end_cursor_ts": None, @@ -130,6 +132,7 @@ "skills_target": [], "skills_force": False, "select": [], + "positional_select": [], "select_file": [], "exclude": [], "dbt_command": None, @@ -206,6 +209,8 @@ class CliNamespace: profile_skip_column_inference: bool profile_skip_contracts: bool profile_skip_write: bool + start_cursor: str | None + end_cursor: str | None start_cursor_ts: str | None max_microbatches: int | None end_cursor_ts: str | None @@ -301,6 +306,7 @@ class CliNamespace: skills_target: list[str] skills_force: bool select: list[str] + positional_select: list[str] select_file: list[str] exclude: list[str] dbt_command: str | None diff --git a/src/sqlbuild/cli/commands/constants.py b/src/sqlbuild/cli/commands/constants.py index 16a46bc828..a7392d5798 100644 --- a/src/sqlbuild/cli/commands/constants.py +++ b/src/sqlbuild/cli/commands/constants.py @@ -132,6 +132,18 @@ CLI_WAREHOUSE_SOURCE: str = "--warehouse" CONNECTION_WAREHOUSE_SOURCE: str = "connection warehouse" CONNECTION_DEFAULT_WAREHOUSE_SOURCE: str = "connection default" +POSITIONAL_SELECT_COMMANDS: frozenset[str] = frozenset( + { + CliCommand.AUDIT, + CliCommand.BUILD, + CliCommand.CHECK, + CliCommand.COMPILE, + CliCommand.FRESHNESS, + CliCommand.PLAN, + CliCommand.SEED, + CliCommand.TEST, + } +) COMMAND_WAREHOUSE_GROUPS: MappingProxyType[CliCommand, WarehouseGroup | None] = MappingProxyType( { CliCommand.AUDIT: WarehouseGroup.BUILD, diff --git a/src/sqlbuild/compiler/planner/_helpers/planning/output_assembly.py b/src/sqlbuild/compiler/planner/_helpers/planning/output_assembly.py index aa45292c50..5b8870b3da 100644 --- a/src/sqlbuild/compiler/planner/_helpers/planning/output_assembly.py +++ b/src/sqlbuild/compiler/planner/_helpers/planning/output_assembly.py @@ -67,7 +67,11 @@ def assemble_base_plan_output( planned_sql_tests=planned_sql_tests, ), ) - plan_output = replace(plan_output, source_freshness=source_freshness) + plan_output = replace( + plan_output, + source_freshness=source_freshness, + cursor_overrides=overrides.cursor_overrides, + ) if pruning.pruned_direct_model_names: plan_output = replace( plan_output, diff --git a/src/sqlbuild/compiler/planner/_helpers/resolve/cursor.py b/src/sqlbuild/compiler/planner/_helpers/resolve/cursor.py index f202904a34..9ddbbc9cca 100644 --- a/src/sqlbuild/compiler/planner/_helpers/resolve/cursor.py +++ b/src/sqlbuild/compiler/planner/_helpers/resolve/cursor.py @@ -2,12 +2,16 @@ from __future__ import annotations +import re +from collections.abc import Iterable from dataclasses import replace from datetime import datetime +from sqlbuild.compiler.compile.models import CompiledModel from sqlbuild.compiler.planner.exceptions import PlannerInputError from sqlbuild.compiler.planner.models import ( CursorBounds, + CursorOverrides, Duration, EmptyCursorInputDecision, MaximumStartPolicyInputs, @@ -17,6 +21,7 @@ CursorGrain, CursorType, CursorWatermarkMode, + MaterializationType, MicrobatchStrategy, ) from sqlbuild.cursor_algebra.constants import GRAIN_ORDER @@ -30,7 +35,10 @@ from sqlbuild.cursor_algebra.main.try_parse import try_parse from sqlbuild.cursor_algebra.models import DateValue, IntegerValue, TimestampValue from sqlbuild.cursor_algebra.types import BoundSentinel, CursorScalar +from sqlbuild.spec.contracts.main.get_config_str import get_config_str +_INTEGER_LITERAL: re.Pattern[str] = re.compile(r"[+-]?\d+") +_LISTED_MODEL_LIMIT: int = 3 _EMPTY_INTEGER_CURSOR_BOUND: int = 0 _EMPTY_TIMESTAMP_CURSOR_BOUND: datetime = datetime(1970, 1, 1) @@ -483,3 +491,66 @@ def _parse_required(*, value: str, cursor_type: str | None) -> CursorScalar: if parsed is None: raise CursorAlgebraError(f"invalid cursor value: {value}") return parsed + + +def type_cursor_overrides( + *, + cursor_overrides: CursorOverrides | None, + selected_models: Iterable[CompiledModel], +) -> CursorOverrides | None: + """Type `--start-cursor`/`--end-cursor` from the selected cursor models, else the literal.""" + + if cursor_overrides is None or not cursor_overrides.has_untyped: + return cursor_overrides + model_names_by_type: dict[CursorType, list[str]] = {} + model: CompiledModel + for model in selected_models: + cursor_type: CursorType | None = _model_cursor_type(model=model) + if cursor_type is not None: + model_names_by_type.setdefault(cursor_type, []).append(model.name) + if len(model_names_by_type) > 1: + raise PlannerInputError( + "--start-cursor and --end-cursor need one cursor type, but the selection mixes " + + " and ".join( + f"{cursor_type} cursor models ({_format_names(names)})" + for cursor_type, names in sorted(model_names_by_type.items()) + ), + code="S303", + help=( + "select models with one cursor type, or set each type explicitly with " + "--start-cursor-ts/--end-cursor-ts and --start-cursor-int/--end-cursor-int" + ), + ) + resolved_type: CursorType = ( + next(iter(model_names_by_type)) + if model_names_by_type + else _literal_cursor_type(cursor_overrides=cursor_overrides) + ) + return cursor_overrides.typed_as(cursor_type=resolved_type) + + +def _model_cursor_type(*, model: CompiledModel) -> CursorType | None: + materialized: str | None = get_config_str(values=model.config.values, key="materialized") + cursor_column: str | None = get_config_str(values=model.config.values, key="cursor") + if materialized != MaterializationType.INCREMENTAL or cursor_column is None: + return None + cursor_type: str | None = get_config_str(values=model.config.values, key="cursor_type") + if cursor_type not in {CursorType.TIMESTAMP, CursorType.INTEGER}: + return None + return CursorType(cursor_type) + + +def _literal_cursor_type(*, cursor_overrides: CursorOverrides) -> CursorType: + values: tuple[str, ...] = tuple( + value for value in (cursor_overrides.start, cursor_overrides.end) if value is not None + ) + if all(_INTEGER_LITERAL.fullmatch(value.strip()) for value in values): + return CursorType.INTEGER + return CursorType.TIMESTAMP + + +def _format_names(names: list[str]) -> str: + ordered: list[str] = sorted(names) + shown: str = ", ".join(ordered[:_LISTED_MODEL_LIMIT]) + hidden: int = len(ordered) - _LISTED_MODEL_LIMIT + return f"{shown} and {hidden} more" if hidden > 0 else shown diff --git a/src/sqlbuild/compiler/planner/main/cursor_window/typed_cursor_overrides.py b/src/sqlbuild/compiler/planner/main/cursor_window/typed_cursor_overrides.py new file mode 100644 index 0000000000..b33a43aec5 --- /dev/null +++ b/src/sqlbuild/compiler/planner/main/cursor_window/typed_cursor_overrides.py @@ -0,0 +1,21 @@ +"""Public typing of untyped `--start-cursor`/`--end-cursor` values.""" + +from __future__ import annotations + +from collections.abc import Iterable + +from sqlbuild.compiler.compile.models import CompiledModel +from sqlbuild.compiler.planner._helpers.resolve.cursor import ( + type_cursor_overrides, +) +from sqlbuild.compiler.planner.models import CursorOverrides + + +def typed_cursor_overrides( + *, + cursor_overrides: CursorOverrides | None, + selected_models: Iterable[CompiledModel] = (), +) -> CursorOverrides | None: + """Read `--start-cursor`/`--end-cursor` as the one cursor type of the selected models.""" + + return type_cursor_overrides(cursor_overrides=cursor_overrides, selected_models=selected_models) diff --git a/src/sqlbuild/compiler/planner/main/execution/execution.py b/src/sqlbuild/compiler/planner/main/execution/execution.py index 1606725a53..b58abab25e 100644 --- a/src/sqlbuild/compiler/planner/main/execution/execution.py +++ b/src/sqlbuild/compiler/planner/main/execution/execution.py @@ -4,11 +4,13 @@ import time from collections.abc import Callable +from dataclasses import replace from typing import Any from sqlbuild.adapter.contract.classes.base_adapter import BaseAdapter from sqlbuild.adapter.relations.main.open_inspection_catalog import open_inspection_catalog -from sqlbuild.compiler.compile.models import CompiledProject +from sqlbuild.compiler.compile.models import CompiledModel, CompiledProject +from sqlbuild.compiler.compile.types import CompiledResourceType from sqlbuild.compiler.planner._helpers.changes.actions import resolve_model_actions from sqlbuild.compiler.planner._helpers.changes.detect import detect_changes from sqlbuild.compiler.planner._helpers.planning.buildability import ( @@ -31,6 +33,9 @@ from sqlbuild.compiler.planner._helpers.planning.warehouse_state import ( gather_planner_warehouse_state, ) +from sqlbuild.compiler.planner._helpers.resolve.cursor import ( + type_cursor_overrides, +) from sqlbuild.compiler.planner._helpers.warehouse.source_freshness import ( build_planner_source_freshness_result, ) @@ -48,6 +53,7 @@ PlannerPolicies, PlannerResolvedActions, PlannerRuntime, + PlannerScope, PlannerScopePruningResult, PlannerScopeResolution, PlannerSelection, @@ -80,10 +86,12 @@ def build_execution_plan( local_config=local_config, on_progress=on_progress, ) - scopes: PlannerScopeResolution = resolve_planner_scopes( + scopes: PlannerScopeResolution + scopes, overrides = _resolve_scopes_and_cursor_overrides( project=project, selection=selection, policies=policies, + overrides=overrides, ) with ( BackgroundSqlTestPlanning( @@ -198,6 +206,33 @@ def build_execution_plan( return plan_output +def _resolve_scopes_and_cursor_overrides( + *, + project: CompiledProject, + selection: PlannerSelection, + policies: PlannerPolicies, + overrides: PlannerOverrides, +) -> tuple[PlannerScopeResolution, PlannerOverrides]: + scopes: PlannerScopeResolution = resolve_planner_scopes( + project=project, + selection=selection, + policies=policies, + ) + selected_scope: PlannerScope = scopes.selected_scope + selected_models: tuple[CompiledModel, ...] = tuple( + selected_scope.models_by_name[key.name] + for key in selected_scope.selected_keys + if key.resource_type == CompiledResourceType.MODEL + and key.name in selected_scope.models_by_name + ) + return scopes, replace( + overrides, + cursor_overrides=type_cursor_overrides( + cursor_overrides=overrides.cursor_overrides, selected_models=selected_models + ), + ) + + def _detect_planner_change_results( *, project: CompiledProject, diff --git a/src/sqlbuild/compiler/planner/models.py b/src/sqlbuild/compiler/planner/models.py index 6c15d2058f..775c00e55d 100644 --- a/src/sqlbuild/compiler/planner/models.py +++ b/src/sqlbuild/compiler/planner/models.py @@ -48,6 +48,7 @@ BackfillAction, ChangeKind, CursorGrain, + CursorType, CursorWatermarkMode, FixtureKey, GraphResourceKind, @@ -192,12 +193,14 @@ class GraphIdentityNode: @dataclass(frozen=True, kw_only=True, init=False) class CursorOverrides: - """Typed cursor override values from CLI flags.""" + """Cursor override values from CLI flags; untyped `start`/`end` await planner typing.""" start_ts: CursorScalar | None = None end_ts: CursorScalar | None = None start_int: CursorScalar | None = None end_int: CursorScalar | None = None + start: str | None = None + end: str | None = None def __init__( self, @@ -206,7 +209,11 @@ def __init__( end_ts: str | CursorScalar | None = None, start_int: str | CursorScalar | None = None, end_int: str | CursorScalar | None = None, + start: str | None = None, + end: str | None = None, ) -> None: + object.__setattr__(self, "start", start) + object.__setattr__(self, "end", end) object.__setattr__( self, "start_ts", @@ -224,6 +231,34 @@ def __init__( self, "end_int", self._parse_integer(field_name="--end-cursor-int", value=end_int) ) + @property + def has_untyped(self) -> bool: + """Return whether `--start-cursor` or `--end-cursor` still needs a cursor type.""" + + return self.start is not None or self.end is not None + + def typed_as(self, *, cursor_type: CursorType) -> CursorOverrides: + """Return these overrides with `--start-cursor`/`--end-cursor` read as `cursor_type`.""" + + if not self.has_untyped: + return self + if cursor_type == CursorType.INTEGER: + return CursorOverrides( + start_ts=self.start_ts, + end_ts=self.end_ts, + start_int=self._parse_integer(field_name="--start-cursor", value=self.start) + or self.start_int, + end_int=self._parse_integer(field_name="--end-cursor", value=self.end) + or self.end_int, + ) + return CursorOverrides( + start_ts=self._parse_timestamp(field_name="--start-cursor", value=self.start) + or self.start_ts, + end_ts=self._parse_timestamp(field_name="--end-cursor", value=self.end) or self.end_ts, + start_int=self.start_int, + end_int=self.end_int, + ) + @staticmethod def _parse_timestamp( *, field_name: str, value: str | CursorScalar | None @@ -1534,6 +1569,7 @@ class PlanOutput: source_freshness: DirectSourceFreshnessPlanningResult | None = None python_identity_fingerprints: dict[tuple[str, str], Fingerprint] = field(default_factory=dict) metadata: dict[str, object] = field(default_factory=dict) + cursor_overrides: CursorOverrides | None = None @property def python_source_entries(self) -> dict[str, SourceEntry]: diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/build/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/build/_test_types.py index 02b1451654..a62336785f 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/build/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/build/_test_types.py @@ -1305,3 +1305,13 @@ class SameRunEmptyInputsE2ETestCase: expected_json_empty_inputs: tuple[str, ...] | None expected_json_waiting_on_empty_inputs: bool | None expected_json_warnings: tuple[str, ...] + + +@dataclass(frozen=True) +class UntypedCursorOverrideSelectionE2ETestCase: + """Untyped cursor flags over a selection that does or does not mix cursor types.""" + + description: str + command: tuple[str, ...] + expected_exit_code: int + expected_output_fragments: tuple[str, ...] diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/build/test_append_cursor_build.py b/tests/e2e/src/sqlbuild/cli/commands/main/build/test_append_cursor_build.py index a41b2e1e6d..e9cd889830 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/build/test_append_cursor_build.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/build/test_append_cursor_build.py @@ -10,6 +10,7 @@ from tests.e2e.src.sqlbuild.cli.commands.main.build._test_types import ( AppendCursorBuildE2ETestCase, TimestampCursorBuildOverrideE2ETestCase, + UntypedCursorOverrideSelectionE2ETestCase, ) from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import ( prepare_inline_project, @@ -17,6 +18,61 @@ run_sqb, ) +_MIXED_CURSOR_PROJECT_FILES: dict[str, str] = { + "sqlbuild_project.toml": dedent( + """ + name = "mixed_cursor_project" + adapter = "duckdb" + + [connection] + database = "mixed_cursor.duckdb" + """ + ).strip() + + "\n", + "seeds/raw_orders.csv": "id,ordered_at\n1,2026-01-01 00:00:00\n2,2026-01-01 01:00:00\n", + "seeds/raw_orders.yml": dedent( + """ + seeds: + - name: raw_orders + description: Raw orders. + columns: + - name: id + type: INTEGER + - name: ordered_at + type: TIMESTAMP + """ + ).strip() + + "\n", + "models/orders.sql": dedent( + """ + MODEL (description "Orders by timestamp cursor.", + materialized incremental, + incremental_strategy append, + cursor ordered_at, + cursor_type timestamp, + cursor_grain second, + ); + + SELECT id, ordered_at + FROM __seed("raw_orders") + """ + ).strip() + + "\n", + "models/order_ids.sql": dedent( + """ + MODEL (description "Orders by integer cursor.", + materialized incremental, + incremental_strategy append, + cursor id, + cursor_type integer, + ); + + SELECT id FROM __seed("raw_orders") + """ + ).strip() + + "\n", +} + @pytest.mark.parametrize( "test_case", @@ -235,7 +291,82 @@ def test_given_append_cursor_project_when_rerunning_build_then_boundary_behavior ), ), ), - ) + ), + TimestampCursorBuildOverrideE2ETestCase( + description="build types untyped cursor flags from the selected timestamp model", + repo_files={ + "sqlbuild_project.toml": dedent( + """ + name = "timestamp_cursor_override_project" + adapter = "duckdb" + + [connection] + database = "timestamp_cursor_override.duckdb" + + [defaults] + materialized = "table" + """ + ).strip() + + "\n", + "sources/raw.yml": dedent( + """ + sources: + - name: raw_orders + description: Test source raw_orders. + schema: main + table: raw_orders + """ + ).strip() + + "\n", + "models/orders.sql": dedent( + """ + MODEL (description "Test model orders.", + materialized incremental, + incremental_strategy append, + cursor ordered_at, + cursor_type timestamp, + cursor_grain second, + ); + + SELECT id, ordered_at + FROM __source("raw_orders") + """ + ).strip() + + "\n", + }, + initial_seed_sql=dedent( + """ + CREATE TABLE main.raw_orders (id INTEGER, ordered_at TIMESTAMP); + + INSERT INTO main.raw_orders VALUES + (1, '2026-01-01 00:00:00'), + (2, '2026-01-01 01:00:00'), + (3, '2026-01-01 02:00:00'); + """ + ).strip(), + command=( + "--no-color", + "build", + "--start-cursor", + "2026-01-01T01:00:00", + "--end-cursor", + "2026-01-01T03:00:00", + ), + expected_exit_code=0, + expected_runtime_sql_fragment=( + "WHERE ordered_at >= TIMESTAMP '2026-01-01T01:00:00' " + "AND ordered_at < TIMESTAMP '2026-01-01T03:00:01'" + ), + expected_query_results=( + ( + ("SELECT id, CAST(ordered_at AS VARCHAR) FROM main.orders ORDER BY id"), + ( + (2, "2026-01-01 01:00:00"), + (3, "2026-01-01 02:00:00"), + ), + ), + ), + ), ], ids=lambda case: case.description, ) @@ -269,3 +400,43 @@ def test_given_timestamp_cursor_start_override_when_building_then_lower_bound_is for query, expected_rows in test_case.expected_query_results: actual_rows: list[tuple[object, ...]] = query_duckdb(db_path=db_path, sql=query) assert tuple(tuple(row) for row in actual_rows) == expected_rows + + +@pytest.mark.parametrize( + "test_case", + [ + UntypedCursorOverrideSelectionE2ETestCase( + description="selection mixing cursor types is rejected", + command=("--no-color", "plan", "--start-cursor", "1"), + expected_exit_code=1, + expected_output_fragments=( + "S303", + "--start-cursor and --end-cursor need one cursor type, but the selection mixes " + "integer cursor models (order_ids) and timestamp cursor models (orders)", + ), + ), + UntypedCursorOverrideSelectionE2ETestCase( + description="positional selection of one cursor type types the flags", + command=("--no-color", "plan", "+order_ids", "--start-cursor", "1"), + expected_exit_code=0, + expected_output_fragments=("order_ids",), + ), + ], + ids=lambda case: case.description, +) +def test_given_untyped_cursor_flags_when_planning_selection_then_types_must_agree( + test_case: UntypedCursorOverrideSelectionE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_inline_project( + tmp_path=tmp_path, + project_name="mixed_cursor_project", + repo_files=_MIXED_CURSOR_PROJECT_FILES, + ) + + result: object = run_sqb(command=test_case.command, project_dir=project_dir) + + assert result.returncode == test_case.expected_exit_code, result.stdout + result.stderr + fragment: str + for fragment in test_case.expected_output_fragments: + assert fragment in result.stdout + result.stderr diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/build/test_selector_surface.py b/tests/e2e/src/sqlbuild/cli/commands/main/build/test_selector_surface.py index da3d915554..44f4bc3ab1 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/build/test_selector_surface.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/build/test_selector_surface.py @@ -69,6 +69,19 @@ expected_stderr_fragments=(), pre_commands=(("--no-color", "build"),), ), + SelectorSurfaceBuildE2ETestCase( + description="positional selector and global flag after the subcommand", + command=("build", "stg_*", "--no-color"), + expected_exit_code=0, + expected_stdout_fragments=( + "Plan ready 3 selected", + "stg_customers", + "stg_orders", + "stg_payments", + ), + expected_stderr_fragments=(), + pre_commands=(("--no-color", "build"),), + ), SelectorSurfaceBuildE2ETestCase( description="malformed path selector with internal plus fails clearly", command=("--no-color", "plan", "--select", "+fact_orders~+daily_activity_rollup"), diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/lineage/test_lineage.py b/tests/e2e/src/sqlbuild/cli/commands/main/lineage/test_lineage.py index fc11cb389c..c783acc950 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/lineage/test_lineage.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/lineage/test_lineage.py @@ -321,7 +321,7 @@ def test_given_unchanged_project_when_lineage_runs_twice_then_reuses_structural_ ( LineageCacheCliTestCase( description="authored dependency change invalidates structural lineage cache", - command=("lineage", "fact_orders", "--format", "json"), + command=("lineage", "fact_orders", "--json"), expected_node_id="model:stg_customers", ), ), diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/scope/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/scope/_test_types.py index fb41210e00..fb4959db4a 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/scope/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/scope/_test_types.py @@ -35,3 +35,11 @@ class LargeScopePerformanceCase: expected_max_warm_seconds: float expected_max_cache_bytes: int command_timeout_seconds: float + + +@dataclass(frozen=True) +class ScopeBareTargetE2eCase: + description: str + target: str + expected_exit_code: int + expected_fragment: str diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/scope/test_scope.py b/tests/e2e/src/sqlbuild/cli/commands/main/scope/test_scope.py index b61c1428bb..e2e2c26196 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/scope/test_scope.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/scope/test_scope.py @@ -8,7 +8,10 @@ import pytest -from tests.e2e.src.sqlbuild.cli.commands.main.scope._test_types import ScopeE2eCase +from tests.e2e.src.sqlbuild.cli.commands.main.scope._test_types import ( + ScopeBareTargetE2eCase, + ScopeE2eCase, +) from tests.e2e.src.sqlbuild.cli.commands.main.scope.helpers import run_scope_alias @@ -59,5 +62,74 @@ def test_given_minimal_project_when_running_scope_aliases_then_outputs_are_offli assert json.loads(broken.stdout)["complete"] is False +@pytest.mark.parametrize( + "test_case", (ScopeE2eCase("bare model name", 0),), ids=lambda case: case.description +) +def test_given_bare_model_name_when_running_scope_then_matches_the_prefixed_report( + tmp_path: Path, test_case: ScopeE2eCase +) -> None: + project_dir: Path = tmp_path / "project" + (project_dir / "models").mkdir(parents=True) + (project_dir / "sqlbuild_project.toml").write_text( + 'name = "scope_e2e"\nadapter = "duckdb"\n', encoding="utf-8" + ) + (project_dir / "models" / "orders.sql").write_text( + "MODEL(description 'Test model orders.');\nSELECT 1 AS id\n", encoding="utf-8" + ) + + prefixed: subprocess.CompletedProcess[str] = run_scope_alias( + alias="sqb", project_dir=project_dir, args=("model:orders", "--json") + ) + bare: subprocess.CompletedProcess[str] = run_scope_alias( + alias="sqb", project_dir=project_dir, args=("orders", "--json") + ) + + assert prefixed.returncode == bare.returncode == test_case.expected_exit_code, bare.stderr + assert bare.stdout == prefixed.stdout + + +@pytest.mark.parametrize( + "test_case", + ( + ScopeBareTargetE2eCase( + description="bare macro name needs its prefix", + target="cents", + expected_exit_code=1, + expected_fragment="scope target 'cents' needs its prefix: macro:cents", + ), + ScopeBareTargetE2eCase( + description="unknown bare name", + target="missing", + expected_exit_code=1, + expected_fragment="unknown scope target 'missing'", + ), + ), + ids=lambda case: case.description, +) +def test_given_bare_name_that_is_not_a_resource_when_running_scope_then_fails_clearly( + tmp_path: Path, test_case: ScopeBareTargetE2eCase +) -> None: + project_dir: Path = tmp_path / "project" + (project_dir / "models").mkdir(parents=True) + (project_dir / "macros").mkdir() + (project_dir / "sqlbuild_project.toml").write_text( + 'name = "scope_e2e"\nadapter = "duckdb"\n', encoding="utf-8" + ) + (project_dir / "models" / "orders.sql").write_text( + "MODEL(description 'Test model orders.');\nSELECT 1 AS id\n", encoding="utf-8" + ) + (project_dir / "macros" / "cents.py").write_text( + "def cents(expression: str) -> str:\n return expression\n", encoding="utf-8" + ) + + result: subprocess.CompletedProcess[str] = run_scope_alias( + alias="sqb", project_dir=project_dir, args=(test_case.target, "--no-color") + ) + + assert result.returncode == test_case.expected_exit_code + assert "C960" in result.stderr + assert test_case.expected_fragment in result.stderr + + if __name__ == "__main__": pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/_test_types.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/_test_types.py index a3fc989973..51a4626013 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/_test_types.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/_test_types.py @@ -43,3 +43,63 @@ class EventExportWarningTestCase: exporter_counts: tuple[tuple[str, int, int, int, int], ...] command_succeeded: bool expected_message: str | None + + +@dataclass(frozen=True) +class GlobalFlagPlacementTestCase: + """One placement of the flags every command accepts.""" + + description: str + argv: tuple[str, ...] + expected_debug: bool + expected_no_color: bool + expected_project_dir: str | None + + +@dataclass(frozen=True) +class PositionalSelectTestCase: + """Selectors given as positional arguments, alone or with --select.""" + + description: str + argv: tuple[str, ...] + expected_select: tuple[str, ...] + + +@dataclass(frozen=True) +class OutputFormatAliasTestCase: + """A --json alias for a command whose output format is chosen with --format.""" + + description: str + argv: tuple[str, ...] + format_attribute: str + expected_format: str + + +@dataclass(frozen=True) +class RejectedArgumentsTestCase: + """Arguments the parser must reject with a usage error.""" + + description: str + argv: tuple[str, ...] + expected_message: str + + +@dataclass(frozen=True) +class DbtProjectDirParsingTestCase: + """A dbt command whose --project-dir belongs to dbt rather than SQLBuild.""" + + description: str + argv: tuple[str, ...] + expected_dbt_project_dir: str | None + expected_dbt_args: tuple[str, ...] + expected_no_color: bool + + +@dataclass(frozen=True) +class UntypedCursorParsingTestCase: + """Untyped --start-cursor/--end-cursor flags kept raw for the planner to type.""" + + description: str + argv: tuple[str, ...] + expected_start: str | None + expected_end: str | None diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/test_parsing.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/test_parsing.py index 1843eca6ed..d5b8cc8fe9 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/test_parsing.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/entry/test_parsing.py @@ -13,7 +13,13 @@ ) from tests.unit.src.sqlbuild.cli.commands._helpers.entry._test_types import ( AuditConcurrencyParsingTestCase, + DbtProjectDirParsingTestCase, + GlobalFlagPlacementTestCase, + OutputFormatAliasTestCase, + PositionalSelectTestCase, QueryDiffParsingTestCase, + RejectedArgumentsTestCase, + UntypedCursorParsingTestCase, VerboseCommandTestCase, VersionFlagTestCase, ) @@ -201,3 +207,275 @@ def test_given_version_flag_when_parsing_then_prints_version_and_exits_zero( assert invocation.args is None assert invocation.exit_code == test_case.expected_exit_code assert f"sqb {installed_version('sqlbuild')}" in capsys.readouterr().out + + +@pytest.mark.parametrize( + "test_case", + [ + GlobalFlagPlacementTestCase( + description="before the subcommand", + argv=("--debug", "--no-color", "--project-dir", "shop", "build"), + expected_debug=True, + expected_no_color=True, + expected_project_dir="shop", + ), + GlobalFlagPlacementTestCase( + description="after the subcommand", + argv=("build", "--debug", "--no-color", "--project-dir", "shop"), + expected_debug=True, + expected_no_color=True, + expected_project_dir="shop", + ), + GlobalFlagPlacementTestCase( + description="legacy project dir alias after the subcommand", + argv=("compile", "--sqb-project-dir", "shop"), + expected_debug=False, + expected_no_color=False, + expected_project_dir="shop", + ), + GlobalFlagPlacementTestCase( + description="root values survive an unflagged subcommand", + argv=("--debug", "--project-dir", "shop", "plan", "--no-color"), + expected_debug=True, + expected_no_color=True, + expected_project_dir="shop", + ), + GlobalFlagPlacementTestCase( + description="after a nested subcommand", + argv=("contract", "diff", "--from", "prod", "--debug", "--project-dir", "shop"), + expected_debug=True, + expected_no_color=False, + expected_project_dir="shop", + ), + GlobalFlagPlacementTestCase( + description="omitted everywhere", + argv=("test",), + expected_debug=False, + expected_no_color=False, + expected_project_dir=None, + ), + ], + ids=lambda case: case.description, +) +def test_given_global_flags_anywhere_when_parsing_then_they_apply_to_the_invocation( + test_case: GlobalFlagPlacementTestCase, +) -> None: + parsed: ParsedCliInvocation = parse_cli_invocation( + argv=test_case.argv, parser=build_cli_parser() + ) + + assert parsed.args is not None + assert parsed.args.debug is test_case.expected_debug + assert parsed.args.no_color is test_case.expected_no_color + assert parsed.args.project_dir == test_case.expected_project_dir + + +@pytest.mark.parametrize( + "test_case", + [ + DbtProjectDirParsingTestCase( + description="dbt init keeps its own project dir", + argv=("dbt", "init", "--project-dir", "dbt_shop"), + expected_dbt_project_dir="dbt_shop", + expected_dbt_args=(), + expected_no_color=False, + ), + DbtProjectDirParsingTestCase( + description="dbt passthrough still receives project dir", + argv=("dbt", "build", "--no-color", "--project-dir", "dbt_shop"), + expected_dbt_project_dir=None, + expected_dbt_args=("--project-dir", "dbt_shop"), + expected_no_color=True, + ), + ], + ids=lambda case: case.description, +) +def test_given_dbt_project_dir_when_parsing_then_it_is_not_the_sqlbuild_project_dir( + test_case: DbtProjectDirParsingTestCase, +) -> None: + parsed: ParsedCliInvocation = parse_cli_invocation( + argv=test_case.argv, parser=build_cli_parser() + ) + + assert parsed.args is not None + assert parsed.args.project_dir is None + assert parsed.args.dbt_project_dir == test_case.expected_dbt_project_dir + assert tuple(parsed.args.dbt_args) == test_case.expected_dbt_args + assert parsed.args.no_color is test_case.expected_no_color + + +@pytest.mark.parametrize( + "test_case", + [ + PositionalSelectTestCase( + description="build graph selector", + argv=("build", "stg_orders+"), + expected_select=("stg_orders+",), + ), + PositionalSelectTestCase( + description="positional and select are combined", + argv=("plan", "stg_orders", "--select", "dim_customers"), + expected_select=("stg_orders", "dim_customers"), + ), + PositionalSelectTestCase( + description="selectors on both sides of an option", + argv=("build", "stg_orders", "--full-refresh", "tag:daily"), + expected_select=("stg_orders", "tag:daily"), + ), + PositionalSelectTestCase( + description="compile", + argv=("compile", "+fact_orders"), + expected_select=("+fact_orders",), + ), + PositionalSelectTestCase( + description="test", argv=("test", "stg_orders"), expected_select=("stg_orders",) + ), + PositionalSelectTestCase( + description="seed", argv=("seed", "waffle_types"), expected_select=("waffle_types",) + ), + PositionalSelectTestCase( + description="freshness", + argv=("freshness", "source:raw__orders"), + expected_select=("source:raw__orders",), + ), + PositionalSelectTestCase( + description="audit", argv=("audit", "fact_orders"), expected_select=("fact_orders",) + ), + PositionalSelectTestCase( + description="check", argv=("check", "fact_orders"), expected_select=("fact_orders",) + ), + PositionalSelectTestCase( + description="select only", argv=("build", "-s", "a", "b"), expected_select=("a", "b") + ), + ], + ids=lambda case: case.description, +) +def test_given_positional_selectors_when_parsing_then_they_join_the_selection( + test_case: PositionalSelectTestCase, +) -> None: + parsed: ParsedCliInvocation = parse_cli_invocation( + argv=test_case.argv, parser=build_cli_parser() + ) + + assert parsed.args is not None + assert tuple(parsed.args.select) == test_case.expected_select + + +@pytest.mark.parametrize( + "test_case", + [ + OutputFormatAliasTestCase( + description="lineage json", + argv=("lineage", "stg_orders", "--json"), + format_attribute="lineage_format", + expected_format="json", + ), + OutputFormatAliasTestCase( + description="lineage default", + argv=("lineage", "stg_orders"), + format_attribute="lineage_format", + expected_format="tree", + ), + OutputFormatAliasTestCase( + description="query json", + argv=("query", "SELECT 1", "--json"), + format_attribute="query_format", + expected_format="json", + ), + OutputFormatAliasTestCase( + description="query explicit format", + argv=("query", "SELECT 1", "--format", "csv"), + format_attribute="query_format", + expected_format="csv", + ), + ], + ids=lambda case: case.description, +) +def test_given_json_alias_when_parsing_then_selects_json_format( + test_case: OutputFormatAliasTestCase, +) -> None: + parsed: ParsedCliInvocation = parse_cli_invocation( + argv=test_case.argv, parser=build_cli_parser() + ) + + assert parsed.args is not None + assert getattr(parsed.args, test_case.format_attribute) == test_case.expected_format + assert parsed.args.json is False + + +@pytest.mark.parametrize( + "test_case", + [ + UntypedCursorParsingTestCase( + description="load keeps untyped values raw", + argv=("load", "--start-cursor", "2026-05-01", "--end-cursor", "2026-05-02"), + expected_start="2026-05-01", + expected_end="2026-05-02", + ), + UntypedCursorParsingTestCase( + description="build accepts start only", + argv=("build", "--start-cursor", "100"), + expected_start="100", + expected_end=None, + ), + ], + ids=lambda case: case.description, +) +def test_given_untyped_cursor_flags_when_parsing_then_values_are_kept_raw( + test_case: UntypedCursorParsingTestCase, +) -> None: + parsed: ParsedCliInvocation = parse_cli_invocation( + argv=test_case.argv, parser=build_cli_parser() + ) + + assert parsed.args is not None + assert (parsed.args.start_cursor, parsed.args.end_cursor) == ( + test_case.expected_start, + test_case.expected_end, + ) + assert parsed.args.start_cursor_ts is None + + +@pytest.mark.parametrize( + "test_case", + [ + RejectedArgumentsTestCase( + description="unknown option among selectors", + argv=("build", "stg_orders", "--no-such-flag", "fact_orders"), + expected_message="unrecognized arguments: --no-such-flag fact_orders", + ), + RejectedArgumentsTestCase( + description="positional on a command without positional selection", + argv=("load", "raw__orders"), + expected_message="unrecognized arguments: raw__orders", + ), + RejectedArgumentsTestCase( + description="json alias with an explicit format", + argv=("lineage", "stg_orders", "--json", "--format", "tree"), + expected_message="argument --format: not allowed with argument --json", + ), + RejectedArgumentsTestCase( + description="untyped and typed cursor flags together", + argv=("build", "--start-cursor", "5", "--end-cursor-int", "9"), + expected_message=( + "--start-cursor/--end-cursor cannot be combined with the -ts or -int cursor flags" + ), + ), + ], + ids=lambda case: case.description, +) +def test_given_conflicting_or_unknown_arguments_when_parsing_then_exits_with_usage_error( + test_case: RejectedArgumentsTestCase, + capsys: pytest.CaptureFixture[str], +) -> None: + parsed: ParsedCliInvocation = parse_cli_invocation( + argv=test_case.argv, parser=build_cli_parser() + ) + + assert parsed.args is None + assert parsed.exit_code == 2 + assert test_case.expected_message in capsys.readouterr().err + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py index 5e45a8382d..8b717e747f 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/_test_types.py @@ -23,3 +23,17 @@ class ScopeColourCase: use_color: bool expected_fragments: tuple[str, ...] unexpected_fragments: tuple[str, ...] + + +@dataclass(frozen=True) +class ScopeTargetCase: + description: str + target: str | None + expected_target: str | None + + +@dataclass(frozen=True) +class ScopeTargetErrorCase: + description: str + target: str + expected_message: str diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py index 68d465ef22..68bff72036 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/helpers.py @@ -4,9 +4,19 @@ from dataclasses import replace +from sqlbuild.compiler.scopes.main.build_scope_lookup import build_scope_lookup from sqlbuild.compiler.scopes.main.preview_scope_move import preview_scope_move from sqlbuild.compiler.scopes.main.query_scope_report import query_scope_report -from sqlbuild.compiler.scopes.models import ScopeLookup, ScopeReport, ScopeReportFilters +from sqlbuild.compiler.scopes.models import ( + OwnershipRoot, + ResourceIdentity, + ResourceRecord, + ScopeIndex, + ScopeLookup, + ScopeReport, + ScopeReportFilters, +) +from sqlbuild.compiler.scopes.types import ResourceKind from tests.unit.src.sqlbuild.compiler.scopes.helpers import report_scope_lookup @@ -21,3 +31,37 @@ def orders_move_report() -> ScopeReport: lookup=lookup, resource="model:orders", destination="models/marts/orders.sql" ) return replace(report, move_preview=move, diagnostics=(*report.diagnostics, *diagnostics)) + + +def bare_target_scope_lookup() -> ScopeLookup: + """Return the report lookup plus a source, seed, function, test and macro-named model.""" + + index: ScopeIndex = report_scope_lookup().index + extra: tuple[ResourceRecord, ...] = ( + ResourceRecord( + ResourceIdentity(ResourceKind.SOURCE, "raw__orders"), + "sources/raw.yml", + OwnershipRoot("sources", resource_kind=ResourceKind.SOURCE), + ), + ResourceRecord( + ResourceIdentity(ResourceKind.SEED, "waffle_types"), + "seeds/waffle_types.csv", + OwnershipRoot("seeds", resource_kind=ResourceKind.SEED), + ), + ResourceRecord( + ResourceIdentity(ResourceKind.FUNCTION, "udf__is_completed_order"), + "functions/sql/udf__is_completed_order.sql", + OwnershipRoot("functions/sql", resource_kind=ResourceKind.FUNCTION), + ), + ResourceRecord( + ResourceIdentity(ResourceKind.TEST, "orders_are_unique"), + "tests/unit/orders_are_unique.sql", + OwnershipRoot("tests/unit", resource_kind=ResourceKind.TEST), + ), + ResourceRecord( + ResourceIdentity(ResourceKind.MODEL, "normalize"), + "models/staging/normalize.sql", + OwnershipRoot("models", resource_kind=ResourceKind.MODEL), + ), + ) + return build_scope_lookup(index=replace(index, resources=(*index.resources, *extra))) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_target.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_target.py new file mode 100644 index 0000000000..adf0b3cc22 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/scope/test_target.py @@ -0,0 +1,83 @@ +"""Bare scope target qualification tests.""" + +from __future__ import annotations + +import pytest + +from sqlbuild.cli.commands._helpers.scope.target import qualify_scope_target +from sqlbuild.cli.commands.exceptions import CliUserError +from tests.unit.src.sqlbuild.cli.commands._helpers.scope._test_types import ( + ScopeTargetCase, + ScopeTargetErrorCase, +) +from tests.unit.src.sqlbuild.cli.commands._helpers.scope.helpers import bare_target_scope_lookup + + +@pytest.mark.parametrize( + "test_case", + ( + ScopeTargetCase("bare model", "orders", "model:orders"), + ScopeTargetCase("bare source", "raw__orders", "source:raw__orders"), + ScopeTargetCase("bare seed", "waffle_types", "seed:waffle_types"), + ScopeTargetCase( + "bare function", "udf__is_completed_order", "function:udf__is_completed_order" + ), + ScopeTargetCase("prefixed model unchanged", "model:orders", "model:orders"), + ScopeTargetCase("prefixed enum unchanged", "enum:order_status", "enum:order_status"), + ScopeTargetCase( + "resource path unchanged", "models/staging/orders.sql", "models/staging/orders.sql" + ), + ScopeTargetCase("prospective scope has no target", None, None), + ), + ids=lambda case: case.description, +) +def test_given_scope_target_when_qualifying_then_bare_resources_gain_their_kind( + test_case: ScopeTargetCase, +) -> None: + assert ( + qualify_scope_target(lookup=bare_target_scope_lookup(), target=test_case.target) + == test_case.expected_target + ) + + +@pytest.mark.parametrize( + "test_case", + ( + ScopeTargetErrorCase( + "bare enum needs prefix", + "order_status", + "scope target 'order_status' needs its prefix: enum:order_status", + ), + ScopeTargetErrorCase( + "bare constant needs prefix", + "warehouse_password", + "scope target 'warehouse_password' needs its prefix: constant:warehouse_password", + ), + ScopeTargetErrorCase( + "bare test needs prefix", + "orders_are_unique", + "scope target 'orders_are_unique' needs its prefix: test:orders_are_unique", + ), + ScopeTargetErrorCase( + "bare model colliding with a macro", + "normalize", + "scope target 'normalize' matches more than one resource or declaration; " + "write model:normalize or macro:normalize", + ), + ScopeTargetErrorCase("unknown bare name", "missing", "unknown scope target 'missing'"), + ), + ids=lambda case: case.description, +) +def test_given_ambiguous_or_prefix_only_bare_name_when_qualifying_then_raises_clear_error( + test_case: ScopeTargetErrorCase, +) -> None: + with pytest.raises(CliUserError) as error: + qualify_scope_target(lookup=bare_target_scope_lookup(), target=test_case.target) + + assert error.value.code == "C960" + assert error.value.message == test_case.expected_message + assert "macro:" in (error.value.help or "") + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/cli/commands/main/entry/test_main.py b/tests/unit/src/sqlbuild/cli/commands/main/entry/test_main.py index 9a41148b7b..d754ab40e9 100644 --- a/tests/unit/src/sqlbuild/cli/commands/main/entry/test_main.py +++ b/tests/unit/src/sqlbuild/cli/commands/main/entry/test_main.py @@ -2137,13 +2137,8 @@ def run_scenario_capture(request: ScenarioCaptureCommandRequest) -> int: "test_case", [ MainTestCase( - description="returns parser error for command local debug", - argv=["build", "--debug"], - expected_exit_code=2, - ), - MainTestCase( - description="returns parser error for command local no color", - argv=["plan", "--no-color"], + description="returns parser error for an unknown command flag", + argv=["build", "--no-such-flag"], expected_exit_code=2, ), MainTestCase( @@ -2154,7 +2149,7 @@ def run_scenario_capture(request: ScenarioCaptureCommandRequest) -> int: ], ids=lambda case: case.description, ) -def test_given_command_local_global_flags_when_running_main_then_it_returns_parser_error( +def test_given_unknown_flags_or_commands_when_running_main_then_it_returns_parser_error( test_case: MainTestCase, capsys: pytest.CaptureFixture[str], ) -> None: @@ -2171,7 +2166,7 @@ def test_given_command_local_global_flags_when_running_main_then_it_returns_pars [ MainTestCase( description="colorizes parser error prefix when color is supported", - argv=["build", "--debug"], + argv=["build", "--no-such-flag"], expected_exit_code=2, ) ], @@ -2197,7 +2192,7 @@ def test_given_parser_error_and_color_support_when_running_main_then_it_colorize [ MainTestCase( description="leaves parser error plain when no color is requested", - argv=["--no-color", "build", "--debug"], + argv=["build", "--no-such-flag", "--no-color"], expected_exit_code=2, ) ], diff --git a/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/_test_types.py b/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/_test_types.py index f40a347736..e0cd13ede3 100644 --- a/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/_test_types.py +++ b/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/_test_types.py @@ -6,7 +6,7 @@ from datetime import date, datetime from sqlbuild.adapter.contract.models import ColumnInfo -from sqlbuild.compiler.planner.models import CursorBounds, ModelCursorSnapshot +from sqlbuild.compiler.planner.models import CursorBounds, CursorOverrides, ModelCursorSnapshot from sqlbuild.spec.contracts.models import SourceEntry @@ -134,3 +134,24 @@ class HookSqlResolutionTestCase: post_hooks: object expected_pre_hooks: object expected_post_hooks: object + + +@dataclass(frozen=True) +class UntypedCursorOverrideTestCase: + """Untyped --start-cursor/--end-cursor values and the selected models' cursor types.""" + + description: str + model_cursor_types: tuple[tuple[str, str | None], ...] + overrides: CursorOverrides + expected_overrides: CursorOverrides + + +@dataclass(frozen=True) +class UntypedCursorOverrideErrorTestCase: + """Untyped cursor values that cannot be typed from the selection.""" + + description: str + model_cursor_types: tuple[tuple[str, str | None], ...] + overrides: CursorOverrides + expected_message_prefix: str + expected_code: str diff --git a/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/helpers.py b/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/helpers.py index a8f69470ce..96ae0645d0 100644 --- a/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/helpers.py +++ b/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/helpers.py @@ -65,3 +65,34 @@ class BracketTableFunctionCallAdapter(DuckDbAdapter): def render_table_function_call(self, *, target: str, call_suffix_sql: str) -> str: arguments_sql: str = call_suffix_sql.removeprefix("(").removesuffix(")") return f"TABLE({target}[{arguments_sql}])" + + +def build_cursor_models( + *, model_cursor_types: tuple[tuple[str, str | None], ...] +) -> tuple[CompiledModel, ...]: + """Build models named by `model_cursor_types`; a None type builds a non-cursor table.""" + + return tuple( + CompiledModel( + key=CompiledObjectKey(resource_type=CompiledResourceType.MODEL, name=name), + deps=(), + name=name, + relative_path=Path(f"models/{name}.sql"), + query_sql="SELECT 1", + config=CompileModelConfig(values=_cursor_model_config_values(cursor_type=cursor_type)), + destination=CompiledRelationLocation( + database=None, schema="staging", name=name, qualified_name=f"staging.{name}" + ), + ) + for name, cursor_type in model_cursor_types + ) + + +def _cursor_model_config_values(*, cursor_type: str | None) -> dict[str, object]: + table: dict[str, object] = {"materialized": "table"} + incremental: dict[str, object] = { + "materialized": "incremental", + "cursor": "updated_at", + "cursor_type": cursor_type, + } + return (table, incremental)[cursor_type is not None] diff --git a/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/test_cursor_override_types.py b/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/test_cursor_override_types.py new file mode 100644 index 0000000000..3bef326593 --- /dev/null +++ b/tests/unit/src/sqlbuild/compiler/planner/_helpers/resolve/test_cursor_override_types.py @@ -0,0 +1,112 @@ +"""Typing of untyped --start-cursor/--end-cursor values from the selected models.""" + +from __future__ import annotations + +import pytest + +from sqlbuild.compiler.planner._helpers.resolve.cursor import type_cursor_overrides +from sqlbuild.compiler.planner.exceptions import PlannerInputError +from sqlbuild.compiler.planner.models import CursorOverrides +from tests.unit.src.sqlbuild.compiler.planner._helpers.resolve._test_types import ( + UntypedCursorOverrideErrorTestCase, + UntypedCursorOverrideTestCase, +) +from tests.unit.src.sqlbuild.compiler.planner._helpers.resolve.helpers import build_cursor_models + + +@pytest.mark.parametrize( + "test_case", + ( + UntypedCursorOverrideTestCase( + description="timestamp selection", + model_cursor_types=(("orders", "timestamp"), ("customers", None)), + overrides=CursorOverrides(start="2026-05-01", end="2026-05-02T12:00:00"), + expected_overrides=CursorOverrides(start_ts="2026-05-01", end_ts="2026-05-02T12:00:00"), + ), + UntypedCursorOverrideTestCase( + description="integer selection", + model_cursor_types=(("events", "integer"), ("event_rollup", "integer")), + overrides=CursorOverrides(start="100"), + expected_overrides=CursorOverrides(start_int="100"), + ), + UntypedCursorOverrideTestCase( + description="no cursor model reads an integer literal as an integer", + model_cursor_types=(), + overrides=CursorOverrides(start="100", end="250"), + expected_overrides=CursorOverrides(start_int="100", end_int="250"), + ), + UntypedCursorOverrideTestCase( + description="no cursor model reads a date as a timestamp", + model_cursor_types=(("customers", None),), + overrides=CursorOverrides(end="2026-05-02"), + expected_overrides=CursorOverrides(end_ts="2026-05-02"), + ), + UntypedCursorOverrideTestCase( + description="typed overrides pass through a mixed selection", + model_cursor_types=(("orders", "timestamp"), ("events", "integer")), + overrides=CursorOverrides(start_ts="2026-05-01", end_int="9"), + expected_overrides=CursorOverrides(start_ts="2026-05-01", end_int="9"), + ), + ), + ids=lambda case: case.description, +) +def test_given_untyped_cursor_values_when_typing_then_uses_the_selection_cursor_type( + test_case: UntypedCursorOverrideTestCase, +) -> None: + typed: CursorOverrides | None = type_cursor_overrides( + cursor_overrides=test_case.overrides, + selected_models=build_cursor_models(model_cursor_types=test_case.model_cursor_types), + ) + + assert typed == test_case.expected_overrides + + +@pytest.mark.parametrize( + "test_case", + ( + UntypedCursorOverrideErrorTestCase( + description="mixed cursor types", + model_cursor_types=( + ("orders", "timestamp"), + ("events", "integer"), + ("payments", "timestamp"), + ), + overrides=CursorOverrides(start="2026-05-01"), + expected_message_prefix=( + "--start-cursor and --end-cursor need one cursor type, but the selection mixes " + "integer cursor models (events) and timestamp cursor models (orders, payments)" + ), + expected_code="S303", + ), + UntypedCursorOverrideErrorTestCase( + description="integer selection with a timestamp value", + model_cursor_types=(("events", "integer"),), + overrides=CursorOverrides(start="2026-05-01"), + expected_message_prefix="--start-cursor value '2026-05-01' is not a valid integer", + expected_code="S000", + ), + UntypedCursorOverrideErrorTestCase( + description="timestamp selection with an invalid value", + model_cursor_types=(("orders", "timestamp"),), + overrides=CursorOverrides(end="yesterday"), + expected_message_prefix="--end-cursor value 'yesterday' is not a valid ISO timestamp", + expected_code="S000", + ), + ), + ids=lambda case: case.description, +) +def test_given_untypable_cursor_values_when_typing_then_raises_planner_input_error( + test_case: UntypedCursorOverrideErrorTestCase, +) -> None: + with pytest.raises(PlannerInputError) as error: + type_cursor_overrides( + cursor_overrides=test_case.overrides, + selected_models=build_cursor_models(model_cursor_types=test_case.model_cursor_types), + ) + + assert error.value.message.startswith(test_case.expected_message_prefix) + assert error.value.code == test_case.expected_code + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/website/examples/waffle-shop/README.md b/website/examples/waffle-shop/README.md index 9c679a8a59..2abd7abd43 100644 --- a/website/examples/waffle-shop/README.md +++ b/website/examples/waffle-shop/README.md @@ -16,6 +16,12 @@ Build the project locally with DuckDB: sqb build ``` +Build one model and everything downstream of it: + +```bash +sqb build stg_orders+ +``` + After the build creates the local DuckDB objects, run SQL unit tests: ```bash diff --git a/website/src/content/docs/docs/cli/audit.mdx b/website/src/content/docs/docs/cli/audit.mdx index 02b3102f8a..b5ca48c715 100644 --- a/website/src/content/docs/docs/cli/audit.mdx +++ b/website/src/content/docs/docs/cli/audit.mdx @@ -10,15 +10,18 @@ Runs all attached audits without rebuilding models. Useful for verifying data qu ## Usage ```bash -sqb --project-dir audit [flags] +sqb audit [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | | `--defer-to` | Resolve model references against another target | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select audits attached to specific models | | `--select-file` | Read selectors from a file | | `--exclude` | Exclude audits attached to specific models | @@ -61,7 +64,7 @@ guarantee immediate cancellation of a query already running in the warehouse. sqb audit # Run audits for marts only -sqb audit --select path:models/marts +sqb audit path:models/marts # Run up to 8 audit queries concurrently sqb audit --concurrency 8 diff --git a/website/src/content/docs/docs/cli/build.mdx b/website/src/content/docs/docs/cli/build.mdx index feb7f7190f..d8047df8a4 100644 --- a/website/src/content/docs/docs/cli/build.mdx +++ b/website/src/content/docs/docs/cli/build.mdx @@ -11,13 +11,17 @@ Compiles, plans, and executes the selected build lifecycle, running the full sel ## Usage ```bash -sqb --project-dir build [flags] +sqb build [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. + ## Global options -These options apply to every `sqb` command and go before the command name, as in -`sqb --debug build`: +These options apply to every `sqb` command. `--project-dir`, `--no-color`, and `--debug` go +before or after the command name, as in `sqb --debug build` or `sqb build --debug`. Under +`sqb dbt`, put `--project-dir` before `dbt`: after it, `--project-dir` is passed to dbt as dbt's +own option. `--version` works only directly after `sqb`, as in `sqb --version`: | Option | Description | |--------|-------------| @@ -42,6 +46,8 @@ These options apply to every `sqb` command and go before the command name, as in | `--fail-fast` | Stop on first failure and skip remaining nodes | | `--concurrency` | Number of worker connections (default: 1) | | `--verbose`, `-v` | Show lifecycle SQL inline after each model | +| `--start-cursor` | Override the start cursor, typed from the selected incremental models' cursor type; fails with `S303` if the selection mixes timestamp and integer cursors. With no cursor model selected, an integer value is read as an integer cursor and anything else as an ISO timestamp | +| `--end-cursor` | Override the end cursor, typed the same way as `--start-cursor` | | `--start-cursor-ts` | Override start cursor for timestamp incremental models (ISO format) | | `--end-cursor-ts` | Override end cursor for timestamp incremental models (ISO format) | | `--start-cursor-int` | Override start cursor for integer incremental models | @@ -51,6 +57,7 @@ These options apply to every `sqb` command and go before the command name, as in | `--reload` | Reload managed sources (passes `is_reload=True` to loaders) | | `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](/docs/concepts/planning/selection-and-staleness) | | `--manifest` | Generate `target/manifest.json` with plan-aware project metadata | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific models | | `--exclude` | Exclude specific models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | @@ -118,7 +125,7 @@ statement or step still prints its `FAIL` row. Use `--defer-to` to resolve unselected model references against another target. This lets you build a subset of models in dev while referencing production tables for everything else: ```bash -sqb build --select fact_orders --defer-to prod +sqb build fact_orders --defer-to prod ``` No `manifest.json` is required. `--defer-to` selects only the namespace used for unselected diff --git a/website/src/content/docs/docs/cli/check.mdx b/website/src/content/docs/docs/cli/check.mdx index 9ff49c8cae..431f4e9fc8 100644 --- a/website/src/content/docs/docs/cli/check.mdx +++ b/website/src/content/docs/docs/cli/check.mdx @@ -10,13 +10,16 @@ Runs Python [checks](/docs/concepts/python-nodes/checks) in isolation. Checks va ## Usage ```bash -sqb --project-dir check [flags] +sqb check [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select checks to run (by name, `check:`, `tag:`, or graph expansion) | | `--exclude` | Exclude checks from the selection | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | @@ -34,10 +37,10 @@ Selecting a non-check node is rejected; use [`sqb build`](/docs/cli/build) to ru sqb check # Run one check and its required dependencies -sqb check --select +check_orders_exported +sqb check +check_orders_exported # Run checks by tag -sqb check --select tag:exports +sqb check tag:exports # Print results as JSON sqb check --json diff --git a/website/src/content/docs/docs/cli/compile.mdx b/website/src/content/docs/docs/cli/compile.mdx index 6522084693..fd4852890a 100644 --- a/website/src/content/docs/docs/cli/compile.mdx +++ b/website/src/content/docs/docs/cli/compile.mdx @@ -12,9 +12,11 @@ Discovers the complete project, resolves references, expands macros, validates S ## Usage ```bash -sqb --project-dir compile [flags] +sqb compile [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | @@ -25,6 +27,7 @@ sqb --project-dir compile [flags] | `--json` | Output the full compile report as JSON | | `--manifest` | Generate `target/manifest.json` with project metadata | | `--lineage-mode` | Column lineage mode: `fast` (default), `rich` (slower, more detail), or `none` | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Analyze and report selected resources | | `--select-file` | Read selectors from a file, one selector per line | | `--exclude` | Remove resources from the selected scope | @@ -47,7 +50,7 @@ Use normal selectors to limit expensive analysis and reporting while retaining c discovery and reference integrity: ```bash -sqb compile --select fact_orders daily_revenue +sqb compile fact_orders daily_revenue sqb compile --select-file selected-models.txt sqb compile --exclude deprecated_model ``` @@ -279,7 +282,7 @@ sqb compile --lineage-mode none sqb compile --manifest # Compile only selected models -sqb compile --select fact_orders daily_revenue +sqb compile fact_orders daily_revenue # Disable SQL analysis sqb compile --no-sql-analysis diff --git a/website/src/content/docs/docs/cli/freshness.mdx b/website/src/content/docs/docs/cli/freshness.mdx index b23550c4c0..674f3f7232 100644 --- a/website/src/content/docs/docs/cli/freshness.mdx +++ b/website/src/content/docs/docs/cli/freshness.mdx @@ -10,9 +10,11 @@ Observes the current data version of each source and reports whether the data ha ## Usage ```bash -sqb --project-dir freshness [flags] +sqb freshness [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | @@ -23,6 +25,7 @@ sqb --project-dir freshness [flags] | `--json` | Output as JSON instead of human-readable text | | `--json-output` | Write JSON output to a file path (also prints text to stdout unless `--json` is set) | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific sources or models (sources upstream of selected models are included) | | `--exclude` | Exclude specific sources or models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | @@ -36,10 +39,10 @@ Without `--select`, all sources in the project are observed. When `--select` is sqb freshness # Observe a specific source -sqb freshness --select raw_orders +sqb freshness raw_orders # Observe sources upstream of a model -sqb freshness --select fact_orders +sqb freshness fact_orders ``` ## Observation without state diff --git a/website/src/content/docs/docs/cli/lineage.mdx b/website/src/content/docs/docs/cli/lineage.mdx index e8ffc17e76..742caeccc2 100644 --- a/website/src/content/docs/docs/cli/lineage.mdx +++ b/website/src/content/docs/docs/cli/lineage.mdx @@ -29,6 +29,7 @@ Exactly one of a positional target or `--select` is required. | `--direction` | `upstream` (default), `downstream`, or `both`. `both` is only available for model lineage. | | `--depth` | How many hops to traverse. An integer or `all` (default: `all`). | | `--format` | Output format: `tree` (default), `list`, or `json`. | +| `--json` | Shorthand for `--format json`. | | `--mode` | Column lineage analysis mode: `rich` (default) or `fast`. | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias). | | `--select`, `-s` | Select resources using the same selector syntax as `sqb compile`, `sqb plan` and `sqb build`, for example `path:models/marts`, `tag:finance` or `stg_*`. | @@ -89,7 +90,7 @@ source:raw__payments -> model:stg_payments Structured output with nodes, edges, and metadata: ```bash -sqb lineage fact_orders --format json +sqb lineage fact_orders --json ``` ```json @@ -161,7 +162,7 @@ raw__payments.amount_cents -> stg_payments.amount_cents direct ### JSON ```bash -sqb lineage fact_orders.total_cents --format json +sqb lineage fact_orders.total_cents --json ``` ```json @@ -228,7 +229,7 @@ sqb lineage --select "stg_orders~daily_activity_rollup" sqb lineage --select "+fact_orders" # JSON output for programmatic consumption -sqb lineage fact_orders --format json --direction upstream +sqb lineage fact_orders --json --direction upstream ``` ## Depth limiting diff --git a/website/src/content/docs/docs/cli/load.mdx b/website/src/content/docs/docs/cli/load.mdx index 94745f706e..c04bde9282 100644 --- a/website/src/content/docs/docs/cli/load.mdx +++ b/website/src/content/docs/docs/cli/load.mdx @@ -21,6 +21,8 @@ sqb --project-dir load [flags] | `--exclude` | Exclude specific sources or loaders | | `--reload` | Pass `is_reload=True` to loader functions | | `--concurrency` | Number of worker connections (default: from settings or 1) | +| `--start-cursor` | Override the start cursor; an integer value sets `start_cursor_int`, anything else is read as an ISO timestamp and sets `start_cursor_ts` | +| `--end-cursor` | Override the end cursor, typed the same way as `--start-cursor` | | `--start-cursor-ts` | Override start cursor for timestamp-based loaders (ISO format) | | `--end-cursor-ts` | Override end cursor for timestamp-based loaders (ISO format) | | `--start-cursor-int` | Override start cursor for integer-based loaders | @@ -46,7 +48,7 @@ sqb load --concurrency 4 sqb load --reload # Override cursor bounds -sqb load --start-cursor-ts "2026-05-01T00:00:00" +sqb load --start-cursor "2026-05-01T00:00:00" ``` ## Execution order diff --git a/website/src/content/docs/docs/cli/plan.mdx b/website/src/content/docs/docs/cli/plan.mdx index aa66b93b67..4cb337be29 100644 --- a/website/src/content/docs/docs/cli/plan.mdx +++ b/website/src/content/docs/docs/cli/plan.mdx @@ -10,9 +10,11 @@ Shows the execution plan without making any changes. Useful for inspecting chang ## Usage ```bash -sqb --project-dir plan [flags] +sqb plan [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | @@ -25,10 +27,13 @@ sqb --project-dir plan [flags] | `--json` | Output the plan as JSON | | `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](/docs/concepts/planning/selection-and-staleness) | | `--full-refresh` | Plan a full rebuild for selected models unless a model sets `full_refresh false`; `full_refresh true` forces a model without this flag | +| `--start-cursor` | Override the start cursor, typed from the selected incremental models' cursor type; fails with `S303` if the selection mixes timestamp and integer cursors. With no cursor model selected, an integer value is read as an integer cursor and anything else as an ISO timestamp | +| `--end-cursor` | Override the end cursor, typed the same way as `--start-cursor` | | `--start-cursor-ts` | Override start cursor for timestamp incremental models (ISO format) | | `--end-cursor-ts` | Override end cursor for timestamp incremental models (ISO format) | | `--start-cursor-int` | Override start cursor for integer incremental models | | `--end-cursor-int` | Override end cursor for integer incremental models | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific models | | `--exclude` | Exclude specific models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | diff --git a/website/src/content/docs/docs/cli/query.mdx b/website/src/content/docs/docs/cli/query.mdx index 5a4a68f7d9..388a675e77 100644 --- a/website/src/content/docs/docs/cli/query.mdx +++ b/website/src/content/docs/docs/cli/query.mdx @@ -28,6 +28,7 @@ File paths are resolved from the current working directory. | `sql` | SQL to execute (positional argument) | | `--file` | Read SQL from a file instead of the command line | | `--format` | Output format: `long` (default), `table`, `json`, or `csv` | +| `--json` | Shorthand for `--format json` | | `--limit` | Maximum rows to return (default: 20) | | `--no-limit` | Disable the row limit | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | @@ -100,7 +101,7 @@ sqb query "SELECT * FROM dev.fact_orders" sqb query "SELECT * FROM dev.dim_customers" --format table --limit 50 # Export to JSON -sqb query "SELECT * FROM dev.daily_revenue" --format json --no-limit +sqb query "SELECT * FROM dev.daily_revenue" --json --no-limit # Run SQL from a file sqb query --file debug_query.sql diff --git a/website/src/content/docs/docs/cli/scope.mdx b/website/src/content/docs/docs/cli/scope.mdx index 95f626b6e4..8dbb6e6943 100644 --- a/website/src/content/docs/docs/cli/scope.mdx +++ b/website/src/content/docs/docs/cli/scope.mdx @@ -24,9 +24,10 @@ sqb scope --at PATH [flags] sqb scope TARGET --as-path PATH [flags] ``` -`TARGET` is either a kind-qualified identity or an exact project-relative resource path: +`TARGET` is a model, source, seed, or function name, a kind-qualified identity, or an exact project-relative resource path: ```bash +sqb scope stg_orders sqb scope model:stg_orders sqb scope test:stg_orders__excludes_cancelled sqb scope scenario:daily_revenue @@ -42,7 +43,7 @@ sqb scope constant:minimum_order_value sqb scope models/staging/orders/stg_orders.sql ``` -Bare names such as `stg_orders` and `order_status` are rejected even if they are currently unique. Qualification keeps commands stable when names collide across resource or declaration kinds. Public declaration identities are `macro:`, `enum:`, and `constant:`. +Models, sources, seeds, and functions share one project-wide namespace, so their bare names, such as `stg_orders`, are accepted and resolve to the qualified identity. Macros, enums, constants, tests, and audits can share names with each other and with resources, so they always need their prefix: a bare name such as `order_status` that names one of them fails with `C960` and shows the qualified form to use, and a bare name that matches both a resource and one of these declarations fails and lists every match. Public declaration identities are `macro:`, `enum:`, and `constant:`. Model-private constants and enums include their owner in the introspection identity: diff --git a/website/src/content/docs/docs/cli/seed.mdx b/website/src/content/docs/docs/cli/seed.mdx index 9094c1258a..ec3a253b85 100644 --- a/website/src/content/docs/docs/cli/seed.mdx +++ b/website/src/content/docs/docs/cli/seed.mdx @@ -10,13 +10,16 @@ Loads seed CSV files into the warehouse as tables. Seeds are fully replaced on e ## Usage ```bash -sqb --project-dir seed [flags] +sqb seed [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select specific seeds by name | | `--exclude` | Exclude specific seeds | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `build` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | @@ -28,5 +31,5 @@ sqb --project-dir seed [flags] sqb seed # Load a specific seed -sqb seed --select seed:waffle_types +sqb seed seed:waffle_types ``` diff --git a/website/src/content/docs/docs/cli/test.mdx b/website/src/content/docs/docs/cli/test.mdx index 2d18416993..faefa6a0e0 100644 --- a/website/src/content/docs/docs/cli/test.mdx +++ b/website/src/content/docs/docs/cli/test.mdx @@ -11,14 +11,17 @@ building models. Useful for validating test logic independently. ## Usage ```bash -sqb --project-dir test [flags] +sqb test [SELECTOR ...] [flags] ``` +Positional `SELECTOR` arguments work like `--select`, and both forms can be combined. `--project-dir `, `--no-color`, and `--debug` can go before or after the command name. + ## Flags | Flag | Description | |------|-------------| | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | +| `SELECTOR ...` | Positional selectors, the same as `--select` | | `--select`, `-s` | Select tests targeting specific models | | `--exclude` | Exclude tests targeting specific models | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | @@ -38,5 +41,5 @@ are strings in JSON so their scale is preserved. sqb test # Run tests for a specific model -sqb test --select stg_orders +sqb test stg_orders ``` diff --git a/website/src/content/docs/docs/concepts/selectors.mdx b/website/src/content/docs/docs/concepts/selectors.mdx index 9789836817..3042ae2ae4 100644 --- a/website/src/content/docs/docs/concepts/selectors.mdx +++ b/website/src/content/docs/docs/concepts/selectors.mdx @@ -8,12 +8,13 @@ Selectors let you scope commands to specific subsets of your project. They work ## Basic usage ```bash +sqb build daily_revenue +sqb build daily_revenue customer_status_snapshot sqb build --select daily_revenue -sqb build --select daily_revenue customer_status_snapshot sqb build --exclude stg_customers ``` -`--select` (or `-s` for short) accepts one or more names. Multiple values are unioned. Space-separated names within one `--select` are also unioned. `--exclude` subtracts from the selected set. +Selectors can be written as positional arguments on `compile`, `plan`, `build`, `test`, `seed`, `freshness`, `audit`, and `check`, or with `--select` (or `-s` for short) on any command that selects. Both forms accept one or more names, can be combined, and are unioned. `--exclude` subtracts from the selected set. When no `--select` is provided, all models are selected. diff --git a/website/src/content/docs/docs/quickstart.mdx b/website/src/content/docs/docs/quickstart.mdx index 3a63e6543e..80cb13f15d 100644 --- a/website/src/content/docs/docs/quickstart.mdx +++ b/website/src/content/docs/docs/quickstart.mdx @@ -184,10 +184,10 @@ The models and functions are no longer listed as first run, and the seed is curr ```bash # Rebuild a specific model -sqb build --select daily_revenue +sqb build daily_revenue # Rebuild models under a path -sqb build --select path:models/marts +sqb build path:models/marts # Full refresh of everything sqb build --full-refresh