diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/diff.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/diff.md index c340299bd1..3a4ecc75a4 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/diff.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/diff.md @@ -6,20 +6,30 @@ Online: https://sqlbuild.com/docs/cli/diff/ +## Contents + +- Usage +- Flags +- Size guard +- Examples +- Exit codes + Compares schemas and optionally row-level data between two targets (e.g. `prod:dev`), or between two read-only SQL queries. See [Data Diffs](../concepts/diff.md) for detailed usage. ## Usage ```bash # Model mode -sqb diff : --select [flags] +sqb diff [:] [] --select [flags] # Query mode -sqb diff --left-query --right-query (--key | --unkeyed) [flags] +sqb diff --left-query --right-query (--key ... | --unkeyed) [flags] sqb diff --left-query --right-query --schema-only [flags] ``` -In model mode, the first argument is a positional `FROM:TO` range, `--select` is required, and exactly one mode is required: `--full`, `--schema-only`, or `--bounded `. +In model mode, the first argument is a positional `FROM:TO` range, `--select` is required, and at most one mode can be given: `--full`, `--schema-only`, or `--bounded `. Without a mode, the diff is a full comparison guarded by table size (see [Size guard](#size-guard)). + +`FROM` alone compares `FROM` with the active target, so with `dev` active, `sqb diff prod -s orders` runs as `sqb diff prod:dev -s orders`. This is the same default `sqb clone --from` uses. It fails when no target is active or `FROM` is the active target; pass `FROM:TO` in those cases. `FROM` and `TO` are configured target names. Their database/schema namespaces remain authoritative, while the `TO` target's named connection executes the complete comparison and must @@ -46,9 +56,9 @@ In query mode, omit `FROM:TO` and model selectors; each query identifies its own | `--max-models` | Fail when the selected scope contains more models than this limit | | `--max-columns` | Fail when either side of a model has more columns than this limit | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | -| `--key` | Row key column; repeat for a composite key. Overrides the model's `unique_key` | +| `--key` | Row key columns, for example `--key order_id line_number`. Repeating the flag also works. Overrides the model's `unique_key` | | `--unkeyed` | Compare exact full-row multisets when there is no stable row key | -| `--exclude-column` | Leave a column out of the comparison; repeatable | +| `--exclude-column` | Leave columns out of the comparison; takes several values, and repeating the flag also works | | `--tolerance` | Numeric tolerance such as `amount:absolute=1` or `rate:relative=0.001`; repeatable | | `--left-query`, `--right-query` | Compare two read-only SQL queries instead of models | | `--left-query-file`, `--right-query-file` | Read each query from a UTF-8 file | @@ -62,9 +72,53 @@ In query mode, omit `FROM:TO` and model selectors; each query identifies its own | `--exclude` | Exclude specific models from diffing | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](../concepts/project-configuration.md#command-group-warehouses) and the connection warehouse | +## Size guard + +Without `--full`, `--bounded` or `--schema-only`, model mode first checks table sizes. Before it reads +any table data, SQLBuild reads each selected table's row count from warehouse metadata on both +sides and compares it with that target's +[`max_full_rows`](../concepts/project-configuration.md#diff-limits) limit. The limit is 10,000,000 +rows unless configured, and `"unlimited"` turns the check off for that target. If every table is +within its limits, the full comparison runs as usual. + +Otherwise, the diff stops before reading any rows and exits with `2`. The error lists each blocked model +with its row count and limit on both sides, then gives the commands to run instead: + +```text +error[C270]: full diff stopped before reading data: 1 of 1 selected models exceed the full-comparison row limit or have an unknown size + orders: prod 12,400,000 rows, limit 10,000,000; dev 12,400,000 rows, limit 10,000,000 +Run one of these instead: + sqb diff prod:dev --full --select orders + sqb diff prod:dev --bounded --select orders + sqb diff prod:dev --schema-only --select orders +``` + +The `--bounded` command appears only when every blocked model has a cursor. With `--json` or +`--json-output`, the error document has `status` `incomplete` and a `size_guard` object holding the +same models, sizes, limits and commands. + +A size counts as over the limit when the warehouse cannot report it, for example for views or for +tables without statistics. Sizes come from metadata only: + +| Adapter | Source | +|---------|--------| +| DuckDB, MotherDuck | `duckdb_tables()` `estimated_size` | +| PostgreSQL | `pg_class.reltuples`; unknown until the table has been analyzed | +| Snowflake | `INFORMATION_SCHEMA.TABLES` `ROW_COUNT` | +| BigQuery | Tables API `numRows` | +| Databricks | Statistics from `DESCRIBE TABLE EXTENDED`; unknown until `ANALYZE TABLE ... COMPUTE STATISTICS` has run | +| SQL Server | `sys.partitions` row counts | +| Custom adapters | Unknown unless the adapter implements `estimate_relation_row_count` | + +Metadata counts can be estimates. The guard only decides whether to start a full comparison; the +comparison itself always counts rows exactly. + ## Examples ```bash +# Full diff against the active target, if the tables are within the size limit +sqb diff prod --select customer_status_snapshot + # Full diff of a specific model sqb diff prod:dev --full --select customer_status_snapshot @@ -82,8 +136,11 @@ sqb diff prod:dev --bounded 14d --exhaustive --select order_lines # Compare two queries on a key sqb diff --left-query-file queries/orders_before.sql --right-query-file queries/orders_after.sql --key order_id + +# Composite key and several excluded columns +sqb diff prod:dev --full --select order_lines --key order_id line_number --exclude-column updated_at loaded_at ``` ## Exit codes -Returns `0` when all selected models have no differences, `1` when any model has schema or row differences. Query mode also returns `2` when a safety, schema, or key check prevented a complete comparison, and `3` when setup, execution, or cleanup failed; see [Query output and exit codes](../concepts/diff.md#query-output-and-exit-codes). +Returns `0` when all selected models have no differences, `1` when any model has schema or row differences, and `2` when the size guard stopped a default full diff before reading data. Query mode also returns `2` when a safety, schema, or key check prevented a complete comparison, and `3` when setup, execution, or cleanup failed; see [Query output and exit codes](../concepts/diff.md#query-output-and-exit-codes). diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/diff.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/diff.md index dd7aec6a84..6a4b8531a1 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/diff.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/diff.md @@ -23,7 +23,8 @@ Online: https://sqlbuild.com/docs/concepts/diff/ SQLBuild can compare schemas and row-level data between two targets. This lets you validate that changes produce the expected results before they reach production. -`sqb diff FROM:TO` compares two targets, for example `prod:dev`. +`sqb diff FROM:TO` compares two targets, for example `prod:dev`. `sqb diff FROM` compares `FROM` +with the active target, the same default `sqb clone --from` uses. `FROM` and `TO` resolve the authoritative database/schema namespaces to compare. SQLBuild uses the `TO` target's named connection for the complete comparison and accesses both @@ -37,7 +38,7 @@ sqb diff prod:dev --full --select customer_status_snapshot ## Comparison modes -Every diff requires exactly one mode: +A diff takes at most one mode flag. Without one, it runs a full diff guarded by table size. ### Full diff @@ -47,6 +48,13 @@ Compares both schema and row-level data for the selected models: sqb diff prod:dev --full --select fact_orders ``` +Leaving out the mode flag also runs a full diff, but only after a size check. Before reading any +table data, SQLBuild reads each selected table's row count from warehouse metadata on both sides. +If a table is larger than that side's +[`max_full_rows`](project-configuration.md#diff-limits) limit (10,000,000 rows unless +configured), or its size is unknown, the diff stops without scanning. See +[size guard](../cli/diff.md#size-guard) for details. `--full` skips the check. + Rows are joined on the model's `unique_key` and compared column by column. The output shows: - Row counts for each side - How many rows are equal, unequal, or only in one side @@ -387,4 +395,4 @@ reconciliation, inspection, materialization, comparison, cleanup, and total dura ## Exit codes -In model mode, `sqb diff` returns exit code `0` when all selected models have no differences, and `1` when any model has schema or row differences. Raw-query mode uses the exit codes in [Query output and exit codes](#query-output-and-exit-codes). This makes it usable in CI pipelines as a validation gate. +In model mode, `sqb diff` returns exit code `0` when all selected models have no differences, `1` when any model has schema or row differences, and `2` when the size guard stopped a default full diff before reading data. Raw-query mode uses the exit codes in [Query output and exit codes](#query-output-and-exit-codes). This makes it usable in CI pipelines as a validation gate. diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/project-configuration.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/project-configuration.md index ed614b0cca..38bc13939b 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/project-configuration.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/project-configuration.md @@ -401,6 +401,33 @@ This permits an intentional developer override while keeping the shared target p `sqlbuild_project.toml`. Other target fields and any execution-limit fields omitted locally continue to inherit their project values. +### Diff limits + +`sqb diff` without `--full`, `--bounded` or `--schema-only` runs a full comparison only when every +selected table is small enough. Each target sets the largest table it will compare in full this +way: + +```toml +[targets.prod.diff] +max_full_rows = 50_000_000 + +[targets.dev.diff] +max_full_rows = "unlimited" +``` + +`max_full_rows` is a positive integer or `"unlimited"`. When it is not set, the limit is +10,000,000 rows. Each side of a comparison uses its own target's limit. See +[`sqb diff`](../cli/diff.md#size-guard) for how sizes are read and what happens when a table is too +large. + +`sqlbuild_local.toml` accepts the same section, and a local value replaces the project value for +that target: + +```toml +[targets.dev.diff] +max_full_rows = 100_000_000 +``` + ## Defaults Project-wide defaults for models, seeds, and functions: diff --git a/src/sqlbuild/adapter/contract/classes/base_adapter.py b/src/sqlbuild/adapter/contract/classes/base_adapter.py index d43aef5d3d..8250838d72 100644 --- a/src/sqlbuild/adapter/contract/classes/base_adapter.py +++ b/src/sqlbuild/adapter/contract/classes/base_adapter.py @@ -43,6 +43,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RowDiffCoverage, RowDiffPreparedRelations, RowDiffResult, @@ -1570,6 +1571,21 @@ def inspect_row_diff_coverage( end_cursor=end_cursor, ) + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Report an unknown row count for custom adapters without metadata support.""" + + return RelationRowCountEstimate( + row_count=None, + detail=f"adapter '{self.adapter_name}' does not report row counts from metadata", + ) + def _build_row_diff_relation_ctes( self, *, diff --git a/src/sqlbuild/adapter/contract/classes/diff.py b/src/sqlbuild/adapter/contract/classes/diff.py index ed0fa770cc..ccb8b2078a 100644 --- a/src/sqlbuild/adapter/contract/classes/diff.py +++ b/src/sqlbuild/adapter/contract/classes/diff.py @@ -7,6 +7,7 @@ from sqlbuild.adapter.contract.models import ( CursorValue, + RelationRowCountEstimate, RowDiffCoverage, RowDiffResult, RowDiffSampleRow, @@ -75,6 +76,18 @@ def inspect_row_diff_coverage( """Return exact bounded row count and cursor extent for one relation.""" ... + @abstractmethod + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Return the metadata row count for one relation without scanning its data.""" + ... + @abstractmethod def sample_unequal_rows( self, diff --git a/src/sqlbuild/adapter/contract/classes/duckdb_backed_adapter.py b/src/sqlbuild/adapter/contract/classes/duckdb_backed_adapter.py index 7db0414c0a..51f224517a 100644 --- a/src/sqlbuild/adapter/contract/classes/duckdb_backed_adapter.py +++ b/src/sqlbuild/adapter/contract/classes/duckdb_backed_adapter.py @@ -50,6 +50,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RowDiffColumnResult, RowDiffCoverage, RowDiffPreparedRelations, @@ -2034,6 +2035,40 @@ def inspect_row_diff_coverage( end_cursor=end_cursor, ) + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Read DuckDB's estimated table size from the catalog.""" + + database_filter: str = ( + "database_name = current_database()" + if database is None + else f"lower(database_name) = lower({_duckdb_string_literal(database)})" + ) + schema_filter: str = ( + "schema_name = current_schema()" + if schema is None + else f"lower(schema_name) = lower({_duckdb_string_literal(schema)})" + ) + row: tuple[Any, ...] | None = self.execute( + connection=connection, + sql=( + "SELECT estimated_size FROM duckdb_tables() WHERE " + f"{database_filter} AND {schema_filter} " + f"AND lower(table_name) = lower({_duckdb_string_literal(name)})" + ), + ).fetchone() + if row is None or row[0] is None: + return RelationRowCountEstimate( + row_count=None, detail="no table size metadata (view or missing table)" + ) + return RelationRowCountEstimate(row_count=int(row[0])) + def diff_schema( self, *, diff --git a/src/sqlbuild/adapter/contract/models.py b/src/sqlbuild/adapter/contract/models.py index 36ae6d97ec..c6e237e612 100644 --- a/src/sqlbuild/adapter/contract/models.py +++ b/src/sqlbuild/adapter/contract/models.py @@ -322,6 +322,14 @@ class RowDiffCoverage: maximum_cursor: object | None = None +@dataclass(frozen=True) +class RelationRowCountEstimate: + """Metadata row count read without scanning data; ``row_count`` is None when unknown.""" + + row_count: int | None + detail: str | None = None + + @dataclass(frozen=True) class RowDiffSampling: """Deterministic key-sampling policy for a row comparison.""" diff --git a/src/sqlbuild/adapters/bigquery/classes/bigquery_adapter.py b/src/sqlbuild/adapters/bigquery/classes/bigquery_adapter.py index 4a131f271a..960a6a6dc2 100644 --- a/src/sqlbuild/adapters/bigquery/classes/bigquery_adapter.py +++ b/src/sqlbuild/adapters/bigquery/classes/bigquery_adapter.py @@ -56,6 +56,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RenderedRetentionChange, RetentionRequest, RetentionState, @@ -127,6 +128,7 @@ INTEGER_METADATA_TYPE_NAME, INTEGER_TYPE_TOKEN, INTEGER_WIRE_TYPE_NAME, + NATIVE_TABLE_TYPE, NOT_FOUND_ERROR_CLASS_NAME, SELECT_STATEMENT_TYPE, TABLE_NAME_WILDCARD, @@ -2518,6 +2520,29 @@ def normalize_row_diff_numeric_type(self, column_type: str) -> str | None: def format_row_diff_decimal_sql(self, value: Decimal) -> str: return format(value, "f") + def estimate_relation_row_count( + self, + *, + connection: _BigQueryConnection, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Read ``numRows`` from the tables API; only native tables report a count.""" + + if schema is None: + return RelationRowCountEstimate(row_count=None, detail="relation has no dataset") + table: Any = connection.client.get_table( + self._build_table_id(database=database, schema=schema, name=name) + ) + table_type: str = str(getattr(table, "table_type", NATIVE_TABLE_TYPE)).upper() + num_rows: object | None = getattr(table, "num_rows", None) + if table_type != NATIVE_TABLE_TYPE or not isinstance(num_rows, int): + return RelationRowCountEstimate( + row_count=None, detail=f"no row count ({table_type.lower()})" + ) + return RelationRowCountEstimate(row_count=num_rows) + def inspect_row_diff_coverage( self, *, diff --git a/src/sqlbuild/adapters/bigquery/constants.py b/src/sqlbuild/adapters/bigquery/constants.py index 8419dafa9f..86105f9128 100644 --- a/src/sqlbuild/adapters/bigquery/constants.py +++ b/src/sqlbuild/adapters/bigquery/constants.py @@ -31,3 +31,4 @@ "clone or snapshot", ) MAX_CLONE_REFUSAL_ERROR_CHAIN: int = 8 +NATIVE_TABLE_TYPE: str = "TABLE" diff --git a/src/sqlbuild/adapters/databricks/classes/databricks_adapter.py b/src/sqlbuild/adapters/databricks/classes/databricks_adapter.py index d8aceadc17..8bf639e22b 100644 --- a/src/sqlbuild/adapters/databricks/classes/databricks_adapter.py +++ b/src/sqlbuild/adapters/databricks/classes/databricks_adapter.py @@ -54,6 +54,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RenderedRetentionChange, RetentionRequest, RetentionState, @@ -117,6 +118,8 @@ DELTA_DEFAULT_DELETED_FILE_RETENTION_DAYS, DELTA_DEFAULT_LOG_RETENTION_DAYS, DELTA_RELATION_FORMAT, + DESCRIBE_DETAIL_ROW_WIDTH, + DETAILED_TABLE_INFORMATION_MARKER, NON_ROW_RESULT_COLUMN_NAMES, RELATION_NOT_FOUND_ERROR_CLASSES, TABLE_RELATION_METADATA_TYPES, @@ -2485,6 +2488,51 @@ def alter_column_types( for statement in statements: self.execute(connection=connection, sql=statement) + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Read ``ANALYZE TABLE`` statistics; ``DESCRIBE DETAIL`` has no row count.""" + + target: str = ".".join( + self.render_identifier(part) for part in (database, schema, name) if part is not None + ) + cursor: Any = connection.cursor() + try: + cursor.execute(f"DESCRIBE TABLE EXTENDED {target}") + rows: list[tuple[Any, ...]] = cursor.fetchall() + finally: + cursor.close() + labels: list[str] = [str(row[0]).strip().lower() if row else "" for row in rows] + detail_start: int = ( + labels.index(DETAILED_TABLE_INFORMATION_MARKER) + 1 + if DETAILED_TABLE_INFORMATION_MARKER in labels + else 0 + ) + details: dict[str, str] = { + str(row[0]).strip().lower(): str(row[1]) + for row in rows[detail_start:] + if len(row) >= DESCRIBE_DETAIL_ROW_WIDTH and row[1] is not None + } + relation_type: str = details.get("type", "").strip().lower() + if relation_type not in TABLE_RELATION_METADATA_TYPES: + return RelationRowCountEstimate( + row_count=None, detail=f"no row count ({relation_type or 'unknown type'})" + ) + statistics_match: re.Match[str] | None = re.search( + r"(\d+)\s+rows", details.get("statistics", "") + ) + if statistics_match is None: + return RelationRowCountEstimate( + row_count=None, + detail="table statistics not collected; run ANALYZE TABLE ... COMPUTE STATISTICS", + ) + return RelationRowCountEstimate(row_count=int(statistics_match.group(1))) + def inspect_row_diff_coverage( self, *, diff --git a/src/sqlbuild/adapters/databricks/constants.py b/src/sqlbuild/adapters/databricks/constants.py index c0e2a8635b..5401a62f46 100644 --- a/src/sqlbuild/adapters/databricks/constants.py +++ b/src/sqlbuild/adapters/databricks/constants.py @@ -14,3 +14,5 @@ "NO_SUCH_CATALOG_EXCEPTION", ) UNDEFINED_TABLE_SQLSTATE: str = "42P01" +DETAILED_TABLE_INFORMATION_MARKER: str = "# detailed table information" +DESCRIBE_DETAIL_ROW_WIDTH: int = 2 diff --git a/src/sqlbuild/adapters/postgres/classes/postgres_adapter.py b/src/sqlbuild/adapters/postgres/classes/postgres_adapter.py index 93c15906bc..302b28d445 100644 --- a/src/sqlbuild/adapters/postgres/classes/postgres_adapter.py +++ b/src/sqlbuild/adapters/postgres/classes/postgres_adapter.py @@ -53,6 +53,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RowDiffColumnResult, RowDiffCoverage, RowDiffPreparedRelations, @@ -98,7 +99,10 @@ ) from sqlbuild.adapters.postgres._helpers.view_rebind import render_postgres_view_rebind from sqlbuild.adapters.postgres.classes.postgres_connection import _PostgresConnection -from sqlbuild.adapters.postgres.constants import TABLE_FUNCTION_RETURN_TYPE +from sqlbuild.adapters.postgres.constants import ( + ROW_COUNT_RELATION_KINDS, + TABLE_FUNCTION_RETURN_TYPE, +) from sqlbuild.compiler.compile.types import FunctionLanguage from sqlbuild.compiler.source_freshness.models import SourceFreshnessRecord from sqlbuild.compiler.sql_analysis.models import SqlLexicalSyntax @@ -2247,6 +2251,46 @@ def inspect_row_diff_coverage( end_cursor=end_cursor, ) + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Read the planner's ``reltuples`` estimate, treating unanalyzed data as unknown.""" + + del database + schema_filter: str = ( + "namespace.nspname = current_schema()" + if schema is None + else f"namespace.nspname = {_quote_sql_string(schema)}" + ) + row: tuple[Any, ...] | None = self.execute( + connection=connection, + sql=( + "SELECT relation.relkind, relation.reltuples, pg_relation_size(relation.oid) " + "FROM pg_class AS relation " + "JOIN pg_namespace AS namespace ON namespace.oid = relation.relnamespace " + f"WHERE {schema_filter} AND relation.relname = {_quote_sql_string(name)}" + ), + ).fetchone() + if row is None: + return RelationRowCountEstimate(row_count=None, detail="relation not found") + if str(row[0]) not in ROW_COUNT_RELATION_KINDS: + return RelationRowCountEstimate( + row_count=None, detail="no table statistics (view or partitioned table)" + ) + estimated_rows: float = float(row[1]) + if estimated_rows > 0: + return RelationRowCountEstimate(row_count=int(estimated_rows)) + if int(row[2]) == 0: + return RelationRowCountEstimate(row_count=0) + return RelationRowCountEstimate( + row_count=None, detail="table statistics not collected; run ANALYZE" + ) + def diff_schema( self, *, diff --git a/src/sqlbuild/adapters/postgres/constants.py b/src/sqlbuild/adapters/postgres/constants.py index 37a9945e05..d7d43f61b3 100644 --- a/src/sqlbuild/adapters/postgres/constants.py +++ b/src/sqlbuild/adapters/postgres/constants.py @@ -1,3 +1,4 @@ """PostgreSQL adapter constants.""" TABLE_FUNCTION_RETURN_TYPE: str = "TABLE" +ROW_COUNT_RELATION_KINDS: frozenset[str] = frozenset({"r", "m"}) diff --git a/src/sqlbuild/adapters/snowflake/classes/snowflake_adapter.py b/src/sqlbuild/adapters/snowflake/classes/snowflake_adapter.py index f56a8bf6a7..8a4bd5bdb3 100644 --- a/src/sqlbuild/adapters/snowflake/classes/snowflake_adapter.py +++ b/src/sqlbuild/adapters/snowflake/classes/snowflake_adapter.py @@ -52,6 +52,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RenderedRetentionChange, RetentionRequest, RetentionState, @@ -1896,6 +1897,26 @@ def _information_schema_relation_exists( schema: str | None, name: str, ) -> bool: + return ( + self._information_schema_table_row( + connection=connection, + database=database, + schema=schema, + name=name, + projection="1", + ) + is not None + ) + + def _information_schema_table_row( + self, + *, + connection: _SnowflakeConnection, + database: str | None, + schema: str | None, + name: str, + projection: str, + ) -> tuple[Any, ...] | None: clauses: list[str] = ["table_name = %s"] params: list[str] = [self._information_schema_identifier(name)] if schema is not None: @@ -1907,13 +1928,13 @@ def _information_schema_relation_exists( cursor: Any = connection.cursor() try: cursor.execute( - "SELECT 1 FROM " + f"SELECT {projection} FROM " + self._information_schema_relation(database=database, name="tables") + " WHERE " + " AND ".join(clauses), tuple(params), ) - return cursor.fetchone() is not None + return cursor.fetchone() finally: cursor.close() @@ -3219,6 +3240,29 @@ def alter_column_types( for statement in statements: self.execute(connection=connection, sql=statement) + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Read ``ROW_COUNT`` from the information schema; views report no count.""" + + row: tuple[Any, ...] | None = self._information_schema_table_row( + connection=connection, + database=database, + schema=schema, + name=name, + projection="row_count", + ) + if row is None: + return RelationRowCountEstimate(row_count=None, detail="relation not found") + if row[0] is None: + return RelationRowCountEstimate(row_count=None, detail="no row count (view)") + return RelationRowCountEstimate(row_count=int(row[0])) + def inspect_row_diff_coverage( self, *, diff --git a/src/sqlbuild/adapters/sqlserver/classes/sqlserver_adapter.py b/src/sqlbuild/adapters/sqlserver/classes/sqlserver_adapter.py index 341356ef4a..2b43f182aa 100644 --- a/src/sqlbuild/adapters/sqlserver/classes/sqlserver_adapter.py +++ b/src/sqlbuild/adapters/sqlserver/classes/sqlserver_adapter.py @@ -55,6 +55,7 @@ RelationGrant, RelationInfo, RelationReadProbe, + RelationRowCountEstimate, RowDiffColumnResult, RowDiffCoverage, RowDiffPreparedRelations, @@ -105,6 +106,7 @@ DIFF_UNSUPPORTED_COMPARISON_TYPES, INFORMATION_SCHEMA_NULLABLE_VALUE, INTEGER_TYPE_TOKEN, + USER_TABLE_OBJECT_TYPE, ) from sqlbuild.compiler.compile.types import FunctionLanguage from sqlbuild.compiler.source_freshness.models import SourceFreshnessRecord @@ -1282,6 +1284,40 @@ def delete_insert_cursor( for stmt in statements: self.execute(connection=connection, sql=stmt) + def estimate_relation_row_count( + self, + *, + connection: Any, + database: str | None, + schema: str | None, + name: str, + ) -> RelationRowCountEstimate: + """Sum heap or clustered-index partition row counts from the catalog.""" + + catalog: str = "" if database is None else f"{self.render_identifier(database)}." + schema_filter: str = ( + "schemas.name = SCHEMA_NAME()" + if schema is None + else f"schemas.name = N{_quote_sql_string(schema)}" + ) + row: tuple[Any, ...] | None = self.execute( + connection=connection, + sql=( + "SELECT objects.type, SUM(partitions.rows) " + f"FROM {catalog}sys.objects AS objects " + f"JOIN {catalog}sys.schemas AS schemas ON schemas.schema_id = objects.schema_id " + f"LEFT JOIN {catalog}sys.partitions AS partitions " + "ON partitions.object_id = objects.object_id AND partitions.index_id IN (0, 1) " + f"WHERE {schema_filter} AND objects.name = N{_quote_sql_string(name)} " + "GROUP BY objects.type" + ), + ).fetchone() + if row is None: + return RelationRowCountEstimate(row_count=None, detail="relation not found") + if str(row[0]).strip().upper() != USER_TABLE_OBJECT_TYPE or row[1] is None: + return RelationRowCountEstimate(row_count=None, detail="no row count (view)") + return RelationRowCountEstimate(row_count=int(row[1])) + def inspect_row_diff_coverage( self, *, diff --git a/src/sqlbuild/adapters/sqlserver/constants.py b/src/sqlbuild/adapters/sqlserver/constants.py index eedbff3431..8c8b78377b 100644 --- a/src/sqlbuild/adapters/sqlserver/constants.py +++ b/src/sqlbuild/adapters/sqlserver/constants.py @@ -11,3 +11,4 @@ ) INFORMATION_SCHEMA_NULLABLE_VALUE: str = "YES" INTEGER_TYPE_TOKEN: str = "INT" +USER_TABLE_OBJECT_TYPE: str = "U" diff --git a/src/sqlbuild/cli/commands/_helpers/diff/execution.py b/src/sqlbuild/cli/commands/_helpers/diff/execution.py index 468a0e7ef1..eb3e0fa90f 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/execution.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/execution.py @@ -12,6 +12,10 @@ from sqlbuild.adapter.contract.models import RelationLookup from sqlbuild.adapter.contract.types import BuiltinAdapter from sqlbuild.adapter.relations.main.relation_lookup import build_relation_lookup +from sqlbuild.cli.commands._helpers.diff.size_guard import ( + build_diff_size_guard_error, + resolve_full_diff_size_limits, +) from sqlbuild.cli.commands._helpers.planning.external_refs import ( resolve_external_sql_reference_resolver, ) @@ -37,6 +41,7 @@ from sqlbuild.compiler.sql_analysis.main.import_polyglot_sql import import_polyglot_sql from sqlbuild.cursor_algebra.models import Duration from sqlbuild.executor.diff.classes.query_artifact_lifecycle import QueryDiffArtifactLifecycle +from sqlbuild.executor.diff.exceptions import FullDiffSizeGuardError from sqlbuild.executor.diff.main.config import parse_cli_tolerance_overrides from sqlbuild.executor.diff.main.execute import execute_diff from sqlbuild.executor.diff.main.execute_query import ( @@ -53,6 +58,7 @@ from sqlbuild.spec.contracts.main.resolve_effective_adapter_name import ( resolve_effective_adapter_name, ) +from sqlbuild.spec.contracts.main.resolve_target_name import resolve_target_name def prepare_direct_diff( @@ -60,10 +66,12 @@ def prepare_direct_diff( ) -> DirectDiffPreparation: """Resolve direct target diff adapter, compiled projects, and limits.""" - if request.from_name is None or request.to_name is None: - raise CliUserError("model diff requires FROM:TO", code="C224") + if request.from_name is None: + raise CliUserError("model diff requires FROM or FROM:TO", code="C224") from_target: str = request.from_name - to_target: str = request.to_name + to_target: str = _resolve_diff_to_target( + request=request, invocation=invocation, from_target=from_target + ) if from_target not in invocation.discovered_inputs.project_config.targets: raise CliUserError(f"unknown diff FROM target '{from_target}'", code="C205") if to_target not in invocation.discovered_inputs.project_config.targets: @@ -115,7 +123,38 @@ def prepare_direct_diff( effective_max_row_only_examples=_effective_max_examples( explicit_value=request.max_row_only_examples, verbose=request.verbose ), + full_size_limits=resolve_full_diff_size_limits( + request=request, + discovered_inputs=invocation.discovered_inputs, + from_target=from_target, + to_target=to_target, + ), + ) + + +def _resolve_diff_to_target( + *, request: DiffCommandRequest, invocation: DiffInvocation, from_target: str +) -> str: + if request.to_name is not None: + return request.to_name + active_target: str | None = resolve_target_name( + project_config=invocation.discovered_inputs.project_config, + local_config=invocation.discovered_inputs.local_config, + selected_target=None, ) + if active_target is None: + raise CliUserError( + "model diff requires FROM:TO when the project has no active target", + code="C271", + help="set default_target, or pass both targets as FROM:TO", + ) + if active_target == from_target: + raise CliUserError( + f"diff FROM '{from_target}' is the active target, so there is nothing to compare", + code="C272", + help="pass both targets as FROM:TO", + ) + return active_target def execute_direct_diff( @@ -146,8 +185,16 @@ def execute_direct_diff( tolerance_overrides=parse_cli_tolerance_overrides( values=request.tolerance_overrides ), + full_size_limits=preparation.full_size_limits, ), ) + except FullDiffSizeGuardError as error: + raise build_diff_size_guard_error( + request=request, + from_target=preparation.from_target, + to_target=preparation.to_target, + error=error, + ) from error finally: preparation.adapter.close(connection) diff --git a/src/sqlbuild/cli/commands/_helpers/diff/invocation.py b/src/sqlbuild/cli/commands/_helpers/diff/invocation.py index fe8e300213..07962d3ebf 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/invocation.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/invocation.py @@ -47,9 +47,9 @@ def _validate_diff_request(*, request: DiffCommandRequest) -> None: selected_modes: int = ( int(request.full) + int(request.schema_only) + int(request.bounded is not None) ) - if selected_modes != 1 and not (query_mode and selected_modes == 0): + if selected_modes > 1: raise CliUserError( - "diff requires exactly one of --full, --schema-only, or --bounded", + "diff accepts at most one of --full, --schema-only, or --bounded", code="C201", ) if request.max_column_examples is not None and request.max_column_examples <= 0: @@ -152,8 +152,8 @@ def _validate_query_diff_request(*, request: DiffCommandRequest) -> None: def _validate_model_diff_request(*, request: DiffCommandRequest) -> None: - if request.from_name is None or request.to_name is None: - raise CliUserError("model diff requires FROM:TO", code="C224") + if request.from_name is None: + raise CliUserError("model diff requires FROM or FROM:TO", code="C224") if request.selected_target is not None: raise CliUserError("model diff does not accept --target", code="C225") if request.left_label is not None or request.right_label is not None: diff --git a/src/sqlbuild/cli/commands/_helpers/diff/json_output.py b/src/sqlbuild/cli/commands/_helpers/diff/json_output.py index 88cd56218e..c3c872d1bd 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/json_output.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/json_output.py @@ -52,21 +52,30 @@ def render_diff_json_output( ) -def render_diff_error_json(*, status: str, code: str, message: str) -> str: +def render_diff_error_json( + *, + status: str, + code: str, + message: str, + help_text: str | None = None, + size_guard: dict[str, object] | None = None, +) -> str: """Render a machine-readable incomplete or execution-failure outcome.""" - return json.dumps( - { - "schema_version": 1, - "status": status, - "outcome": status, - "error": {"code": code, "message": message}, - "models": [], - "query_comparisons": [], - }, - indent=2, - sort_keys=True, - ) + error: dict[str, object] = {"code": code, "message": message} + if help_text is not None: + error["help"] = help_text + payload: dict[str, object] = { + "schema_version": 1, + "status": status, + "outcome": status, + "error": error, + "models": [], + "query_comparisons": [], + } + if size_guard is not None: + payload["size_guard"] = size_guard + return json.dumps(payload, indent=2, sort_keys=True) def _json_safe_value(value: object) -> object: diff --git a/src/sqlbuild/cli/commands/_helpers/diff/size_guard.py b/src/sqlbuild/cli/commands/_helpers/diff/size_guard.py new file mode 100644 index 0000000000..bfb88e35fc --- /dev/null +++ b/src/sqlbuild/cli/commands/_helpers/diff/size_guard.py @@ -0,0 +1,193 @@ +"""Default full model diff size limits and guard error rendering.""" + +from __future__ import annotations + +import json +import shlex + +from sqlbuild.cli.commands.exceptions import DiffSizeGuardError +from sqlbuild.cli.commands.models import DiffCommandRequest +from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs +from sqlbuild.executor.diff.exceptions import FullDiffSizeGuardError +from sqlbuild.executor.diff.models import FullDiffModelSize, FullDiffSideSize, FullDiffSizeLimits +from sqlbuild.spec.contracts.constants import DEFAULT_DIFF_MAX_FULL_ROWS +from sqlbuild.spec.contracts.main.resolve_target_config import resolve_target_config + +_BOUNDED_WINDOW_PLACEHOLDER: str = "" + + +def resolve_full_diff_size_limits( + *, + request: DiffCommandRequest, + discovered_inputs: DiscoveredProjectInputs, + from_target: str, + to_target: str, +) -> FullDiffSizeLimits | None: + """Return per-side row limits when no explicit comparison mode was requested.""" + + if request.full or request.schema_only or request.bounded is not None: + return None + return FullDiffSizeLimits( + left_target=from_target, + right_target=to_target, + left_max_rows=_target_max_full_rows( + discovered_inputs=discovered_inputs, target_name=from_target + ), + right_max_rows=_target_max_full_rows( + discovered_inputs=discovered_inputs, target_name=to_target + ), + ) + + +def build_diff_size_guard_error( + *, + request: DiffCommandRequest, + from_target: str, + to_target: str, + error: FullDiffSizeGuardError, +) -> DiffSizeGuardError: + """Describe every blocked model and the explicit commands that replace the guarded run.""" + + commands: dict[str, str | None] = _replacement_commands( + request=request, + from_target=from_target, + to_target=to_target, + blocked=error.blocked, + ) + lines: list[str] = [error.message] + model: FullDiffModelSize + for model in error.blocked: + lines.append( + f" {model.name}: {_side_text(model.left)}; {_side_text(model.right)}" + + ("" if model.has_cursor else "; no cursor") + ) + lines.append("Run one of these instead:") + lines.append(f" {commands['full']}") + if commands["bounded"] is not None: + lines.append(f" {commands['bounded']}") + lines.append(f" {commands['schema_only']}") + over_targets: tuple[str, ...] = _over_limit_targets(blocked=error.blocked) + bounded_help: str = ( + "" + if commands["bounded"] is None + else ", --bounded compares a recent cursor window " + f"(replace {_BOUNDED_WINDOW_PLACEHOLDER} with a duration such as 7d)" + ) + return DiffSizeGuardError( + "\n".join(lines), + code="C270", + help=( + f"--full compares every row{bounded_help}, and --schema-only compares columns " + "only; or raise " + + " and ".join(f"targets.{target}.diff.max_full_rows" for target in over_targets) + ), + details={ + "from_target": from_target, + "to_target": to_target, + "models": [_model_payload(model) for model in error.blocked], + "commands": commands, + }, + ) + + +def _over_limit_targets(*, blocked: tuple[FullDiffModelSize, ...]) -> tuple[str, ...]: + targets: dict[str, None] = {} + model: FullDiffModelSize + for model in blocked: + side: FullDiffSideSize + for side in (model.left, model.right): + if side.exceeds_limit: + targets[side.target] = None + return tuple(targets) + + +def _target_max_full_rows( + *, discovered_inputs: DiscoveredProjectInputs, target_name: str +) -> int | None: + configured: int | str | None = resolve_target_config( + project_config=discovered_inputs.project_config, + local_config=discovered_inputs.local_config, + target_name=target_name, + ).diff.max_full_rows + if configured is None: + return DEFAULT_DIFF_MAX_FULL_ROWS + return configured if isinstance(configured, int) else None + + +def _side_text(side: FullDiffSideSize) -> str: + limit: str = "unlimited" if side.max_rows is None else f"{side.max_rows:,}" + if side.row_count is None: + size: str = "size unknown" if side.detail is None else f"size unknown: {side.detail}" + else: + size = f"{side.row_count:,} rows" + return f"{side.target} {size}, limit {limit}" + + +def _model_payload(model: FullDiffModelSize) -> dict[str, object]: + return { + "name": model.name, + "has_cursor": model.has_cursor, + "from": _side_payload(model.left), + "to": _side_payload(model.right), + } + + +def _side_payload(side: FullDiffSideSize) -> dict[str, object]: + return { + "target": side.target, + "relation": side.relation, + "row_count": side.row_count, + "max_full_rows": side.max_rows, + "exceeds_limit": side.exceeds_limit, + "detail": side.detail, + } + + +def _replacement_commands( + *, + request: DiffCommandRequest, + from_target: str, + to_target: str, + blocked: tuple[FullDiffModelSize, ...], +) -> dict[str, str | None]: + scope: tuple[str, ...] = _scope_arguments(request=request) + project: tuple[str, ...] = ( + () if request.project_dir is None else ("--project-dir", str(request.project_dir)) + ) + prefix: tuple[str, ...] = ("sqb", *project, "diff", f"{from_target}:{to_target}") + bounded_available: bool = all(model.has_cursor for model in blocked) + return { + "full": _join((*prefix, "--full", *scope)), + "bounded": ( + _join((*prefix, "--bounded", _BOUNDED_WINDOW_PLACEHOLDER, *scope)) + if bounded_available + else None + ), + "schema_only": _join((*prefix, "--schema-only", *scope)), + } + + +def _scope_arguments(*, request: DiffCommandRequest) -> tuple[str, ...]: + arguments: list[str] = [] + if request.select: + arguments.extend(("--select", *request.select)) + if request.exclude: + arguments.extend(("--exclude", *request.exclude)) + if request.unique_key_override: + arguments.extend(("--key", *request.unique_key_override)) + if request.unkeyed: + arguments.append("--unkeyed") + if request.excluded_columns_override: + arguments.extend(("--exclude-column", *request.excluded_columns_override)) + tolerance: str + for tolerance in request.tolerance_overrides: + arguments.extend(("--tolerance", tolerance)) + if request.cli_vars: + arguments.extend(("--vars", json.dumps(request.cli_vars, sort_keys=True))) + return tuple(arguments) + + +def _join(parts: tuple[str, ...]) -> str: + return " ".join( + part if part == _BOUNDED_WINDOW_PLACEHOLDER else shlex.quote(part) for part in parts + ) diff --git a/src/sqlbuild/cli/commands/_helpers/diff/validation.py b/src/sqlbuild/cli/commands/_helpers/diff/validation.py index ca30f9fa26..8320f2f72f 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/validation.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/validation.py @@ -5,16 +5,15 @@ from sqlbuild.cli.commands.exceptions import CliUserError -def parse_diff_name_range(name_range: str | None) -> tuple[str, str]: - """Parse a diff name range in FROM:TO form.""" +def parse_diff_name_range(name_range: str | None) -> tuple[str, str | None]: + """Parse a diff name range in FROM:TO or FROM form; a bare FROM leaves TO unset.""" if name_range is None: raise CliUserError("diff requires FROM:TO", code="C208") - if name_range.count(":") != 1: - raise CliUserError("diff range must be FROM:TO", code="C209") from_name: str + separator: str to_name: str - from_name, to_name = name_range.split(":", 1) - if not from_name or not to_name: - raise CliUserError("diff range must be FROM:TO", code="C209") - return from_name, to_name + from_name, separator, to_name = name_range.partition(":") + if not from_name or (separator and (not to_name or separator in to_name)): + raise CliUserError("diff range must be FROM:TO or FROM", code="C209") + return from_name, to_name or None diff --git a/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py b/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py index 6e402f2cf4..568f633925 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py @@ -10,7 +10,11 @@ ) from sqlbuild.cli.commands._helpers.skills.update import maintain_sqlbuild_skills from sqlbuild.cli.commands.classes.cli_namespace import CliNamespace -from sqlbuild.cli.commands.exceptions import CliUserError, QueryDiffExecutionError +from sqlbuild.cli.commands.exceptions import ( + CliUserError, + QueryDiffExecutionError, + QueryDiffOutcomeError, +) from sqlbuild.cli.commands.main.entrypoint._dispatch_with_observability import ( dispatch_with_observability, ) @@ -51,9 +55,8 @@ def dispatch_and_handle_errors( if output_error is not None: effective_error = output_error _write_compile_machine_error(args=args, error=effective_error, fallback_code="C000") - outcome_status: object | None = getattr(effective_error, "status", None) - if outcome_status is not None: - print(f"Query diff outcome {outcome_status}", file=sys.stderr) + if isinstance(effective_error, QueryDiffOutcomeError): + print(f"Query diff outcome {effective_error.status}", file=sys.stderr) print( format_expected_error(error=effective_error, fallback_code="C000", use_color=use_color), file=sys.stderr, @@ -162,6 +165,8 @@ def _write_diff_machine_error( status=str(getattr(error, "status", "execution_failed")), code=error.code, message=error.message, + help_text=error.help, + size_guard=getattr(error, "details", None), ) if json_path is not None: try: diff --git a/src/sqlbuild/cli/commands/_helpers/entry/parser.py b/src/sqlbuild/cli/commands/_helpers/entry/parser.py index 26c367b3d0..80a6e37819 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/parser.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/parser.py @@ -441,7 +441,7 @@ def _add_data_parsers( clone_parser.add_argument("--event-output", type=Path, default=None, help=argparse.SUPPRESS) diff_parser: argparse.ArgumentParser = subparsers.add_parser(CliCommand.DIFF) - diff_parser.add_argument("target_range", nargs="?", metavar="FROM:TO") + diff_parser.add_argument("target_range", nargs="?", metavar="FROM[:TO]") _add_sql_analysis_override(diff_parser) diff_parser.add_argument("--full", action="store_true", default=False) diff_parser.add_argument("--schema-only", action="store_true", default=False) @@ -472,7 +472,11 @@ def _add_data_parsers( "--right-label", default=None, help="display label for the right query" ) diff_parser.add_argument( - "--key", action="append", default=[], help="row identity column; repeat for composite keys" + "--key", + nargs="+", + action="extend", + default=[], + help="row identity columns; several values or repeated flags form a composite key", ) diff_parser.add_argument( "--unkeyed", @@ -481,7 +485,11 @@ def _add_data_parsers( help="compare exact full-row multiplicities without a key", ) diff_parser.add_argument( - "--exclude-column", action="append", default=[], help="column to omit from row comparison" + "--exclude-column", + nargs="+", + action="extend", + default=[], + help="columns to omit from row comparison; takes several values or repeated flags", ) diff_parser.add_argument( "--tolerance", diff --git a/src/sqlbuild/cli/commands/exceptions.py b/src/sqlbuild/cli/commands/exceptions.py index fd9bc6a624..db0843f5fc 100644 --- a/src/sqlbuild/cli/commands/exceptions.py +++ b/src/sqlbuild/cli/commands/exceptions.py @@ -70,3 +70,21 @@ def __init__(self, message: str, *, code: str | None = None, help: str | None = code=code, help=help, ) + + +class DiffSizeGuardError(CliUserError): + """A default full model diff stopped before reading data because of table sizes.""" + + status: str = "incomplete" + exit_code: int = 2 + + def __init__( + self, + message: str, + *, + details: dict[str, object], + code: str | None = None, + help: str | None = None, + ) -> None: + super().__init__(message, code=code, help=help) + self.details = details diff --git a/src/sqlbuild/cli/commands/models.py b/src/sqlbuild/cli/commands/models.py index 1733b95d8d..bb0f8c2b84 100644 --- a/src/sqlbuild/cli/commands/models.py +++ b/src/sqlbuild/cli/commands/models.py @@ -75,7 +75,7 @@ from sqlbuild.cost.types import CostStatus from sqlbuild.executor.build.models import BuildExecutionResult, SeedExecutionResult from sqlbuild.executor.clone.models import CloneExecutionResult -from sqlbuild.executor.diff.models import DiffExecutionResult +from sqlbuild.executor.diff.models import DiffExecutionResult, FullDiffSizeLimits from sqlbuild.executor.janitor.models import JanitorPlan from sqlbuild.executor.load.models import LoadExecutionResult from sqlbuild.executor.python_nodes.models import PythonNodeExecutionResult @@ -523,6 +523,7 @@ class DirectDiffPreparation: connection_config: dict[str, object] effective_max_column_examples: int effective_max_row_only_examples: int + full_size_limits: FullDiffSizeLimits | None = None @dataclass(frozen=True) diff --git a/src/sqlbuild/compiler/discovery/_helpers/yml/project.py b/src/sqlbuild/compiler/discovery/_helpers/yml/project.py index d2de618858..82cdc324bc 100644 --- a/src/sqlbuild/compiler/discovery/_helpers/yml/project.py +++ b/src/sqlbuild/compiler/discovery/_helpers/yml/project.py @@ -20,6 +20,7 @@ ) from sqlbuild.compiler.compile.constants import MAX_MICROBATCHES_CONFIG_KEY, TEMPLATE_OPEN_TOKEN from sqlbuild.compiler.discovery._helpers.validation.supported_keys import unsupported_keys_help +from sqlbuild.compiler.discovery._helpers.yml.target_diff import load_target_diff from sqlbuild.compiler.discovery._helpers.yml.target_warehouses import ( load_target_warehouses, reject_unsupported_target_warehouses, @@ -1332,6 +1333,7 @@ def _project_target(local: LocalTargetConfig) -> TargetConfig: missing_migration_origin=local.missing_migration_origin or defaults.missing_migration_origin, execution_limits=local.execution_limits, + diff=local.diff, warehouses=local.warehouses, ) @@ -1420,6 +1422,9 @@ def _load_local_targets(*, payload: object, file_path: Path) -> dict[str, LocalT policy=MissingMigrationOriginPolicy, ), execution_limits=execution_limits, + diff=load_target_diff( + payload=target_mapping.get("diff"), target_name=target_name, file_path=file_path + ), warehouses=load_target_warehouses( payload=target_mapping.get("warehouses"), target_name=target_name, @@ -1497,6 +1502,7 @@ def _validate_target_keys( "missing_migration_origin", "clone", "execution_limits", + "diff", "warehouses", } ), diff --git a/src/sqlbuild/compiler/discovery/_helpers/yml/target_diff.py b/src/sqlbuild/compiler/discovery/_helpers/yml/target_diff.py new file mode 100644 index 0000000000..e85c326c03 --- /dev/null +++ b/src/sqlbuild/compiler/discovery/_helpers/yml/target_diff.py @@ -0,0 +1,44 @@ +"""Loading and validation for per-target diff limits.""" + +from __future__ import annotations + +from pathlib import Path +from typing import cast + +from sqlbuild.compiler.discovery._helpers.validation.supported_keys import ( + reject_unknown_mapping_keys, +) +from sqlbuild.compiler.discovery.exceptions import ProjectConfigError +from sqlbuild.spec.contracts.constants import UNLIMITED_DIFF_FULL_ROWS +from sqlbuild.spec.contracts.models import TargetDiffConfig + +_MAX_FULL_ROWS_KEY: str = "max_full_rows" + + +def load_target_diff(*, payload: object, target_name: str, file_path: Path) -> TargetDiffConfig: + """Parse one target's optional `[targets..diff]` section.""" + + label: str = f"targets.{target_name}.diff" + if payload is None: + return TargetDiffConfig() + if not isinstance(payload, dict): + raise ProjectConfigError(f"{file_path} {label} must be a mapping") + mapping: dict[str, object] = cast(dict[str, object], payload) + reject_unknown_mapping_keys( + mapping=mapping, + allowed=frozenset({_MAX_FULL_ROWS_KEY}), + file_path=file_path, + label=label, + error_class=ProjectConfigError, + ) + value: object | None = mapping.get(_MAX_FULL_ROWS_KEY) + if value is None: + return TargetDiffConfig() + if value == UNLIMITED_DIFF_FULL_ROWS: + return TargetDiffConfig(max_full_rows="unlimited") + if isinstance(value, bool) or not isinstance(value, int) or value < 1: + raise ProjectConfigError( + f"{file_path} {label}.{_MAX_FULL_ROWS_KEY} must be an integer >= 1 or " + f"'{UNLIMITED_DIFF_FULL_ROWS}'" + ) + return TargetDiffConfig(max_full_rows=value) diff --git a/src/sqlbuild/executor/diff/_helpers/size_guard.py b/src/sqlbuild/executor/diff/_helpers/size_guard.py new file mode 100644 index 0000000000..64f1eeb404 --- /dev/null +++ b/src/sqlbuild/executor/diff/_helpers/size_guard.py @@ -0,0 +1,88 @@ +"""Metadata size guard for default full model comparisons.""" + +from __future__ import annotations + +from typing import Any + +from sqlbuild.adapter.contract.classes.base_adapter import BaseAdapter +from sqlbuild.adapter.contract.models import RelationRowCountEstimate +from sqlbuild.executor.diff._helpers.selection import qualified_name +from sqlbuild.executor.diff.exceptions import FullDiffSizeGuardError +from sqlbuild.executor.diff.models import FullDiffModelSize, FullDiffSideSize, FullDiffSizeLimits +from sqlbuild.spec.contracts.main.get_config_str import get_config_str + + +def enforce_full_diff_size_limits( + *, + adapter: BaseAdapter, + connection: Any, + model_pairs: tuple[tuple[str, Any, Any], ...], + limits: FullDiffSizeLimits, +) -> None: + """Stop before reading data when any side is over its limit or has an unknown size.""" + + sizes: tuple[FullDiffModelSize, ...] = tuple( + FullDiffModelSize( + name=name, + left=_side_size( + adapter=adapter, + connection=connection, + model=left_model, + target=limits.left_target, + max_rows=limits.left_max_rows, + ), + right=_side_size( + adapter=adapter, + connection=connection, + model=right_model, + target=limits.right_target, + max_rows=limits.right_max_rows, + ), + has_cursor=get_config_str(values=right_model.config.values, key="cursor") is not None, + ) + for name, left_model, right_model in model_pairs + ) + blocked: tuple[FullDiffModelSize, ...] = tuple(size for size in sizes if size.exceeds_limit) + if blocked: + raise FullDiffSizeGuardError( + f"full diff stopped before reading data: {len(blocked)} of {len(sizes)} " + "selected models exceed the full-comparison row limit or have an unknown size", + blocked=blocked, + ) + + +def _side_size( + *, + adapter: BaseAdapter, + connection: Any, + model: Any, + target: str, + max_rows: int | None, +) -> FullDiffSideSize: + relation: str = qualified_name(adapter=adapter, model=model) + if max_rows is None: + return FullDiffSideSize(target=target, relation=relation, row_count=None, max_rows=None) + estimate: RelationRowCountEstimate = _estimate_row_count( + adapter=adapter, connection=connection, model=model + ) + return FullDiffSideSize( + target=target, + relation=relation, + row_count=estimate.row_count, + max_rows=max_rows, + detail=estimate.detail, + ) + + +def _estimate_row_count( + *, adapter: BaseAdapter, connection: Any, model: Any +) -> RelationRowCountEstimate: + try: + return adapter.estimate_relation_row_count( + connection=connection, + database=model.destination.database, + schema=model.destination.schema, + name=model.destination.name, + ) + except Exception as error: + return RelationRowCountEstimate(row_count=None, detail=f"metadata lookup failed: {error}") diff --git a/src/sqlbuild/executor/diff/exceptions.py b/src/sqlbuild/executor/diff/exceptions.py new file mode 100644 index 0000000000..78bffc3251 --- /dev/null +++ b/src/sqlbuild/executor/diff/exceptions.py @@ -0,0 +1,16 @@ +"""Diff execution exceptions.""" + +from __future__ import annotations + +from sqlbuild.errors.contracts.exceptions import ExecutorInputError +from sqlbuild.executor.diff.models import FullDiffModelSize + + +class FullDiffSizeGuardError(ExecutorInputError): + """A default full comparison stopped before reading data because a table is too large.""" + + code: str = "X320" + + def __init__(self, message: str, *, blocked: tuple[FullDiffModelSize, ...]) -> None: + super().__init__(message) + self.blocked = blocked diff --git a/src/sqlbuild/executor/diff/main/execute.py b/src/sqlbuild/executor/diff/main/execute.py index bc73468727..83395cbe0e 100644 --- a/src/sqlbuild/executor/diff/main/execute.py +++ b/src/sqlbuild/executor/diff/main/execute.py @@ -8,6 +8,7 @@ from sqlbuild.errors.contracts.exceptions import ExecutorInputError from sqlbuild.executor.diff._helpers.execution import execute_model_diff from sqlbuild.executor.diff._helpers.selection import is_disabled +from sqlbuild.executor.diff._helpers.size_guard import enforce_full_diff_size_limits from sqlbuild.executor.diff.models import DiffExecutionOptions, DiffExecutionResult, ModelDiffResult @@ -34,7 +35,7 @@ def execute_diff( f"{options.max_models}", code="X303", ) - results: list[ModelDiffResult] = [] + model_pairs: list[tuple[str, Any, Any]] = [] name: str for name in selected_names: left_model: Any | None = left_models.get(name) @@ -44,15 +45,23 @@ def execute_diff( f"diff selected model '{name}' does not exist in both environments", code="X301", ) - - results.append( - execute_model_diff( - adapter=adapter, - connection=connection, - name=name, - left_model=left_model, - right_model=right_model, - options=options, - ) + model_pairs.append((name, left_model, right_model)) + if options.full_size_limits is not None: + enforce_full_diff_size_limits( + adapter=adapter, + connection=connection, + model_pairs=tuple(model_pairs), + limits=options.full_size_limits, + ) + results: list[ModelDiffResult] = [ + execute_model_diff( + adapter=adapter, + connection=connection, + name=pair_name, + left_model=left_model, + right_model=right_model, + options=options, ) + for pair_name, left_model, right_model in model_pairs + ] return DiffExecutionResult(model_results=tuple(results)) diff --git a/src/sqlbuild/executor/diff/models.py b/src/sqlbuild/executor/diff/models.py index 4f158ac1aa..c22939d702 100644 --- a/src/sqlbuild/executor/diff/models.py +++ b/src/sqlbuild/executor/diff/models.py @@ -52,6 +52,51 @@ class RowDiffSamplingOverride: exhaustive: bool = False +@dataclass(frozen=True) +class FullDiffSizeLimits: + """Per-side row limits guarding a default full comparison; ``None`` means unlimited.""" + + left_target: str + right_target: str + left_max_rows: int | None + right_max_rows: int | None + + +@dataclass(frozen=True) +class FullDiffSideSize: + """Metadata row count for one side of a guarded full comparison.""" + + target: str + relation: str + row_count: int | None + max_rows: int | None + detail: str | None = None + + @property + def exceeds_limit(self) -> bool: + """Return whether this side blocks the comparison; unknown sizes count as over.""" + + return self.max_rows is not None and ( + self.row_count is None or self.row_count > self.max_rows + ) + + +@dataclass(frozen=True) +class FullDiffModelSize: + """Both sides' metadata row counts for one model in a guarded full comparison.""" + + name: str + left: FullDiffSideSize + right: FullDiffSideSize + has_cursor: bool + + @property + def exceeds_limit(self) -> bool: + """Return whether either side blocks the comparison.""" + + return self.left.exceeds_limit or self.right.exceeds_limit + + @dataclass(frozen=True) class DiffExecutionOptions: """Row-diff mode, diagnostic, and invocation override options.""" @@ -69,6 +114,7 @@ class DiffExecutionOptions: excluded_columns_override: tuple[str, ...] = () tolerance_overrides: RowDiffTolerances | None = None comparison_name: str | None = None + full_size_limits: FullDiffSizeLimits | None = None @dataclass(frozen=True) diff --git a/src/sqlbuild/spec/contracts/_helpers/targets.py b/src/sqlbuild/spec/contracts/_helpers/targets.py index 0150104fdf..8a5ae5395d 100644 --- a/src/sqlbuild/spec/contracts/_helpers/targets.py +++ b/src/sqlbuild/spec/contracts/_helpers/targets.py @@ -11,6 +11,7 @@ LocalTargetConfig, ProjectConfig, TargetConfig, + TargetDiffConfig, TargetWarehousesConfig, ) @@ -126,6 +127,13 @@ def resolve_target_config( project_clone=project_target.clone, local_clone=local_target.clone, ), + diff=TargetDiffConfig( + max_full_rows=( + local_target.diff.max_full_rows + if local_target.diff.max_full_rows is not None + else project_target.diff.max_full_rows + ) + ), warehouses=TargetWarehousesConfig( build=( local_target.warehouses.build diff --git a/src/sqlbuild/spec/contracts/constants.py b/src/sqlbuild/spec/contracts/constants.py index 9ad47f86c4..917e001590 100644 --- a/src/sqlbuild/spec/contracts/constants.py +++ b/src/sqlbuild/spec/contracts/constants.py @@ -17,3 +17,5 @@ "snapshot", ) CONNECTION_WAREHOUSE_KEY: str = "warehouse" +UNLIMITED_DIFF_FULL_ROWS: str = "unlimited" +DEFAULT_DIFF_MAX_FULL_ROWS: int = 10_000_000 diff --git a/src/sqlbuild/spec/contracts/models.py b/src/sqlbuild/spec/contracts/models.py index 09d524ad73..4abc24f472 100644 --- a/src/sqlbuild/spec/contracts/models.py +++ b/src/sqlbuild/spec/contracts/models.py @@ -5,6 +5,7 @@ from dataclasses import dataclass, field from decimal import Decimal from pathlib import Path +from typing import Literal from sqlbuild.compiler.auditing.models import MeasurementThresholds from sqlbuild.compiler.auditing.types import AuditSeverity @@ -103,6 +104,13 @@ class ExecutionLimitsConfig: remediation: str | None = None +@dataclass(frozen=True) +class TargetDiffConfig: + """Optional target-scoped data-diff limits.""" + + max_full_rows: int | Literal["unlimited"] | None = None + + @dataclass(frozen=True) class TargetWarehousesConfig: """Optional per-command-group default warehouses for one target.""" @@ -142,6 +150,7 @@ class TargetConfig: time_travel_retention_decrease: RetentionDecreasePolicy = RetentionDecreasePolicy.DENY missing_migration_origin: MissingMigrationOriginPolicy = MissingMigrationOriginPolicy.ALLOW execution_limits: ExecutionLimitsConfig = field(default_factory=ExecutionLimitsConfig) + diff: TargetDiffConfig = field(default_factory=TargetDiffConfig) warehouses: TargetWarehousesConfig = field(default_factory=TargetWarehousesConfig) @@ -167,6 +176,7 @@ class LocalTargetConfig: time_travel_retention_decrease: RetentionDecreasePolicy | None = None missing_migration_origin: MissingMigrationOriginPolicy | None = None execution_limits: ExecutionLimitsConfig = field(default_factory=ExecutionLimitsConfig) + diff: TargetDiffConfig = field(default_factory=TargetDiffConfig) warehouses: TargetWarehousesConfig = field(default_factory=TargetWarehousesConfig) diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py index 5bc7644286..2a7c7a25ca 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py @@ -81,3 +81,18 @@ class DiffJsonOutputE2ETestCase: expected_scope: str expected_population: int expected_compared: int + + +@dataclass(frozen=True) +class DefaultFullDiffE2ETestCase: + """One model diff without a mode flag, a defaulted TO target, or multi-value flags.""" + + description: str + command: tuple[str, ...] + expected_exit_code: int + local_config_suffix: str = "" + expected_stdout_fragments: tuple[str, ...] = field(default_factory=tuple) + expected_stderr_fragments: tuple[str, ...] = field(default_factory=tuple) + unexpected_stdout_fragments: tuple[str, ...] = field(default_factory=tuple) + expected_json_status: str | None = None + expected_json_blocked_models: tuple[str, ...] = field(default_factory=tuple) diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_default_full_diff.py b/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_default_full_diff.py new file mode 100644 index 0000000000..65a5513c1d --- /dev/null +++ b/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_default_full_diff.py @@ -0,0 +1,253 @@ +"""E2E tests for default TO targets, the guarded default full diff and multi-value flags.""" + +from __future__ import annotations + +import json +import subprocess +from pathlib import Path +from typing import Any + +import pytest + +from tests.e2e.src.sqlbuild.cli.commands.main.diff._test_types import ( + DefaultFullDiffE2ETestCase, +) +from tests.e2e.src.sqlbuild.cli.commands.main.diff.helpers import ( + build_both_environments, + prepare_diff_project, +) +from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import run_sqb + +_SMALL_DEV_LIMIT: str = "\n[targets.dev.diff]\nmax_full_rows = 1\n" +_UNLIMITED: str = ( + '\n[targets.prod.diff]\nmax_full_rows = "unlimited"\n' + '\n[targets.dev.diff]\nmax_full_rows = "unlimited"\n' +) + + +@pytest.mark.parametrize( + "test_case", + [ + DefaultFullDiffE2ETestCase( + description="bare FROM compares with the active target and runs a full diff", + command=( + "--no-color", + "diff", + "prod", + "--select", + "orders_snapshot", + "customer_totals", + ), + expected_exit_code=0, + expected_stdout_fragments=( + "prod vs dev", + "orders_snapshot", + "customer_totals", + "No changed columns.", + ), + ), + DefaultFullDiffE2ETestCase( + description="table over the TO limit stops before reading data", + command=( + "--no-color", + "diff", + "prod", + "--select", + "orders_snapshot", + "customer_totals", + ), + local_config_suffix=_SMALL_DEV_LIMIT, + expected_exit_code=2, + expected_stderr_fragments=( + "error[C270]: full diff stopped before reading data: 2 of 2 selected models", + "orders_snapshot: prod 3 rows, limit 10,000,000; dev 3 rows, limit 1", + "customer_totals: prod 2 rows, limit 10,000,000; dev 2 rows, limit 1; no cursor", + "Run one of these instead:", + "sqb --project-dir {project_dir} diff prod:dev --full " + "--select orders_snapshot customer_totals", + "sqb --project-dir {project_dir} diff prod:dev --schema-only " + "--select orders_snapshot customer_totals", + ), + unexpected_stdout_fragments=("No changed columns.",), + ), + DefaultFullDiffE2ETestCase( + description="model with a cursor offers a bounded command", + command=("--no-color", "diff", "prod", "--select", "orders_snapshot"), + local_config_suffix=_SMALL_DEV_LIMIT, + expected_exit_code=2, + expected_stderr_fragments=( + "sqb --project-dir {project_dir} diff prod:dev --bounded " + "--select orders_snapshot", + ), + ), + DefaultFullDiffE2ETestCase( + description="view with unknown size is blocked at the default limit", + command=("--no-color", "diff", "prod", "--select", "stg_orders"), + expected_exit_code=2, + expected_stderr_fragments=( + "stg_orders: prod size unknown: no table size metadata (view or missing table), " + "limit 10,000,000", + ), + ), + DefaultFullDiffE2ETestCase( + description="unlimited targets skip the guard", + command=("--no-color", "diff", "prod", "--select", "stg_orders", "--key", "order_id"), + local_config_suffix=_UNLIMITED, + expected_exit_code=0, + expected_stdout_fragments=("stg_orders", "No changed columns."), + ), + DefaultFullDiffE2ETestCase( + description="explicit full bypasses the guard", + command=("--no-color", "diff", "prod", "--full", "--select", "orders_snapshot"), + local_config_suffix=_SMALL_DEV_LIMIT, + expected_exit_code=0, + expected_stdout_fragments=("No changed columns.",), + ), + DefaultFullDiffE2ETestCase( + description="explicit schema only bypasses the guard", + command=("--no-color", "diff", "prod", "--schema-only", "--select", "stg_orders"), + expected_exit_code=0, + expected_stdout_fragments=("No schema differences.",), + ), + DefaultFullDiffE2ETestCase( + description="two mode flags are rejected", + command=( + "--no-color", + "diff", + "prod", + "--full", + "--schema-only", + "--select", + "orders_snapshot", + ), + expected_exit_code=1, + expected_stderr_fragments=("diff accepts at most one of --full, --schema-only",), + ), + DefaultFullDiffE2ETestCase( + description="bare FROM equal to the active target is rejected", + command=("--no-color", "diff", "dev", "--select", "orders_snapshot"), + expected_exit_code=1, + expected_stderr_fragments=( + "error[C272]: diff FROM 'dev' is the active target", + "pass both targets as FROM:TO", + ), + ), + DefaultFullDiffE2ETestCase( + description="key and exclude column take several values", + command=( + "--no-color", + "diff", + "prod:dev", + "--full", + "--select", + "orders_snapshot", + "--key", + "order_id", + "customer_id", + "--exclude-column", + "status", + "amount_cents", + ), + expected_exit_code=0, + expected_stdout_fragments=("order_id, customer_id", "No changed columns."), + ), + DefaultFullDiffE2ETestCase( + description="query mode key takes several values", + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 1 AS order_id, 1 AS line_number, 5 AS amount UNION ALL SELECT 1, 2, 7", + "--right-query", + "SELECT 1 AS order_id, 1 AS line_number, 5 AS amount UNION ALL SELECT 1, 2, 7", + "--key", + "order_id", + "line_number", + ), + expected_exit_code=0, + expected_stdout_fragments=("order_id, line_number",), + ), + ], + ids=lambda case: case.description, +) +def test_given_model_diff_without_mode_when_running_then_size_guard_and_defaults_apply( + test_case: DefaultFullDiffE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_diff_project(tmp_path) + build_both_environments(project_dir=project_dir) + local_config_path: Path = project_dir / "sqlbuild_local.toml" + local_config_path.write_text( + local_config_path.read_text(encoding="utf-8") + test_case.local_config_suffix, + encoding="utf-8", + ) + + result: subprocess.CompletedProcess[str] = run_sqb( + command=test_case.command, + project_dir=project_dir, + ) + + output: str = result.stdout + result.stderr + assert result.returncode == test_case.expected_exit_code, output + fragment: str + for fragment in test_case.expected_stdout_fragments: + assert fragment in result.stdout, output + for fragment in test_case.expected_stderr_fragments: + assert fragment.format(project_dir=project_dir) in result.stderr, output + for fragment in test_case.unexpected_stdout_fragments: + assert fragment not in result.stdout, output + + +@pytest.mark.parametrize( + "test_case", + [ + DefaultFullDiffE2ETestCase( + description="json output carries the blocked sizes and commands", + command=("--no-color", "diff", "prod:dev", "--select", "orders_snapshot", "--json"), + local_config_suffix=_SMALL_DEV_LIMIT, + expected_exit_code=2, + expected_json_status="incomplete", + expected_json_blocked_models=("orders_snapshot",), + ), + ], + ids=lambda case: case.description, +) +def test_given_blocked_default_full_diff_when_writing_json_then_document_carries_guard_details( + test_case: DefaultFullDiffE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_diff_project(tmp_path) + build_both_environments(project_dir=project_dir) + local_config_path: Path = project_dir / "sqlbuild_local.toml" + local_config_path.write_text( + local_config_path.read_text(encoding="utf-8") + test_case.local_config_suffix, + encoding="utf-8", + ) + + result: subprocess.CompletedProcess[str] = run_sqb( + command=test_case.command, + project_dir=project_dir, + ) + + assert result.returncode == test_case.expected_exit_code, result.stdout + result.stderr + payload: dict[str, Any] = json.loads(result.stdout) + assert payload["status"] == test_case.expected_json_status + size_guard: dict[str, Any] = payload["size_guard"] + assert [model["name"] for model in size_guard["models"]] == list( + test_case.expected_json_blocked_models + ) + assert size_guard["models"][0]["to"] == { + "detail": None, + "exceeds_limit": True, + "max_full_rows": 1, + "relation": size_guard["models"][0]["to"]["relation"], + "row_count": 3, + "target": "dev", + } + assert size_guard["commands"]["full"] == ( + f"sqb --project-dir {project_dir} diff prod:dev --full --select orders_snapshot" + ) + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/integration/src/sqlbuild/executor/diff/_test_types.py b/tests/integration/src/sqlbuild/executor/diff/_test_types.py index f6edf4088e..6c8731fe6f 100644 --- a/tests/integration/src/sqlbuild/executor/diff/_test_types.py +++ b/tests/integration/src/sqlbuild/executor/diff/_test_types.py @@ -19,3 +19,27 @@ class QueryArtifactCleanupFailureTestCase: run_id: str expected_error: str expected_artifact_count: int + + +@dataclass(frozen=True) +class FullDiffSizeGuardPassTestCase: + """One guarded full diff that is allowed to read data.""" + + description: str + left_max_rows: int | None + right_max_rows: int | None + expected_metadata_lookups: int + + +@dataclass(frozen=True) +class FullDiffSizeGuardBlockTestCase: + """One guarded full diff that must stop before reading data.""" + + description: str + left_max_rows: int | None + right_max_rows: int | None + right_relation_kind: str + expected_left_row_count: int | None + expected_right_row_count: int | None + expected_left_exceeds: bool + expected_right_exceeds: bool diff --git a/tests/integration/src/sqlbuild/executor/diff/helpers.py b/tests/integration/src/sqlbuild/executor/diff/helpers.py new file mode 100644 index 0000000000..0796bf5785 --- /dev/null +++ b/tests/integration/src/sqlbuild/executor/diff/helpers.py @@ -0,0 +1,102 @@ +"""Helpers for diff executor integration tests.""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any + +import duckdb + +from sqlbuild.adapters.duckdb.classes.duckdb_adapter import DuckDbAdapter +from sqlbuild.executor.diff.models import DiffExecutionOptions, FullDiffSizeLimits + + +class RecordingDuckDbAdapter(DuckDbAdapter): + """DuckDB adapter that records every executed statement.""" + + def __init__(self) -> None: + super().__init__() + self.statements: list[str] = [] + + def execute(self, *, connection: Any, sql: str) -> Any: + self.statements.append(sql) + return super().execute(connection=connection, sql=sql) + + +@dataclass(frozen=True) +class FakeDestination: + """Compiled relation location.""" + + database: str | None + schema: str | None + name: str + qualified_name: str | None = None + + +@dataclass(frozen=True) +class FakeConfig: + """Compiled model configuration values.""" + + values: dict[str, object] = field(default_factory=dict) + + +@dataclass(frozen=True) +class FakeModel: + """Compiled model with a destination.""" + + name: str + destination: FakeDestination + config: FakeConfig + + +@dataclass(frozen=True) +class FakeProject: + """Compiled project models.""" + + models: tuple[FakeModel, ...] + + +def orders_project(schema: str) -> FakeProject: + """Return a project whose orders model lives in the given schema.""" + + return FakeProject( + models=( + FakeModel( + name="orders", + destination=FakeDestination(database=None, schema=schema, name="orders"), + config=FakeConfig(values={"unique_key": ["order_id"]}), + ), + ) + ) + + +def prod_dev_orders_connection(*, right_relation_kind: str) -> duckdb.DuckDBPyConnection: + """Return a connection with prod.orders (6 rows) and dev.orders (8 rows).""" + + connection: duckdb.DuckDBPyConnection = duckdb.connect(":memory:") + connection.execute("CREATE SCHEMA prod") + connection.execute("CREATE SCHEMA dev") + connection.execute( + "CREATE TABLE prod.orders AS SELECT range AS order_id, range * 10 AS amount FROM range(6)" + ) + connection.execute( + f"CREATE {right_relation_kind} dev.orders AS " + "SELECT range AS order_id, range * 10 AS amount FROM range(8)" + ) + return connection + + +def guarded_full_options( + *, left_max_rows: int | None, right_max_rows: int | None +) -> DiffExecutionOptions: + """Return guarded full-diff options for prod against dev.""" + + return DiffExecutionOptions( + schema_only=False, + full_size_limits=FullDiffSizeLimits( + left_target="prod", + right_target="dev", + left_max_rows=left_max_rows, + right_max_rows=right_max_rows, + ), + ) diff --git a/tests/integration/src/sqlbuild/executor/diff/test_full_size_guard.py b/tests/integration/src/sqlbuild/executor/diff/test_full_size_guard.py new file mode 100644 index 0000000000..c36e6bc986 --- /dev/null +++ b/tests/integration/src/sqlbuild/executor/diff/test_full_size_guard.py @@ -0,0 +1,151 @@ +"""Integration coverage for the metadata size guard on default full model diffs.""" + +from __future__ import annotations + +from typing import Any + +import duckdb +import pytest + +from sqlbuild.executor.diff.exceptions import FullDiffSizeGuardError +from sqlbuild.executor.diff.main.execute import execute_diff +from sqlbuild.executor.diff.models import ( + DiffExecutionResult, + FullDiffModelSize, +) +from tests.integration.src.sqlbuild.executor.diff._test_types import ( + FullDiffSizeGuardBlockTestCase, + FullDiffSizeGuardPassTestCase, +) +from tests.integration.src.sqlbuild.executor.diff.helpers import ( + RecordingDuckDbAdapter, + guarded_full_options, + orders_project, + prod_dev_orders_connection, +) + + +@pytest.mark.parametrize( + "test_case", + [ + FullDiffSizeGuardPassTestCase( + description="both sides within limits", + left_max_rows=6, + right_max_rows=100, + expected_metadata_lookups=2, + ), + FullDiffSizeGuardPassTestCase( + description="unlimited side skips its lookup", + left_max_rows=None, + right_max_rows=8, + expected_metadata_lookups=1, + ), + FullDiffSizeGuardPassTestCase( + description="both sides unlimited skip every lookup", + left_max_rows=None, + right_max_rows=None, + expected_metadata_lookups=0, + ), + ], + ids=lambda case: case.description, +) +def test_given_tables_within_limits_when_running_default_full_diff_then_compares_rows( + test_case: FullDiffSizeGuardPassTestCase, +) -> None: + adapter: RecordingDuckDbAdapter = RecordingDuckDbAdapter() + connection: duckdb.DuckDBPyConnection = prod_dev_orders_connection(right_relation_kind="TABLE") + try: + result: DiffExecutionResult = execute_diff( + adapter=adapter, + connection=connection, + left_project=orders_project("prod"), + right_project=orders_project("dev"), + selected_names=("orders",), + options=guarded_full_options( + left_max_rows=test_case.left_max_rows, + right_max_rows=test_case.right_max_rows, + ), + ) + finally: + connection.close() + + row_result: Any = result.model_results[0].row_result + assert row_result is not None + assert row_result.left_count == 6 + assert row_result.right_count == 8 + lookups: int = sum("duckdb_tables()" in sql for sql in adapter.statements) + assert lookups == test_case.expected_metadata_lookups + + +@pytest.mark.parametrize( + "test_case", + [ + FullDiffSizeGuardBlockTestCase( + description="right side over its limit", + left_max_rows=100, + right_max_rows=7, + right_relation_kind="TABLE", + expected_left_row_count=6, + expected_right_row_count=8, + expected_left_exceeds=False, + expected_right_exceeds=True, + ), + FullDiffSizeGuardBlockTestCase( + description="left side over its limit", + left_max_rows=5, + right_max_rows=100, + right_relation_kind="TABLE", + expected_left_row_count=6, + expected_right_row_count=8, + expected_left_exceeds=True, + expected_right_exceeds=False, + ), + FullDiffSizeGuardBlockTestCase( + description="view with unknown size counts as over", + left_max_rows=100, + right_max_rows=100, + right_relation_kind="VIEW", + expected_left_row_count=6, + expected_right_row_count=None, + expected_left_exceeds=False, + expected_right_exceeds=True, + ), + ], + ids=lambda case: case.description, +) +def test_given_table_over_limit_when_running_default_full_diff_then_stops_before_reading_data( + test_case: FullDiffSizeGuardBlockTestCase, +) -> None: + adapter: RecordingDuckDbAdapter = RecordingDuckDbAdapter() + connection: duckdb.DuckDBPyConnection = prod_dev_orders_connection( + right_relation_kind=test_case.right_relation_kind + ) + try: + with pytest.raises(FullDiffSizeGuardError) as error_info: + execute_diff( + adapter=adapter, + connection=connection, + left_project=orders_project("prod"), + right_project=orders_project("dev"), + selected_names=("orders",), + options=guarded_full_options( + left_max_rows=test_case.left_max_rows, + right_max_rows=test_case.right_max_rows, + ), + ) + finally: + connection.close() + + blocked: tuple[FullDiffModelSize, ...] = error_info.value.blocked + assert [model.name for model in blocked] == ["orders"] + assert blocked[0].left.row_count == test_case.expected_left_row_count + assert blocked[0].right.row_count == test_case.expected_right_row_count + assert blocked[0].left.exceeds_limit is test_case.expected_left_exceeds + assert blocked[0].right.exceeds_limit is test_case.expected_right_exceeds + assert blocked[0].has_cursor is False + assert adapter.statements + assert all("duckdb_tables()" in sql for sql in adapter.statements) + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/adapters/_test_types.py b/tests/unit/src/sqlbuild/adapters/_test_types.py index 13b0b95252..dc73592300 100644 --- a/tests/unit/src/sqlbuild/adapters/_test_types.py +++ b/tests/unit/src/sqlbuild/adapters/_test_types.py @@ -133,3 +133,34 @@ class AdapterStateTableDdlTestCase: adapter: StrictAdapter render_method: str expected_create_fragment: str + + +@dataclass(frozen=True) +class RelationRowCountEstimateTestCase: + description: str + adapter: BaseAdapter + metadata_rows: tuple[tuple[object, ...], ...] + expected_row_count: int | None + expected_detail: str | None + expected_sql_fragment: str + database: str | None = "analytics" + schema: str | None = "dev" + + +@dataclass(frozen=True) +class BigQueryRowCountEstimateTestCase: + description: str + table_type: str + num_rows: int | None + expected_row_count: int | None + expected_table_id: str = "proj.dev.orders" + + +@dataclass(frozen=True) +class DuckDbRowCountEstimateTestCase: + description: str + setup_sql: tuple[str, ...] + schema: str | None + name: str + expected_row_count: int | None + expected_detail: str | None = None diff --git a/tests/unit/src/sqlbuild/adapters/helpers.py b/tests/unit/src/sqlbuild/adapters/helpers.py index b5d256ca35..c2f811c3f7 100644 --- a/tests/unit/src/sqlbuild/adapters/helpers.py +++ b/tests/unit/src/sqlbuild/adapters/helpers.py @@ -3,6 +3,7 @@ from __future__ import annotations from collections.abc import Callable, Iterator +from dataclasses import dataclass from typing import Any @@ -73,3 +74,68 @@ def execute(*, connection: Any, sql: str) -> GrantRows: del connection return execute + + +class MetadataCursor: + """A cursor that records metadata SQL and returns fixed rows.""" + + def __init__(self, *, rows: tuple[tuple[object, ...], ...], executed: list[str]) -> None: + self._rows: tuple[tuple[object, ...], ...] = rows + self._executed: list[str] = executed + + def execute(self, sql: str, params: tuple[object, ...] | None = None) -> MetadataCursor: + self._executed.append(f"{sql} {params or ()}") + return self + + def fetchone(self) -> tuple[object, ...] | None: + return next(iter(self._rows), None) + + def fetchall(self) -> list[tuple[object, ...]]: + return list(self._rows) + + def close(self) -> None: + return None + + +class MetadataConnection: + """A connection whose cursors share one SQL log and fixed rows.""" + + def __init__(self, rows: tuple[tuple[object, ...], ...]) -> None: + self.rows: tuple[tuple[object, ...], ...] = rows + self.executed: list[str] = [] + + def cursor(self) -> MetadataCursor: + return MetadataCursor(rows=self.rows, executed=self.executed) + + +@dataclass(frozen=True) +class FakeBigQueryTable: + """BigQuery tables API metadata.""" + + table_type: str + num_rows: int | None + + +class FakeBigQueryClient: + """BigQuery client that records requested table ids.""" + + def __init__(self, table: FakeBigQueryTable) -> None: + self.table: FakeBigQueryTable = table + self.requested: list[str] = [] + + def get_table(self, table_id: str) -> FakeBigQueryTable: + self.requested.append(table_id) + return self.table + + +@dataclass(frozen=True) +class FakeBigQueryConnection: + """BigQuery connection holding a fake client.""" + + client: FakeBigQueryClient + + +def execute_through_cursor(*, connection: MetadataConnection, sql: str) -> MetadataCursor: + """Adapter execute stub that runs SQL on a fresh fake cursor.""" + + return connection.cursor().execute(sql) diff --git a/tests/unit/src/sqlbuild/adapters/test_relation_row_count_estimate.py b/tests/unit/src/sqlbuild/adapters/test_relation_row_count_estimate.py new file mode 100644 index 0000000000..a286283dff --- /dev/null +++ b/tests/unit/src/sqlbuild/adapters/test_relation_row_count_estimate.py @@ -0,0 +1,300 @@ +"""Metadata-only relation row counts used by the default full diff size guard.""" + +from __future__ import annotations + +from typing import Any, cast + +import duckdb +import pytest + +from sqlbuild.adapter.contract.models import RelationRowCountEstimate +from sqlbuild.adapters.bigquery.classes.bigquery_adapter import BigQueryAdapter +from sqlbuild.adapters.databricks.classes.databricks_adapter import DatabricksAdapter +from sqlbuild.adapters.duckdb.classes.duckdb_adapter import DuckDbAdapter +from sqlbuild.adapters.postgres.classes.postgres_adapter import PostgresAdapter +from sqlbuild.adapters.snowflake.classes.snowflake_adapter import SnowflakeAdapter +from sqlbuild.adapters.sqlserver.classes.sqlserver_adapter import SqlServerAdapter +from tests.unit.src.sqlbuild.adapters._test_types import ( + BigQueryRowCountEstimateTestCase, + DuckDbRowCountEstimateTestCase, + RelationRowCountEstimateTestCase, +) +from tests.unit.src.sqlbuild.adapters.helpers import ( + FakeBigQueryClient, + FakeBigQueryConnection, + FakeBigQueryTable, + MetadataConnection, + execute_through_cursor, +) + +_DATABRICKS_DETAIL_HEADER: tuple[tuple[object, ...], ...] = ( + ("order_id", "bigint", None), + ("", "", ""), + ("# Detailed Table Information", "", ""), +) + + +@pytest.mark.parametrize( + "test_case", + [ + RelationRowCountEstimateTestCase( + description="postgres analyzed table reports reltuples", + adapter=PostgresAdapter(), + metadata_rows=(("r", 1234.0, 8192),), + expected_row_count=1234, + expected_detail=None, + expected_sql_fragment="relation.reltuples", + ), + RelationRowCountEstimateTestCase( + description="postgres unanalyzed table with data is unknown", + adapter=PostgresAdapter(), + metadata_rows=(("r", -1.0, 8192),), + expected_row_count=None, + expected_detail="table statistics not collected; run ANALYZE", + expected_sql_fragment="pg_relation_size", + ), + RelationRowCountEstimateTestCase( + description="postgres table without pages is empty", + adapter=PostgresAdapter(), + metadata_rows=(("r", -1.0, 0),), + expected_row_count=0, + expected_detail=None, + expected_sql_fragment="namespace.nspname = 'dev'", + ), + RelationRowCountEstimateTestCase( + description="postgres view is unknown", + adapter=PostgresAdapter(), + metadata_rows=(("v", -1.0, 0),), + expected_row_count=None, + expected_detail="no table statistics (view or partitioned table)", + expected_sql_fragment="relation.relname = 'orders'", + ), + RelationRowCountEstimateTestCase( + description="postgres missing relation is unknown", + adapter=PostgresAdapter(), + metadata_rows=(), + expected_row_count=None, + expected_detail="relation not found", + expected_sql_fragment="pg_class", + ), + RelationRowCountEstimateTestCase( + description="snowflake table reports information schema row count", + adapter=SnowflakeAdapter(), + metadata_rows=((5000,),), + expected_row_count=5000, + expected_detail=None, + expected_sql_fragment="SELECT row_count FROM", + ), + RelationRowCountEstimateTestCase( + description="snowflake view has no row count", + adapter=SnowflakeAdapter(), + metadata_rows=((None,),), + expected_row_count=None, + expected_detail="no row count (view)", + expected_sql_fragment="('ORDERS', 'DEV', 'ANALYTICS')", + ), + RelationRowCountEstimateTestCase( + description="snowflake missing relation is unknown", + adapter=SnowflakeAdapter(), + metadata_rows=(), + expected_row_count=None, + expected_detail="relation not found", + expected_sql_fragment="information_schema.tables", + ), + RelationRowCountEstimateTestCase( + description="databricks analyzed table reports statistics rows", + adapter=DatabricksAdapter(), + metadata_rows=_DATABRICKS_DETAIL_HEADER + + (("Type", "MANAGED", ""), ("Statistics", "2048 bytes, 42 rows", "")), + expected_row_count=42, + expected_detail=None, + expected_sql_fragment="DESCRIBE TABLE EXTENDED `analytics`.`dev`.`orders`", + ), + RelationRowCountEstimateTestCase( + description="databricks table without statistics is unknown", + adapter=DatabricksAdapter(), + metadata_rows=_DATABRICKS_DETAIL_HEADER + (("Type", "MANAGED", ""),), + expected_row_count=None, + expected_detail="table statistics not collected; run ANALYZE TABLE ... COMPUTE STATISTICS", + expected_sql_fragment="DESCRIBE TABLE EXTENDED", + ), + RelationRowCountEstimateTestCase( + description="databricks column named statistics is not read as table statistics", + adapter=DatabricksAdapter(), + metadata_rows=(("statistics", "12 rows", None), *_DATABRICKS_DETAIL_HEADER[1:]) + + (("Type", "EXTERNAL", ""),), + expected_row_count=None, + expected_detail="table statistics not collected; run ANALYZE TABLE ... COMPUTE STATISTICS", + expected_sql_fragment="DESCRIBE TABLE EXTENDED", + ), + RelationRowCountEstimateTestCase( + description="databricks view is unknown", + adapter=DatabricksAdapter(), + metadata_rows=_DATABRICKS_DETAIL_HEADER + (("Type", "VIEW", ""),), + expected_row_count=None, + expected_detail="no row count (view)", + expected_sql_fragment="DESCRIBE TABLE EXTENDED", + ), + RelationRowCountEstimateTestCase( + description="sql server table sums partition rows", + adapter=SqlServerAdapter(), + metadata_rows=(("U ", 777),), + expected_row_count=777, + expected_detail=None, + expected_sql_fragment="FROM [analytics].sys.objects", + ), + RelationRowCountEstimateTestCase( + description="sql server view is unknown", + adapter=SqlServerAdapter(), + metadata_rows=(("V ", None),), + expected_row_count=None, + expected_detail="no row count (view)", + expected_sql_fragment="partitions.index_id IN (0, 1)", + ), + RelationRowCountEstimateTestCase( + description="sql server missing relation is unknown", + adapter=SqlServerAdapter(), + metadata_rows=(), + expected_row_count=None, + expected_detail="relation not found", + expected_sql_fragment="schemas.name = N'dev'", + ), + ], + ids=lambda case: case.description, +) +def test_given_catalog_metadata_when_estimating_row_count_then_reads_only_metadata( + test_case: RelationRowCountEstimateTestCase, monkeypatch: pytest.MonkeyPatch +) -> None: + connection: MetadataConnection = MetadataConnection(test_case.metadata_rows) + + monkeypatch.setattr(test_case.adapter, "execute", execute_through_cursor) + + estimate: RelationRowCountEstimate = test_case.adapter.estimate_relation_row_count( + connection=connection, + database=test_case.database, + schema=test_case.schema, + name="orders", + ) + + assert estimate.row_count == test_case.expected_row_count + assert estimate.detail == test_case.expected_detail + assert len(connection.executed) == 1 + assert test_case.expected_sql_fragment in connection.executed[0] + + +@pytest.mark.parametrize( + "test_case", + [ + BigQueryRowCountEstimateTestCase( + description="native table reports num_rows", + table_type="TABLE", + num_rows=9_000, + expected_row_count=9_000, + ), + BigQueryRowCountEstimateTestCase( + description="view is unknown", + table_type="VIEW", + num_rows=None, + expected_row_count=None, + ), + BigQueryRowCountEstimateTestCase( + description="external table is unknown", + table_type="EXTERNAL", + num_rows=0, + expected_row_count=None, + ), + ], + ids=lambda case: case.description, +) +def test_given_bigquery_table_metadata_when_estimating_row_count_then_uses_tables_api( + test_case: BigQueryRowCountEstimateTestCase, +) -> None: + client: FakeBigQueryClient = FakeBigQueryClient( + FakeBigQueryTable(table_type=test_case.table_type, num_rows=test_case.num_rows) + ) + connection: Any = cast(Any, FakeBigQueryConnection(client=client)) + + estimate: RelationRowCountEstimate = BigQueryAdapter().estimate_relation_row_count( + connection=connection, database="proj", schema="dev", name="orders" + ) + + assert estimate.row_count == test_case.expected_row_count + assert client.requested == [test_case.expected_table_id] + + +@pytest.mark.parametrize( + "test_case", + [ + DuckDbRowCountEstimateTestCase( + description="table reports estimated size", + setup_sql=( + "CREATE SCHEMA dev", + "CREATE TABLE dev.orders AS SELECT range AS order_id FROM range(7)", + ), + schema="dev", + name="orders", + expected_row_count=7, + ), + DuckDbRowCountEstimateTestCase( + description="name matching ignores case", + setup_sql=( + "CREATE SCHEMA dev", + "CREATE TABLE dev.orders AS SELECT range AS order_id FROM range(3)", + ), + schema="DEV", + name="Orders", + expected_row_count=3, + ), + DuckDbRowCountEstimateTestCase( + description="unqualified schema uses the current schema", + setup_sql=("CREATE TABLE orders AS SELECT range AS order_id FROM range(4)",), + schema=None, + name="orders", + expected_row_count=4, + ), + DuckDbRowCountEstimateTestCase( + description="view is unknown", + setup_sql=( + "CREATE SCHEMA dev", + "CREATE VIEW dev.orders AS SELECT range AS order_id FROM range(5)", + ), + schema="dev", + name="orders", + expected_row_count=None, + expected_detail="no table size metadata (view or missing table)", + ), + DuckDbRowCountEstimateTestCase( + description="table in another schema is not matched", + setup_sql=( + "CREATE SCHEMA prod", + "CREATE TABLE prod.orders AS SELECT range AS order_id FROM range(5)", + ), + schema="dev", + name="orders", + expected_row_count=None, + expected_detail="no table size metadata (view or missing table)", + ), + ], + ids=lambda case: case.description, +) +def test_given_duckdb_relation_when_estimating_row_count_then_reads_catalog_size( + test_case: DuckDbRowCountEstimateTestCase, +) -> None: + connection: duckdb.DuckDBPyConnection = duckdb.connect(":memory:") + try: + statement: str + for statement in test_case.setup_sql: + connection.execute(statement) + + estimate: RelationRowCountEstimate = DuckDbAdapter().estimate_relation_row_count( + connection=connection, database=None, schema=test_case.schema, name=test_case.name + ) + finally: + connection.close() + + assert estimate.row_count == test_case.expected_row_count + assert estimate.detail == test_case.expected_detail + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py index 896b7cdcd0..51d0a04e06 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py @@ -1,7 +1,9 @@ from dataclasses import dataclass +from pathlib import Path +from typing import Literal from sqlbuild.cli.commands.models import DiffExampleRenderOptions -from sqlbuild.executor.diff.models import DiffExecutionResult +from sqlbuild.executor.diff.models import DiffExecutionResult, FullDiffSizeLimits @dataclass(frozen=True) @@ -28,3 +30,41 @@ class RenderDiffEvidenceTestCase: expected_first_difference: int | None expected_truncated: bool expected_suppressed: bool + + +@dataclass(frozen=True) +class FullDiffSizeLimitsTestCase: + description: str + full: bool + schema_only: bool + bounded: str | None + prod_max_full_rows: int | Literal["unlimited"] | None + dev_max_full_rows: int | Literal["unlimited"] | None + expected_limits: FullDiffSizeLimits | None + + +@dataclass(frozen=True) +class DiffSizeGuardErrorTestCase: + description: str + select: tuple[str, ...] + unique_key_override: tuple[str, ...] + has_cursor: bool + expected_full_command: str + expected_bounded_command: str | None + expected_message_fragments: tuple[str, ...] + expected_absent_fragments: tuple[str, ...] = () + project_dir: Path | None = None + + +@dataclass(frozen=True) +class ParseDiffNameRangeTestCase: + description: str + name_range: str + expected_result: tuple[str, str | None] + + +@dataclass(frozen=True) +class ParseDiffNameRangeErrorTestCase: + description: str + name_range: str | None + expected_code: str diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/helpers.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/helpers.py new file mode 100644 index 0000000000..5767145518 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/helpers.py @@ -0,0 +1,86 @@ +"""Shared builders for diff CLI helper unit tests.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Literal + +from sqlbuild.cli.commands.models import DiffCommandRequest +from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs +from sqlbuild.executor.diff.models import FullDiffModelSize, FullDiffSideSize +from sqlbuild.spec.contracts.models import ( + LocalConfig, + ProjectConfig, + TargetConfig, + TargetDiffConfig, +) + + +def diff_request( + *, + full: bool = False, + schema_only: bool = False, + bounded: str | None = None, + select: tuple[str, ...] = ("orders",), + unique_key_override: tuple[str, ...] = (), + project_dir: Path | None = None, +) -> DiffCommandRequest: + """Return a model diff request from prod with the TO target left to the default.""" + + return DiffCommandRequest( + project_dir=project_dir, + no_color=True, + no_sql_validation=False, + from_name="prod", + to_name=None, + full=full, + schema_only=schema_only, + bounded=bounded, + select=select, + unique_key_override=unique_key_override, + ) + + +def prod_dev_inputs( + *, + prod_max_full_rows: int | Literal["unlimited"] | None, + dev_max_full_rows: int | Literal["unlimited"] | None, +) -> DiscoveredProjectInputs: + """Return discovered inputs with prod and dev targets carrying the given diff limits.""" + + return DiscoveredProjectInputs( + project_config=ProjectConfig( + name="shop", + adapter="duckdb", + targets={ + "prod": TargetConfig( + schema="prod", diff=TargetDiffConfig(max_full_rows=prod_max_full_rows) + ), + "dev": TargetConfig( + schema="dev", diff=TargetDiffConfig(max_full_rows=dev_max_full_rows) + ), + }, + ), + local_config=LocalConfig(), + ) + + +def blocked_orders(*, has_cursor: bool) -> tuple[FullDiffModelSize, ...]: + """Return one blocked model: prod over its limit and dev a view of unknown size.""" + + return ( + FullDiffModelSize( + name="orders", + left=FullDiffSideSize( + target="prod", relation="prod.orders", row_count=12_500_000, max_rows=10_000_000 + ), + right=FullDiffSideSize( + target="dev", + relation="dev.orders", + row_count=None, + max_rows=10_000_000, + detail="no row count (view)", + ), + has_cursor=has_cursor, + ), + ) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_size_guard.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_size_guard.py new file mode 100644 index 0000000000..fb4604316d --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_size_guard.py @@ -0,0 +1,186 @@ +"""Default full diff limits and size guard error rendering.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any, cast + +import pytest + +from sqlbuild.cli.commands._helpers.diff.size_guard import ( + build_diff_size_guard_error, + resolve_full_diff_size_limits, +) +from sqlbuild.cli.commands.exceptions import DiffSizeGuardError +from sqlbuild.executor.diff.exceptions import FullDiffSizeGuardError +from sqlbuild.executor.diff.models import FullDiffSizeLimits +from tests.unit.src.sqlbuild.cli.commands._helpers.diff._test_types import ( + DiffSizeGuardErrorTestCase, + FullDiffSizeLimitsTestCase, +) +from tests.unit.src.sqlbuild.cli.commands._helpers.diff.helpers import ( + blocked_orders, + diff_request, + prod_dev_inputs, +) + + +@pytest.mark.parametrize( + "test_case", + [ + FullDiffSizeLimitsTestCase( + description="unset limits use the built-in default", + full=False, + schema_only=False, + bounded=None, + prod_max_full_rows=None, + dev_max_full_rows=None, + expected_limits=FullDiffSizeLimits( + left_target="prod", + right_target="dev", + left_max_rows=10_000_000, + right_max_rows=10_000_000, + ), + ), + FullDiffSizeLimitsTestCase( + description="each side uses its own target limit", + full=False, + schema_only=False, + bounded=None, + prod_max_full_rows=500, + dev_max_full_rows="unlimited", + expected_limits=FullDiffSizeLimits( + left_target="prod", right_target="dev", left_max_rows=500, right_max_rows=None + ), + ), + FullDiffSizeLimitsTestCase( + description="explicit full bypasses the guard", + full=True, + schema_only=False, + bounded=None, + prod_max_full_rows=1, + dev_max_full_rows=1, + expected_limits=None, + ), + FullDiffSizeLimitsTestCase( + description="explicit schema only bypasses the guard", + full=False, + schema_only=True, + bounded=None, + prod_max_full_rows=1, + dev_max_full_rows=1, + expected_limits=None, + ), + FullDiffSizeLimitsTestCase( + description="explicit bounded bypasses the guard", + full=False, + schema_only=False, + bounded="7d", + prod_max_full_rows=1, + dev_max_full_rows=1, + expected_limits=None, + ), + ], + ids=lambda case: case.description, +) +def test_given_mode_flags_and_target_limits_when_resolving_then_guard_limits_match( + test_case: FullDiffSizeLimitsTestCase, +) -> None: + limits: FullDiffSizeLimits | None = resolve_full_diff_size_limits( + request=diff_request( + full=test_case.full, schema_only=test_case.schema_only, bounded=test_case.bounded + ), + discovered_inputs=prod_dev_inputs( + prod_max_full_rows=test_case.prod_max_full_rows, + dev_max_full_rows=test_case.dev_max_full_rows, + ), + from_target="prod", + to_target="dev", + ) + + assert limits == test_case.expected_limits + + +@pytest.mark.parametrize( + "test_case", + [ + DiffSizeGuardErrorTestCase( + description="model with cursor offers bounded command", + select=("orders",), + unique_key_override=(), + has_cursor=True, + expected_full_command="sqb diff prod:dev --full --select orders", + expected_bounded_command="sqb diff prod:dev --bounded --select orders", + expected_message_fragments=( + "orders: prod 12,500,000 rows, limit 10,000,000; " + "dev size unknown: no row count (view), limit 10,000,000", + "Run one of these instead:", + "sqb diff prod:dev --schema-only --select orders", + ), + ), + DiffSizeGuardErrorTestCase( + description="model without cursor omits bounded command and keeps key flags", + select=("tag:daily", "path:models/marts"), + unique_key_override=("order_id", "line number"), + has_cursor=False, + expected_full_command=( + "sqb diff prod:dev --full --select tag:daily path:models/marts " + "--key order_id 'line number'" + ), + expected_bounded_command=None, + expected_message_fragments=("; no cursor",), + expected_absent_fragments=("--bounded",), + ), + DiffSizeGuardErrorTestCase( + description="project dir is a global flag placed before diff", + select=("orders",), + unique_key_override=(), + has_cursor=False, + expected_full_command=( + "sqb --project-dir projects/shop diff prod:dev --full --select orders" + ), + expected_bounded_command=None, + expected_message_fragments=(), + project_dir=Path("projects/shop"), + ), + ], + ids=lambda case: case.description, +) +def test_given_blocked_models_when_building_guard_error_then_names_sizes_and_commands( + test_case: DiffSizeGuardErrorTestCase, +) -> None: + error: DiffSizeGuardError = build_diff_size_guard_error( + request=diff_request( + select=test_case.select, + unique_key_override=test_case.unique_key_override, + project_dir=test_case.project_dir, + ), + from_target="prod", + to_target="dev", + error=FullDiffSizeGuardError( + "full diff stopped", blocked=blocked_orders(has_cursor=test_case.has_cursor) + ), + ) + + assert error.code == "C270" + assert error.exit_code == 2 + assert error.status == "incomplete" + assert test_case.expected_full_command in error.message + fragment: str + for fragment in test_case.expected_message_fragments: + assert fragment in error.message + for fragment in test_case.expected_absent_fragments: + assert fragment not in error.message + assert fragment not in str(error.help) + commands: dict[str, Any] = cast(dict[str, Any], error.details["commands"]) + assert commands["full"] == test_case.expected_full_command + assert commands["bounded"] == test_case.expected_bounded_command + models: list[dict[str, Any]] = cast(list[dict[str, Any]], error.details["models"]) + assert models[0]["from"]["row_count"] == 12_500_000 + assert models[0]["to"]["row_count"] is None + assert models[0]["to"]["exceeds_limit"] is True + assert "targets.prod.diff.max_full_rows and targets.dev.diff.max_full_rows" in str(error.help) + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_validation.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_validation.py new file mode 100644 index 0000000000..bc5290d355 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_validation.py @@ -0,0 +1,68 @@ +"""Diff target range parsing.""" + +from __future__ import annotations + +import pytest + +from sqlbuild.cli.commands._helpers.diff.validation import parse_diff_name_range +from sqlbuild.cli.commands.exceptions import CliUserError +from tests.unit.src.sqlbuild.cli.commands._helpers.diff._test_types import ( + ParseDiffNameRangeErrorTestCase, + ParseDiffNameRangeTestCase, +) + + +@pytest.mark.parametrize( + "test_case", + [ + ParseDiffNameRangeTestCase( + description="explicit range keeps both targets", + name_range="prod:dev", + expected_result=("prod", "dev"), + ), + ParseDiffNameRangeTestCase( + description="bare target leaves TO for the active target", + name_range="prod", + expected_result=("prod", None), + ), + ], + ids=lambda case: case.description, +) +def test_given_target_range_when_parsing_then_returns_from_and_optional_to( + test_case: ParseDiffNameRangeTestCase, +) -> None: + assert parse_diff_name_range(test_case.name_range) == test_case.expected_result + + +@pytest.mark.parametrize( + "test_case", + [ + ParseDiffNameRangeErrorTestCase( + description="missing range", name_range=None, expected_code="C208" + ), + ParseDiffNameRangeErrorTestCase( + description="empty target", name_range="", expected_code="C209" + ), + ParseDiffNameRangeErrorTestCase( + description="empty TO", name_range="prod:", expected_code="C209" + ), + ParseDiffNameRangeErrorTestCase( + description="empty FROM", name_range=":dev", expected_code="C209" + ), + ParseDiffNameRangeErrorTestCase( + description="too many separators", name_range="a:b:c", expected_code="C209" + ), + ], + ids=lambda case: case.description, +) +def test_given_malformed_target_range_when_parsing_then_raises_coded_error( + test_case: ParseDiffNameRangeErrorTestCase, +) -> None: + with pytest.raises(CliUserError) as error_info: + parse_diff_name_range(test_case.name_range) + + assert error_info.value.code == test_case.expected_code + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/tests/unit/src/sqlbuild/compiler/discovery/_helpers/_test_types.py b/tests/unit/src/sqlbuild/compiler/discovery/_helpers/_test_types.py index 5929a72e83..d23f59b5f9 100644 --- a/tests/unit/src/sqlbuild/compiler/discovery/_helpers/_test_types.py +++ b/tests/unit/src/sqlbuild/compiler/discovery/_helpers/_test_types.py @@ -1005,3 +1005,19 @@ class TargetWarehousesDiscoveryTestCase: expected_build: str | None local_contents: str = "" adapter_file_contents: str = "" + + +@dataclass(frozen=True) +class TargetDiffConfigTestCase: + description: str + project_contents: str + expected_max_full_rows: int | str | None + local_contents: str = "" + + +@dataclass(frozen=True) +class TargetDiffConfigErrorTestCase: + description: str + project_contents: str + expected_error_fragment: str + local_contents: str = "" diff --git a/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_target_diff.py b/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_target_diff.py new file mode 100644 index 0000000000..b0c6b96a8c --- /dev/null +++ b/tests/unit/src/sqlbuild/compiler/discovery/_helpers/test_target_diff.py @@ -0,0 +1,133 @@ +"""Loading, validation and local precedence of per-target diff limits.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from sqlbuild.compiler.discovery._helpers.yml.project import ( + load_local_config, + load_project_config, +) +from sqlbuild.compiler.discovery.exceptions import ProjectConfigError +from sqlbuild.spec.contracts.main.resolve_target_config import resolve_target_config +from tests.unit.src.sqlbuild.compiler.discovery._helpers._test_types import ( + TargetDiffConfigErrorTestCase, + TargetDiffConfigTestCase, +) + +_PROJECT: str = 'name = "shop"\nadapter = "duckdb"\n\n[targets.dev]\nschema = "dev"\n' + + +@pytest.mark.parametrize( + "test_case", + [ + TargetDiffConfigTestCase( + description="target without a diff section leaves the limit unset", + project_contents=_PROJECT, + expected_max_full_rows=None, + ), + TargetDiffConfigTestCase( + description="positive integer limit is kept", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = 25_000\n", + expected_max_full_rows=25_000, + ), + TargetDiffConfigTestCase( + description="unlimited keyword disables the limit", + project_contents=_PROJECT + '\n[targets.dev.diff]\nmax_full_rows = "unlimited"\n', + expected_max_full_rows="unlimited", + ), + TargetDiffConfigTestCase( + description="local limit replaces the project limit", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = 25_000\n", + local_contents="[targets.dev.diff]\nmax_full_rows = 90_000\n", + expected_max_full_rows=90_000, + ), + TargetDiffConfigTestCase( + description="local unlimited replaces a project integer limit", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = 25_000\n", + local_contents='[targets.dev.diff]\nmax_full_rows = "unlimited"\n', + expected_max_full_rows="unlimited", + ), + TargetDiffConfigTestCase( + description="local target without a diff section inherits the project limit", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = 25_000\n", + local_contents='[targets.dev]\nschema = "dev_alice"\n', + expected_max_full_rows=25_000, + ), + ], + ids=lambda case: case.description, +) +def test_given_target_diff_section_when_loading_then_effective_limit_follows_local_precedence( + test_case: TargetDiffConfigTestCase, tmp_path: Path +) -> None: + (tmp_path / "sqlbuild_project.toml").write_text(test_case.project_contents, encoding="utf-8") + (tmp_path / "sqlbuild_local.toml").write_text(test_case.local_contents, encoding="utf-8") + + max_full_rows: int | str | None = resolve_target_config( + project_config=load_project_config(project_dir=tmp_path), + local_config=load_local_config(project_dir=tmp_path), + target_name="dev", + ).diff.max_full_rows + + assert max_full_rows == test_case.expected_max_full_rows + + +@pytest.mark.parametrize( + "test_case", + [ + TargetDiffConfigErrorTestCase( + description="zero is rejected", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = 0\n", + expected_error_fragment="targets.dev.diff.max_full_rows must be an integer >= 1", + ), + TargetDiffConfigErrorTestCase( + description="negative integer is rejected", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = -5\n", + expected_error_fragment="targets.dev.diff.max_full_rows must be an integer >= 1", + ), + TargetDiffConfigErrorTestCase( + description="boolean is rejected", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = true\n", + expected_error_fragment="targets.dev.diff.max_full_rows must be an integer >= 1", + ), + TargetDiffConfigErrorTestCase( + description="other strings are rejected", + project_contents=_PROJECT + '\n[targets.dev.diff]\nmax_full_rows = "none"\n', + expected_error_fragment="or 'unlimited'", + ), + TargetDiffConfigErrorTestCase( + description="float is rejected", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_full_rows = 1.5\n", + expected_error_fragment="targets.dev.diff.max_full_rows must be an integer >= 1", + ), + TargetDiffConfigErrorTestCase( + description="unknown diff key is rejected", + project_contents=_PROJECT + "\n[targets.dev.diff]\nmax_rows = 10\n", + expected_error_fragment="targets.dev.diff has unknown keys: max_rows", + ), + TargetDiffConfigErrorTestCase( + description="invalid local limit is rejected", + project_contents=_PROJECT, + local_contents="[targets.dev.diff]\nmax_full_rows = 0\n", + expected_error_fragment="targets.dev.diff.max_full_rows must be an integer >= 1", + ), + ], + ids=lambda case: case.description, +) +def test_given_invalid_target_diff_section_when_loading_then_config_error_names_key( + test_case: TargetDiffConfigErrorTestCase, tmp_path: Path +) -> None: + (tmp_path / "sqlbuild_project.toml").write_text(test_case.project_contents, encoding="utf-8") + (tmp_path / "sqlbuild_local.toml").write_text(test_case.local_contents, encoding="utf-8") + + with pytest.raises(ProjectConfigError) as error_info: + load_project_config(project_dir=tmp_path) + load_local_config(project_dir=tmp_path) + + assert test_case.expected_error_fragment in str(error_info.value) + + +if __name__ == "__main__": + pytest.main([__file__, "-vv"]) diff --git a/website/src/content/docs/docs/cli/diff.mdx b/website/src/content/docs/docs/cli/diff.mdx index 4c2768d785..12960b32e6 100644 --- a/website/src/content/docs/docs/cli/diff.mdx +++ b/website/src/content/docs/docs/cli/diff.mdx @@ -9,14 +9,16 @@ Compares schemas and optionally row-level data between two targets (e.g. `prod:d ```bash # Model mode -sqb diff : --select [flags] +sqb diff [:] [] --select [flags] # Query mode -sqb diff --left-query --right-query (--key | --unkeyed) [flags] +sqb diff --left-query --right-query (--key ... | --unkeyed) [flags] sqb diff --left-query --right-query --schema-only [flags] ``` -In model mode, the first argument is a positional `FROM:TO` range, `--select` is required, and exactly one mode is required: `--full`, `--schema-only`, or `--bounded `. +In model mode, the first argument is a positional `FROM:TO` range, `--select` is required, and at most one mode can be given: `--full`, `--schema-only`, or `--bounded `. Without a mode, the diff is a full comparison guarded by table size (see [Size guard](#size-guard)). + +`FROM` alone compares `FROM` with the active target, so with `dev` active, `sqb diff prod -s orders` runs as `sqb diff prod:dev -s orders`. This is the same default `sqb clone --from` uses. It fails when no target is active or `FROM` is the active target; pass `FROM:TO` in those cases. `FROM` and `TO` are configured target names. Their database/schema namespaces remain authoritative, while the `TO` target's named connection executes the complete comparison and must @@ -43,9 +45,9 @@ In query mode, omit `FROM:TO` and model selectors; each query identifies its own | `--max-models` | Fail when the selected scope contains more models than this limit | | `--max-columns` | Fail when either side of a model has more columns than this limit | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | -| `--key` | Row key column; repeat for a composite key. Overrides the model's `unique_key` | +| `--key` | Row key columns, for example `--key order_id line_number`. Repeating the flag also works. Overrides the model's `unique_key` | | `--unkeyed` | Compare exact full-row multisets when there is no stable row key | -| `--exclude-column` | Leave a column out of the comparison; repeatable | +| `--exclude-column` | Leave columns out of the comparison; takes several values, and repeating the flag also works | | `--tolerance` | Numeric tolerance such as `amount:absolute=1` or `rate:relative=0.001`; repeatable | | `--left-query`, `--right-query` | Compare two read-only SQL queries instead of models | | `--left-query-file`, `--right-query-file` | Read each query from a UTF-8 file | @@ -59,9 +61,53 @@ In query mode, omit `FROM:TO` and model selectors; each query identifies its own | `--exclude` | Exclude specific models from diffing | | `--warehouse ` | Snowflake warehouse for this invocation; overrides the target's `query` [warehouse group](/docs/concepts/project-configuration#command-group-warehouses) and the connection warehouse | +## Size guard + +Without `--full`, `--bounded` or `--schema-only`, model mode first checks table sizes. Before it reads +any table data, SQLBuild reads each selected table's row count from warehouse metadata on both +sides and compares it with that target's +[`max_full_rows`](/docs/concepts/project-configuration#diff-limits) limit. The limit is 10,000,000 +rows unless configured, and `"unlimited"` turns the check off for that target. If every table is +within its limits, the full comparison runs as usual. + +Otherwise, the diff stops before reading any rows and exits with `2`. The error lists each blocked model +with its row count and limit on both sides, then gives the commands to run instead: + +```text +error[C270]: full diff stopped before reading data: 1 of 1 selected models exceed the full-comparison row limit or have an unknown size + orders: prod 12,400,000 rows, limit 10,000,000; dev 12,400,000 rows, limit 10,000,000 +Run one of these instead: + sqb diff prod:dev --full --select orders + sqb diff prod:dev --bounded --select orders + sqb diff prod:dev --schema-only --select orders +``` + +The `--bounded` command appears only when every blocked model has a cursor. With `--json` or +`--json-output`, the error document has `status` `incomplete` and a `size_guard` object holding the +same models, sizes, limits and commands. + +A size counts as over the limit when the warehouse cannot report it, for example for views or for +tables without statistics. Sizes come from metadata only: + +| Adapter | Source | +|---------|--------| +| DuckDB, MotherDuck | `duckdb_tables()` `estimated_size` | +| PostgreSQL | `pg_class.reltuples`; unknown until the table has been analyzed | +| Snowflake | `INFORMATION_SCHEMA.TABLES` `ROW_COUNT` | +| BigQuery | Tables API `numRows` | +| Databricks | Statistics from `DESCRIBE TABLE EXTENDED`; unknown until `ANALYZE TABLE ... COMPUTE STATISTICS` has run | +| SQL Server | `sys.partitions` row counts | +| Custom adapters | Unknown unless the adapter implements `estimate_relation_row_count` | + +Metadata counts can be estimates. The guard only decides whether to start a full comparison; the +comparison itself always counts rows exactly. + ## Examples ```bash +# Full diff against the active target, if the tables are within the size limit +sqb diff prod --select customer_status_snapshot + # Full diff of a specific model sqb diff prod:dev --full --select customer_status_snapshot @@ -79,8 +125,11 @@ sqb diff prod:dev --bounded 14d --exhaustive --select order_lines # Compare two queries on a key sqb diff --left-query-file queries/orders_before.sql --right-query-file queries/orders_after.sql --key order_id + +# Composite key and several excluded columns +sqb diff prod:dev --full --select order_lines --key order_id line_number --exclude-column updated_at loaded_at ``` ## Exit codes -Returns `0` when all selected models have no differences, `1` when any model has schema or row differences. Query mode also returns `2` when a safety, schema, or key check prevented a complete comparison, and `3` when setup, execution, or cleanup failed; see [Query output and exit codes](/docs/concepts/diff#query-output-and-exit-codes). +Returns `0` when all selected models have no differences, `1` when any model has schema or row differences, and `2` when the size guard stopped a default full diff before reading data. Query mode also returns `2` when a safety, schema, or key check prevented a complete comparison, and `3` when setup, execution, or cleanup failed; see [Query output and exit codes](/docs/concepts/diff#query-output-and-exit-codes). diff --git a/website/src/content/docs/docs/concepts/diff.mdx b/website/src/content/docs/docs/concepts/diff.mdx index 6805e5e852..71231de94b 100644 --- a/website/src/content/docs/docs/concepts/diff.mdx +++ b/website/src/content/docs/docs/concepts/diff.mdx @@ -7,7 +7,8 @@ import Frame from '../../../../components/mintlify/Frame.astro'; SQLBuild can compare schemas and row-level data between two targets. This lets you validate that changes produce the expected results before they reach production. -`sqb diff FROM:TO` compares two targets, for example `prod:dev`. +`sqb diff FROM:TO` compares two targets, for example `prod:dev`. `sqb diff FROM` compares `FROM` +with the active target, the same default `sqb clone --from` uses. `FROM` and `TO` resolve the authoritative database/schema namespaces to compare. SQLBuild uses the `TO` target's named connection for the complete comparison and accesses both @@ -25,7 +26,7 @@ sqb diff prod:dev --full --select customer_status_snapshot ## Comparison modes -Every diff requires exactly one mode: +A diff takes at most one mode flag. Without one, it runs a full diff guarded by table size. ### Full diff @@ -35,6 +36,13 @@ Compares both schema and row-level data for the selected models: sqb diff prod:dev --full --select fact_orders ``` +Leaving out the mode flag also runs a full diff, but only after a size check. Before reading any +table data, SQLBuild reads each selected table's row count from warehouse metadata on both sides. +If a table is larger than that side's +[`max_full_rows`](/docs/concepts/project-configuration#diff-limits) limit (10,000,000 rows unless +configured), or its size is unknown, the diff stops without scanning. See +[size guard](/docs/cli/diff#size-guard) for details. `--full` skips the check. + Rows are joined on the model's `unique_key` and compared column by column. The output shows: - Row counts for each side - How many rows are equal, unequal, or only in one side @@ -375,4 +383,4 @@ reconciliation, inspection, materialization, comparison, cleanup, and total dura ## Exit codes -In model mode, `sqb diff` returns exit code `0` when all selected models have no differences, and `1` when any model has schema or row differences. Raw-query mode uses the exit codes in [Query output and exit codes](#query-output-and-exit-codes). This makes it usable in CI pipelines as a validation gate. +In model mode, `sqb diff` returns exit code `0` when all selected models have no differences, `1` when any model has schema or row differences, and `2` when the size guard stopped a default full diff before reading data. Raw-query mode uses the exit codes in [Query output and exit codes](#query-output-and-exit-codes). This makes it usable in CI pipelines as a validation gate. diff --git a/website/src/content/docs/docs/concepts/project-configuration.mdx b/website/src/content/docs/docs/concepts/project-configuration.mdx index c0fedb7f13..0d65602585 100644 --- a/website/src/content/docs/docs/concepts/project-configuration.mdx +++ b/website/src/content/docs/docs/concepts/project-configuration.mdx @@ -378,6 +378,33 @@ This permits an intentional developer override while keeping the shared target p `sqlbuild_project.toml`. Other target fields and any execution-limit fields omitted locally continue to inherit their project values. +### Diff limits + +`sqb diff` without `--full`, `--bounded` or `--schema-only` runs a full comparison only when every +selected table is small enough. Each target sets the largest table it will compare in full this +way: + +```toml +[targets.prod.diff] +max_full_rows = 50_000_000 + +[targets.dev.diff] +max_full_rows = "unlimited" +``` + +`max_full_rows` is a positive integer or `"unlimited"`. When it is not set, the limit is +10,000,000 rows. Each side of a comparison uses its own target's limit. See +[`sqb diff`](/docs/cli/diff#size-guard) for how sizes are read and what happens when a table is too +large. + +`sqlbuild_local.toml` accepts the same section, and a local value replaces the project value for +that target: + +```toml +[targets.dev.diff] +max_full_rows = 100_000_000 +``` + ## Defaults Project-wide defaults for models, seeds, and functions: