diff --git a/docs/query-diffs.md b/docs/query-diffs.md index b2baeab5cb..2bce1b210a 100644 --- a/docs/query-diffs.md +++ b/docs/query-diffs.md @@ -37,6 +37,8 @@ from a UTF-8 file, and choose keyed or unkeyed comparison explicitly: sqb diff \ --left-query 'SELECT order_id, amount_cents FROM analytics.orders_before' \ --right-query 'SELECT order_id, amount_cents FROM analytics.orders_after' \ + --left-label before \ + --right-label after \ --key order_id ``` @@ -56,6 +58,9 @@ Query mode performs a full comparison when no comparison mode flag is present. ` also available. `--bounded` is model-specific; filter both raw queries directly when a bounded query comparison is needed. +Raw-query discovery loads only project configuration, target configuration, and the selected +adapter. It does not import unrelated project sinks, hooks, loaders, or providers. + ### Sampling, tolerances, and exclusions Keyed query comparisons support deterministic union-key sampling and the same bounded mismatch @@ -85,6 +90,31 @@ sqb diff \ `--exhaustive` disables inherited or requested sampling. Unkeyed comparison is always exhaustive and exact, so it does not accept sampling or numeric tolerances. +### Bounded example values + +Comparison remains exact regardless of how examples are rendered. By default, long values are +limited to 160 characters and show context around the first differing character. Every shortened +value is marked as truncated and includes its original length. + +Control evidence independently from the number of collected examples: + +```bash +# Increase the displayed context without changing comparison scope. +sqb diff --left-query-file before.sql --right-query-file after.sql \ + --key order_id --max-value-length 500 + +# Retain keys and aggregate counts but suppress example values. +sqb diff --left-query-file before.sql --right-query-file after.sql \ + --key order_id --no-example-values + +# Explicitly opt in to complete values. --verbose alone remains bounded. +sqb diff --left-query-file before.sql --right-query-file after.sql \ + --key order_id --full-example-values +``` + +Ordinary keys are kept complete. Exceptionally long keys are shortened with an explicit length and +stable digest so an operator can still identify them without flooding terminal output. + ## Stable query results and cleanup SQLBuild materializes each raw query once in the selected target's default schema before comparing @@ -115,6 +145,20 @@ error. ## Structured output and exit behavior -Use `--json-output PATH` in either mode. Query results include `input_kind = "query"`; model results -use `input_kind = "model"`. Exit code `0` means no differences were found. Exit code `1` means -differences were found or the comparison could not execute successfully. +Use `--json-output PATH` in either mode, or `--json` to emit one JSON document to stdout while all +lifecycle progress remains on stderr. Query results include `input_kind = "query"`, named left and +right inputs, `query_comparisons`, sparse nonzero `column_mismatches`, lifecycle timings, and +explicit example truncation metadata. Existing `models` output remains available for compatibility. + +Raw-query automation distinguishes complete findings from incomplete evidence and execution +failure: + +| Exit | Outcome | +|---:|---| +| `0` | Complete comparison with no differences | +| `1` | Complete comparison with findings | +| `2` | Safety, schema, or key preflight prevented complete value evidence | +| `3` | Setup, connection, materialization, comparison execution, ownership, or cleanup failed | + +Human output and JSON use the same outcome. Lifecycle messages include elapsed connection, +reconciliation, inspection, materialization, comparison, cleanup, and total durations. diff --git a/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md b/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md index 3eabe15f72..35b96a6f79 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/SKILL.md @@ -7780,6 +7780,8 @@ SQLBuild templates such as `ref()` are not expanded. sqb diff \ --left-query 'SELECT order_id, amount_cents FROM analytics.orders_before' \ --right-query 'SELECT order_id, amount_cents FROM analytics.orders_after' \ + --left-label before \ + --right-label after \ --key order_id sqb diff \ @@ -7793,6 +7795,9 @@ sqb diff \ Query mode defaults to a full comparison and also supports `--schema-only`. It does not accept `--bounded`; put the desired filter in both SQL inputs instead. +Raw-query mode loads configuration and the selected adapter without importing unrelated project +sinks, hooks, loaders, or providers. + ### Comparison modes Model mode requires exactly one mode. Raw-query mode defaults to full when no mode is supplied: @@ -7979,19 +7984,47 @@ sqb diff prod:dev --full --select customer_status_snapshot --max-column-examples Example limits only control diagnostic values printed after comparison. They are separate from `row_diff_sample_rows`, which controls how many unique keys receive the wide value comparison. +Long values default to 160 rendered characters. SQLBuild shows context around the first difference, +marks truncation, and reports the original length without changing exact comparison counts. Control +value evidence independently from example counts: + +```bash +sqb diff \ + --left-query-file queries/orders_before.sql \ + --right-query-file queries/orders_after.sql \ + --key order_id \ + --max-value-length 500 + +sqb diff \ + --left-query-file queries/orders_before.sql \ + --right-query-file queries/orders_after.sql \ + --key order_id \ + --no-example-values + +sqb diff \ + --left-query-file queries/orders_before.sql \ + --right-query-file queries/orders_after.sql \ + --key order_id \ + --full-example-values +``` + +`--verbose` increases the example count but does not remove the value-length bound. + ### Structured output Use `--json-output PATH` to write a stable structured result while retaining the normal terminal -summary: +summary, or `--json` to write one JSON document to stdout with progress on stderr: ```bash sqb diff prod:dev --bounded 14d --sample-rows 50000 --select order_lines --json-output diff.json +sqb diff --left-query-file before.sql --right-query-file after.sql --key order_id --json ``` Each model records `schema_only`, `exhaustive`, or `sampled` comparison scope; requested and observed cursor coverage; bounded population and compared key counts; seed and configured limit; row result -counts; and changed-column counts. The top-level status distinguishes `no_differences_found` from -`differences_found`. +counts; and changed-column counts. Query output additionally includes named inputs, +`query_comparisons`, sparse nonzero `column_mismatches`, lifecycle timings, and explicit evidence +truncation metadata. Existing `models` output remains for compatibility. ### Invocation safety limits @@ -8022,9 +8055,10 @@ sqb diff prod:dev --full --select tag:acceptance ### Exit codes -`sqb diff` returns exit code `0` when no differences are found, and `1` when any input has schema or -row differences or the comparison cannot execute successfully. This makes it usable in CI pipelines -as a validation gate. +Complete comparisons return `0` when no differences are found and `1` for findings. Raw-query mode +returns `2` when safety, schema, or key preflight prevents complete value evidence, and `3` for +setup, connection, materialization, comparison execution, ownership publication, or cleanup +failure. Human and JSON output carry the same outcome. ## Execution Observability @@ -14541,17 +14575,23 @@ and falls back to a full row comparison when no cursor is configured. | `--sample-seed` | Override the deterministic sample seed | | `--exhaustive` | Disable inherited sampling for this invocation | | `--json-output` | Write structured comparison scope, coverage, and results to a JSON file | +| `--json` | Write one structured JSON document to stdout; progress remains on stderr | | `--max-models` | Fail when the selected scope contains more models than this limit | | `--max-columns` | Fail when either side of a model has more columns than this limit | | `--left-query` | Literal read-only SQL for the left side | | `--left-query-file` | UTF-8 SQL file for the left side | | `--right-query` | Literal read-only SQL for the right side | | `--right-query-file` | UTF-8 SQL file for the right side | +| `--left-label` | Display and structured-output label for the left query | +| `--right-label` | Display and structured-output label for the right query | | `--target` | Target connection for raw-query mode; defaults to the active target | | `--key` | Row identity column; repeat for a composite key or to override model configuration | | `--unkeyed` | Compare exact full-row multiplicities without a key | | `--exclude-column` | Column to omit from row comparison; repeat as needed | | `--tolerance` | Per-column numeric rule such as `amount:absolute=0.01`; repeat as needed | +| `--max-value-length` | Maximum rendered characters per example value (default: 160) | +| `--no-example-values` | Retain example keys and counts while suppressing values | +| `--full-example-values` | Explicitly opt in to complete example values | | `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) | | `--select`, `-s` | Select specific models to diff (required in v1) | | `--exclude` | Exclude specific models from diffing | @@ -14578,6 +14618,8 @@ sqb diff prod:dev --bounded 14d --exhaustive --select order_lines sqb diff \ --left-query-file queries/orders_before.sql \ --right-query-file queries/orders_after.sql \ + --left-label before \ + --right-label after \ --key account_id \ --key order_id ``` diff --git a/src/sqlbuild/cli/commands/_helpers/diff/evidence.py b/src/sqlbuild/cli/commands/_helpers/diff/evidence.py new file mode 100644 index 0000000000..0ae5378b32 --- /dev/null +++ b/src/sqlbuild/cli/commands/_helpers/diff/evidence.py @@ -0,0 +1,108 @@ +"""Bounded, diff-aware rendering for row-diff evidence.""" + +from __future__ import annotations + +import hashlib + +from sqlbuild.cli.commands.models import DiffExampleRenderOptions, RenderedDiffExampleValue + +_MAX_RENDERED_KEY_LENGTH: int = 256 + + +def render_example_pair( + *, + left_value: object, + right_value: object, + options: DiffExampleRenderOptions, +) -> tuple[RenderedDiffExampleValue, RenderedDiffExampleValue]: + """Render two unequal values around their first differing character.""" + + left_text: str = str(left_value) + right_text: str = str(right_value) + first_difference: int | None = _first_difference(left=left_text, right=right_text) + return ( + _render_value(text=left_text, first_difference=first_difference, options=options), + _render_value(text=right_text, first_difference=first_difference, options=options), + ) + + +def display_example_value(value: RenderedDiffExampleValue) -> str: + """Render one bounded value for terminal output.""" + + if value.suppressed: + return f"" + rendered: str = value.text or "" + if value.truncated: + return f"{rendered} [truncated; {value.original_length:,} chars]" + return rendered + + +def render_key_value(value: object) -> str: + """Keep ordinary keys complete and identify exceptionally long shortened keys.""" + + text: str = str(value) + if len(text) <= _MAX_RENDERED_KEY_LENGTH: + return text + digest: str = hashlib.sha256(text.encode("utf-8")).hexdigest()[:12] + return f"{text[:_MAX_RENDERED_KEY_LENGTH]}… [truncated; {len(text):,} chars; sha256:{digest}]" + + +def example_value_payload(value: RenderedDiffExampleValue) -> dict[str, object]: + """Build stable structured evidence for one rendered value.""" + + return { + "value": value.text, + "original_length": value.original_length, + "truncated": value.truncated, + "suppressed": value.suppressed, + "first_difference": value.first_difference, + } + + +def _render_value( + *, + text: str, + first_difference: int | None, + options: DiffExampleRenderOptions, +) -> RenderedDiffExampleValue: + original_length: int = len(text) + if options.suppress_values: + return RenderedDiffExampleValue( + text=None, + original_length=original_length, + truncated=False, + suppressed=True, + first_difference=first_difference, + ) + limit: int | None = options.max_value_length + if limit is None or original_length <= limit: + return RenderedDiffExampleValue( + text=text, + original_length=original_length, + truncated=False, + suppressed=False, + first_difference=first_difference, + ) + difference: int = first_difference or 0 + context_before: int = min(limit // 3, max(limit - 1, 0)) + start: int = min(max(difference - context_before, 0), original_length - limit) + end: int = start + limit + fragment: str = text[start:end] + rendered: str = f"{'…' if start else ''}{fragment}{'…' if end < original_length else ''}" + return RenderedDiffExampleValue( + text=rendered, + original_length=original_length, + truncated=True, + suppressed=False, + first_difference=first_difference, + ) + + +def _first_difference(*, left: str, right: str) -> int | None: + index: int + for index, (left_char, right_char) in enumerate(zip(left, right, strict=False)): + if left_char != right_char: + return index + if len(left) != len(right): + return min(len(left), len(right)) + return None diff --git a/src/sqlbuild/cli/commands/_helpers/diff/execution.py b/src/sqlbuild/cli/commands/_helpers/diff/execution.py index 804fe903f2..b8463e48c4 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/execution.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/execution.py @@ -3,9 +3,10 @@ from __future__ import annotations import sys +import time from datetime import UTC, datetime from pathlib import Path -from typing import Any +from typing import Any, TextIO from sqlbuild.adapter.contract.classes.base_adapter import BaseAdapter from sqlbuild.adapter.contract.models import RelationLookup @@ -20,12 +21,13 @@ resolve_target_connection_config, ) from sqlbuild.cli.commands.constants import READ_ONLY_QUERY_ROOT_KEYS -from sqlbuild.cli.commands.exceptions import CliUserError +from sqlbuild.cli.commands.exceptions import CliUserError, QueryDiffExecutionError from sqlbuild.cli.commands.models import ( DiffCommandRequest, DiffInvocation, DirectDiffPreparation, QueryDiffPreparation, + QueryDiffRunOutcome, VirtualDiffPreparation, VirtualDiffRunOutcome, ) @@ -219,6 +221,8 @@ def prepare_query_diff( connection_config=connection_config, left_sql=left_sql, right_sql=right_sql, + left_label=(request.left_label or "left query").strip(), + right_label=(request.right_label or "right query").strip(), selected_target=target_name, database=database, schema=schema, @@ -237,99 +241,45 @@ def prepare_query_diff( def execute_query_diff( *, request: DiffCommandRequest, preparation: QueryDiffPreparation -) -> DiffExecutionResult: +) -> QueryDiffRunOutcome: """Materialize, compare, and clean one raw-query pair.""" adapter: BaseAdapter = preparation.adapter + total_started_at: float = time.perf_counter() + phase_seconds: dict[str, float] = {} print("Query diff connection START", file=sys.stderr) + connection_started_at: float = time.perf_counter() try: connection: Any = adapter.connect(preparation.connection_config) except Exception: - print("Query diff connection ERROR", file=sys.stderr) + connection_elapsed: float = time.perf_counter() - connection_started_at + print(f"Query diff connection ERROR ({connection_elapsed:.2f}s)", file=sys.stderr) + print( + f"Query diff total ERROR ({time.perf_counter() - total_started_at:.2f}s)", + file=sys.stderr, + ) raise - print("Query diff connection OK", file=sys.stderr) - left: QueryDiffArtifact = QueryDiffArtifactLifecycle.build( - adapter=adapter, - run_id=preparation.run_id, - side="left", - database=preparation.database, - schema=preparation.schema, - ) - right: QueryDiffArtifact = QueryDiffArtifactLifecycle.build( - adapter=adapter, - run_id=preparation.run_id, - side="right", - database=preparation.database, - schema=preparation.schema, - ) + phase_seconds["connection"] = time.perf_counter() - connection_started_at + print(f"Query diff connection OK ({phase_seconds['connection']:.2f}s)", file=sys.stderr) + left, right = _build_query_diff_artifacts(adapter=adapter, preparation=preparation) created: list[QueryDiffArtifact] = [] cleanup_errors: list[Exception] = [] + model_result: ModelDiffResult | None = None try: - print("Query diff cleanup START", file=sys.stderr) - try: - cleanup_result: QueryDiffArtifactCleanupResult = ( - QueryDiffArtifactLifecycle.cleanup_expired( - adapter=adapter, - connection=connection, - database=preparation.database, - schema=preparation.schema, - now=datetime.now(UTC), - ) - ) - except Exception as error: - print( - "Query diff cleanup WARNING recovery cleanup could not inspect historical " - f"ownership evidence: {error}", - file=sys.stderr, - ) - else: - print( - f"Query diff cleanup OK ({len(cleanup_result.cleaned)} expired)", - file=sys.stderr, - ) - for relation in cleanup_result.untracked: - print( - "Query diff cleanup WARNING reserved-name relation lacks matching " - f"ownership evidence: {relation.name}", - file=sys.stderr, - ) - artifact_lookup: RelationLookup = build_relation_lookup( + phase_seconds["reconciliation"] = _reconcile_query_diff_artifacts( adapter=adapter, connection=connection, - locations=tuple( - (artifact.database, artifact.schema, artifact.name) for artifact in (left, right) - ), - ) - for artifact in (left, right): - if artifact_lookup.exists( - database=artifact.database, - schema=artifact.schema, - name=artifact.name, - ): - raise CliUserError( - f"query-diff artifact already exists: {artifact.relation}", - code="C245", - help="wait for its TTL cleanup or investigate the reserved-name relation", - ) - ttl: Duration | None = Duration.parse(preparation.artifact_ttl) - if ttl is None: - raise CliUserError("diff.query_artifact_ttl must be a positive duration", code="C228") - expires_at: datetime = ttl.add_to(datetime.now(UTC)) - _preflight_query_columns( - adapter=adapter, - connection=connection, - sql=preparation.left_sql, - label="left", - max_columns=request.max_columns, + preparation=preparation, ) - _preflight_query_columns( + expires_at, phase_seconds["inspection"] = _inspect_query_diff_inputs( adapter=adapter, connection=connection, - sql=preparation.right_sql, - label="right", + preparation=preparation, + artifacts=(left, right), max_columns=request.max_columns, ) print("Query diff materialization START", file=sys.stderr) + materialization_started_at: float = time.perf_counter() try: QueryDiffArtifactLifecycle.materialize( adapter=adapter, @@ -348,16 +298,22 @@ def execute_query_diff( ) created.append(right) except Exception: - print("Query diff materialization ERROR", file=sys.stderr) + phase_seconds["materialization"] = time.perf_counter() - materialization_started_at + print( + f"Query diff materialization ERROR ({phase_seconds['materialization']:.2f}s)", + file=sys.stderr, + ) raise + phase_seconds["materialization"] = time.perf_counter() - materialization_started_at print( f"Query diff materialization OK ({preparation.schema}; expires " - f"{expires_at.isoformat()})", + f"{expires_at.isoformat()}; {phase_seconds['materialization']:.2f}s)", file=sys.stderr, ) print("Query diff comparison START", file=sys.stderr) + comparison_started_at: float = time.perf_counter() try: - model_result: ModelDiffResult = execute_prepared_query_diff( + model_result = execute_prepared_query_diff( adapter=adapter, connection=connection, left_relation=left.relation, @@ -375,38 +331,237 @@ def execute_query_diff( tolerance_overrides=parse_cli_tolerance_overrides( values=request.tolerance_overrides ), + comparison_name=(f"{preparation.left_label} vs {preparation.right_label}"), ), ) except Exception: - print("Query diff comparison ERROR", file=sys.stderr) + phase_seconds["comparison"] = time.perf_counter() - comparison_started_at + print( + f"Query diff comparison ERROR ({phase_seconds['comparison']:.2f}s)", + file=sys.stderr, + ) raise - print("Query diff comparison OK", file=sys.stderr) - return DiffExecutionResult(model_results=(model_result,)) + phase_seconds["comparison"] = time.perf_counter() - comparison_started_at + print(f"Query diff comparison OK ({phase_seconds['comparison']:.2f}s)", file=sys.stderr) finally: active_error: BaseException | None = sys.exc_info()[1] - print("Query diff immediate cleanup START", file=sys.stderr) - for artifact in reversed(created): - try: - QueryDiffArtifactLifecycle.cleanup( - adapter=adapter, - connection=connection, - artifact=artifact, - ) - except Exception as error: - cleanup_errors.append(error) + cleanup_errors, phase_seconds["cleanup"] = _cleanup_query_diff_artifacts( + adapter=adapter, + connection=connection, + created=created, + ) + phase_seconds["total"] = time.perf_counter() - total_started_at + total_status: str = "ERROR" if active_error is not None or cleanup_errors else "OK" + print( + f"Query diff total {total_status} ({phase_seconds['total']:.2f}s)", + file=sys.stderr, + ) if cleanup_errors: - print( - f"Query diff immediate cleanup ERROR ({len(cleanup_errors)} failed)", - file=sys.stderr, - ) - else: - print( - f"Query diff immediate cleanup OK ({len(created)} removed)", - file=sys.stderr, + raise cleanup_errors[0] + if model_result is None: + raise QueryDiffExecutionError( + "query diff comparison completed without a result", code="C256" + ) + result: DiffExecutionResult = DiffExecutionResult(model_results=(model_result,)) + outcome, exit_code = _resolve_query_diff_outcome(request=request, result=result) + return QueryDiffRunOutcome( + result=result, + phase_seconds=phase_seconds, + outcome=outcome, + exit_code=exit_code, + ) + + +def _build_query_diff_artifacts( + *, adapter: BaseAdapter, preparation: QueryDiffPreparation +) -> tuple[QueryDiffArtifact, QueryDiffArtifact]: + return ( + QueryDiffArtifactLifecycle.build( + adapter=adapter, + run_id=preparation.run_id, + side="left", + database=preparation.database, + schema=preparation.schema, + ), + QueryDiffArtifactLifecycle.build( + adapter=adapter, + run_id=preparation.run_id, + side="right", + database=preparation.database, + schema=preparation.schema, + ), + ) + + +def _cleanup_query_diff_artifacts( + *, + adapter: BaseAdapter, + connection: Any, + created: list[QueryDiffArtifact], +) -> tuple[list[Exception], float]: + print("Query diff immediate cleanup START", file=sys.stderr) + started_at: float = time.perf_counter() + errors: list[Exception] = [] + for artifact in reversed(created): + try: + QueryDiffArtifactLifecycle.cleanup( + adapter=adapter, + connection=connection, + artifact=artifact, ) + except Exception as error: + errors.append(error) + try: adapter.close(connection) - if cleanup_errors and active_error is None: - raise cleanup_errors[0] + except Exception as error: + errors.append(error) + elapsed: float = time.perf_counter() - started_at + if errors: + print( + f"Query diff immediate cleanup ERROR ({len(errors)} failed; {elapsed:.2f}s)", + file=sys.stderr, + ) + else: + print( + f"Query diff immediate cleanup OK ({len(created)} removed; {elapsed:.2f}s)", + file=sys.stderr, + ) + return errors, elapsed + + +def _resolve_query_diff_outcome( + *, request: DiffCommandRequest, result: DiffExecutionResult +) -> tuple[str, int]: + if not request.schema_only and _query_schema_prevented_row_comparison(result=result): + return "incomplete", 2 + if _query_result_has_findings(result=result): + return "findings", 1 + return "pass", 0 + + +def _query_schema_prevented_row_comparison(*, result: DiffExecutionResult) -> bool: + return any( + model.row_result is None + and ( + model.schema_result.added_columns + or model.schema_result.removed_columns + or model.schema_result.type_changed_columns + ) + for model in result.model_results + ) + + +def _query_result_has_findings(*, result: DiffExecutionResult) -> bool: + return any( + model.schema_result.added_columns + or model.schema_result.removed_columns + or model.schema_result.type_changed_columns + or ( + model.row_result is not None + and ( + model.row_result.unequal_count + or model.row_result.left_only_count + or model.row_result.right_only_count + ) + ) + for model in result.model_results + ) + + +def _inspect_query_diff_inputs( + *, + adapter: BaseAdapter, + connection: Any, + preparation: QueryDiffPreparation, + artifacts: tuple[QueryDiffArtifact, QueryDiffArtifact], + max_columns: int | None, +) -> tuple[datetime, float]: + print("Query diff inspection START", file=sys.stderr) + started_at: float = time.perf_counter() + try: + artifact_lookup: RelationLookup = build_relation_lookup( + adapter=adapter, + connection=connection, + locations=tuple( + (artifact.database, artifact.schema, artifact.name) for artifact in artifacts + ), + ) + for artifact in artifacts: + if artifact_lookup.exists( + database=artifact.database, + schema=artifact.schema, + name=artifact.name, + ): + raise CliUserError( + f"query-diff artifact already exists: {artifact.relation}", + code="C245", + help="wait for its TTL cleanup or investigate the reserved-name relation", + ) + ttl: Duration | None = Duration.parse(preparation.artifact_ttl) + if ttl is None: + raise CliUserError("diff.query_artifact_ttl must be a positive duration", code="C228") + _preflight_query_columns( + adapter=adapter, + connection=connection, + sql=preparation.left_sql, + label=preparation.left_label, + max_columns=max_columns, + ) + _preflight_query_columns( + adapter=adapter, + connection=connection, + sql=preparation.right_sql, + label=preparation.right_label, + max_columns=max_columns, + ) + except Exception: + elapsed: float = time.perf_counter() - started_at + print( + f"Query diff inspection ERROR ({elapsed:.2f}s)", + file=sys.stderr, + ) + raise + elapsed = time.perf_counter() - started_at + print(f"Query diff inspection OK ({elapsed:.2f}s)", file=sys.stderr) + return ttl.add_to(datetime.now(UTC)), elapsed + + +def _reconcile_query_diff_artifacts( + *, + adapter: BaseAdapter, + connection: Any, + preparation: QueryDiffPreparation, +) -> float: + print("Query diff reconciliation START", file=sys.stderr) + started_at: float = time.perf_counter() + try: + result: QueryDiffArtifactCleanupResult = QueryDiffArtifactLifecycle.cleanup_expired( + adapter=adapter, + connection=connection, + database=preparation.database, + schema=preparation.schema, + now=datetime.now(UTC), + ) + except Exception as error: + elapsed: float = time.perf_counter() - started_at + print( + "Query diff reconciliation WARNING recovery cleanup could not inspect historical " + f"ownership evidence: {error} ({elapsed:.2f}s)", + file=sys.stderr, + ) + return elapsed + elapsed = time.perf_counter() - started_at + print( + f"Query diff reconciliation OK ({len(result.cleaned)} expired; {elapsed:.2f}s)", + file=sys.stderr, + ) + for relation in result.untracked: + print( + "Query diff reconciliation WARNING reserved-name relation lacks matching " + f"ownership evidence: {relation.name}", + file=sys.stderr, + ) + return elapsed def _resolve_query_input(*, inline: str | None, file: Path | None, label: str) -> str: @@ -494,22 +649,23 @@ def _preflight_query_columns( label: str, max_columns: int | None, ) -> None: + display_label: str = label if label.lower().endswith("query") else f"{label} query" columns: tuple[str, ...] = adapter.query_column_names(connection=connection, sql=sql) if not columns: - raise CliUserError(f"{label} query returned no columns", code="C237") + raise CliUserError(f"{display_label} returned no columns", code="C237") normalized: tuple[str, ...] = tuple(column.lower() for column in columns) duplicates: tuple[str, ...] = tuple( sorted({column for column in normalized if normalized.count(column) > 1}) ) if duplicates: raise CliUserError( - f"{label} query returns duplicate column names: {', '.join(duplicates)}", + f"{display_label} returns duplicate column names: {', '.join(duplicates)}", code="C238", help="add unique aliases before comparing query results", ) if max_columns is not None and len(columns) > max_columns: raise CliUserError( - f"{label} query has {len(columns)} columns, exceeding --max-columns {max_columns}", + f"{display_label} has {len(columns)} columns, exceeding --max-columns {max_columns}", code="C239", ) @@ -552,8 +708,9 @@ def execute_virtual_diff( ) -> VirtualDiffRunOutcome: """Execute a virtual environment diff.""" + progress_stream: TextIO = sys.stderr if request.json_output else sys.stdout planning_progress: PlanningProgressReporter = PlanningProgressReporter( - stream=sys.stdout, + stream=progress_stream, use_color=preparation.use_color, ) connection_progress: ConnectionProgressReporter = ConnectionProgressReporter( @@ -561,7 +718,7 @@ def execute_virtual_diff( project_config=invocation.discovered_inputs.project_config, local_config=invocation.discovered_inputs.local_config, ), - stream=sys.stdout, + stream=progress_stream, use_color=preparation.use_color, ) ( diff --git a/src/sqlbuild/cli/commands/_helpers/diff/invocation.py b/src/sqlbuild/cli/commands/_helpers/diff/invocation.py index dc3e4abfc9..2b5c50a41b 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/invocation.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/invocation.py @@ -4,21 +4,35 @@ from pathlib import Path -from sqlbuild.cli.commands.exceptions import CliUserError +from sqlbuild.cli.commands.exceptions import CliUserError, QueryDiffIncompleteError from sqlbuild.cli.commands.models import DiffCommandRequest, DiffInvocation from sqlbuild.compiler.discovery.main.discover import discover_project_inputs +from sqlbuild.compiler.discovery.main.discover_configuration import ( + discover_project_configuration, +) from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs def resolve_diff_invocation(*, request: DiffCommandRequest) -> DiffInvocation: """Validate diff flags and discover project inputs.""" - _validate_diff_request(request=request) + try: + _validate_diff_request(request=request) + except CliUserError as error: + if is_query_diff_request(request=request): + raise QueryDiffIncompleteError( + error.message, + code=error.code, + help=error.help, + ) from error + raise effective_project_dir: Path = ( request.project_dir if request.project_dir is not None else Path.cwd() ) - discovered_inputs: DiscoveredProjectInputs = discover_project_inputs( - project_dir=effective_project_dir + discovered_inputs: DiscoveredProjectInputs = ( + discover_project_configuration(project_dir=effective_project_dir) + if is_query_diff_request(request=request) + else discover_project_inputs(project_dir=effective_project_dir) ) is_virtual_mode: bool = discovered_inputs.project_config.settings.virtual_environments if not request.select and not is_virtual_mode and not is_query_diff_request(request=request): @@ -52,6 +66,18 @@ def _validate_diff_request(*, request: DiffCommandRequest) -> None: raise CliUserError("diff --max-models must be positive", code="C210") if request.max_columns is not None and request.max_columns <= 0: raise CliUserError("diff --max-columns must be positive", code="C211") + if request.max_value_length is not None and request.max_value_length <= 0: + raise CliUserError("diff --max-value-length must be positive", code="C246") + if request.full_example_values and request.max_value_length is not None: + raise CliUserError( + "diff --full-example-values cannot be combined with --max-value-length", + code="C247", + ) + if request.full_example_values and request.suppress_example_values: + raise CliUserError( + "diff --full-example-values cannot be combined with --no-example-values", + code="C248", + ) if request.unkeyed and request.unique_key_override: raise CliUserError("diff --unkeyed cannot be combined with --key", code="C226") if request.unkeyed and (request.sample_rows is not None or request.sample_seed is not None): @@ -97,6 +123,10 @@ def _validate_query_diff_request(*, request: DiffCommandRequest) -> None: raise CliUserError("raw-query diff does not use --allow-partial-diff", code="C215") if request.max_models is not None: raise CliUserError("raw-query diff does not accept --max-models", code="C243") + if request.left_label is not None and not request.left_label.strip(): + raise CliUserError("raw-query diff --left-label must not be empty", code="C249") + if request.right_label is not None and not request.right_label.strip(): + raise CliUserError("raw-query diff --right-label must not be empty", code="C250") if request.left_query is None and request.left_query_file is None: raise CliUserError("raw-query diff requires a left query", code="C216") if request.left_query is not None and request.left_query_file is not None: @@ -130,3 +160,5 @@ def _validate_model_diff_request(*, request: DiffCommandRequest) -> None: raise CliUserError("model diff requires FROM:TO", code="C224") if request.selected_target is not None: raise CliUserError("model diff does not accept --target", code="C225") + if request.left_label is not None or request.right_label is not None: + raise CliUserError("model diff does not accept query labels", code="C251") diff --git a/src/sqlbuild/cli/commands/_helpers/diff/json_output.py b/src/sqlbuild/cli/commands/_helpers/diff/json_output.py index c7a8c141f9..88cd56218e 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/json_output.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/json_output.py @@ -3,13 +3,87 @@ from __future__ import annotations import json +import math from datetime import date, datetime from pathlib import Path -from sqlbuild.adapter.contract.models import RowDiffCoverage, RowDiffResult +from sqlbuild.adapter.contract.models import RowDiffCoverage, RowDiffResult, RowDiffSampleRow +from sqlbuild.cli.commands._helpers.diff.evidence import ( + example_value_payload, + render_example_pair, +) from sqlbuild.cli.commands._helpers.diff.output import has_diff_failures +from sqlbuild.cli.commands.constants import DEFAULT_DIFF_MAX_VALUE_LENGTH +from sqlbuild.cli.commands.models import DiffExampleRenderOptions +from sqlbuild.executor.diff.constants import DIFF_INPUT_KIND_QUERY from sqlbuild.executor.diff.models import DiffExecutionResult, ModelDiffResult +_DEFAULT_EXAMPLE_RENDER_OPTIONS: DiffExampleRenderOptions = DiffExampleRenderOptions( + max_value_length=DEFAULT_DIFF_MAX_VALUE_LENGTH +) + + +def render_diff_json_output( + *, + result: DiffExecutionResult, + from_label: str, + to_label: str, + example_render_options: DiffExampleRenderOptions = _DEFAULT_EXAMPLE_RENDER_OPTIONS, + phase_seconds: dict[str, float] | None = None, + outcome: str | None = None, +) -> str: + """Render one stable structured diff document.""" + + return json.dumps( + _json_safe_value( + _diff_payload( + result=result, + from_label=from_label, + to_label=to_label, + example_render_options=example_render_options, + phase_seconds=phase_seconds, + outcome=outcome, + ) + ), + allow_nan=False, + indent=2, + sort_keys=True, + default=_json_default, + ) + + +def render_diff_error_json(*, status: str, code: str, message: str) -> str: + """Render a machine-readable incomplete or execution-failure outcome.""" + + return json.dumps( + { + "schema_version": 1, + "status": status, + "outcome": status, + "error": {"code": code, "message": message}, + "models": [], + "query_comparisons": [], + }, + indent=2, + sort_keys=True, + ) + + +def _json_safe_value(value: object) -> object: + if isinstance(value, float) and not math.isfinite(value): + if math.isnan(value): + label: str = "NaN" + elif value > 0: + label = "Infinity" + else: + label = "-Infinity" + return {"kind": "non_finite_number", "value": label} + if isinstance(value, dict): + return {key: _json_safe_value(item) for key, item in value.items()} + if isinstance(value, list | tuple): + return [_json_safe_value(item) for item in value] + return value + def write_diff_json_output( *, @@ -17,30 +91,88 @@ def write_diff_json_output( result: DiffExecutionResult, from_label: str, to_label: str, + example_render_options: DiffExampleRenderOptions = _DEFAULT_EXAMPLE_RENDER_OPTIONS, + phase_seconds: dict[str, float] | None = None, + outcome: str | None = None, ) -> None: """Write a stable structured diff document when a path is requested.""" if path is None: return + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text( + render_diff_json_output( + result=result, + from_label=from_label, + to_label=to_label, + example_render_options=example_render_options, + phase_seconds=phase_seconds, + outcome=outcome, + ) + + "\n", + encoding="utf-8", + ) + + +def _diff_payload( + *, + result: DiffExecutionResult, + from_label: str, + to_label: str, + example_render_options: DiffExampleRenderOptions, + phase_seconds: dict[str, float] | None, + outcome: str | None, +) -> dict[str, object]: + model_payloads: list[dict[str, object]] = [ + _model_payload( + model=model, + from_label=from_label, + to_label=to_label, + example_render_options=example_render_options, + ) + for model in result.model_results + ] + resolved_outcome: str = outcome or ("findings" if has_diff_failures(result) else "pass") + status: str = { + "pass": "no_differences_found", + "findings": "differences_found", + }.get(resolved_outcome, resolved_outcome) payload: dict[str, object] = { "schema_version": 1, - "status": "differences_found" if has_diff_failures(result) else "no_differences_found", + "status": status, + "outcome": resolved_outcome, "from": from_label, "to": to_label, - "models": [_model_payload(model=model) for model in result.model_results], + "models": model_payloads, } - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text( - json.dumps(payload, indent=2, sort_keys=True, default=_json_default) + "\n", - encoding="utf-8", - ) + query_payloads: list[dict[str, object]] = [ + model_payload + for model, model_payload in zip(result.model_results, model_payloads, strict=True) + if model.input_kind == DIFF_INPUT_KIND_QUERY + ] + if query_payloads: + payload["input_kind"] = DIFF_INPUT_KIND_QUERY + payload["query_comparisons"] = query_payloads + if phase_seconds is not None: + payload["phase_seconds"] = { + phase: round(seconds, 6) for phase, seconds in sorted(phase_seconds.items()) + } + return payload -def _model_payload(*, model: ModelDiffResult) -> dict[str, object]: +def _model_payload( + *, + model: ModelDiffResult, + from_label: str, + to_label: str, + example_render_options: DiffExampleRenderOptions, +) -> dict[str, object]: rows: RowDiffResult | None = model.row_result payload: dict[str, object] = { "input_kind": model.input_kind, "name": model.name, + "from_label": from_label, + "to_label": to_label, "from_relation": model.left_relation, "to_relation": model.right_relation, "unique_key": list(model.unique_key), @@ -88,10 +220,66 @@ def _model_payload(*, model: ModelDiffResult) -> dict[str, object]: "from_only_count": rows.left_only_count, "to_only_count": rows.right_only_count, "changed_columns": {column.name: column.mismatched_count for column in rows.column_results}, + "column_mismatches": [ + {"column": column.name, "rows": column.mismatched_count} + for column in rows.column_results + if column.mismatched_count > 0 + ], } + payload["examples"] = _examples_payload( + model=model, + options=example_render_options, + ) return payload +def _examples_payload( + *, model: ModelDiffResult, options: DiffExampleRenderOptions +) -> dict[str, object]: + unequal_rows: list[dict[str, object]] = [] + values_truncated: bool = False + sample_row: RowDiffSampleRow + for sample_row in model.unequal_row_samples: + cells: list[dict[str, object]] = [] + for cell in sample_row.changed_cells: + left_value, right_value = render_example_pair( + left_value=cell.left_value, + right_value=cell.right_value, + options=options, + ) + values_truncated = values_truncated or left_value.truncated or right_value.truncated + cells.append( + { + "column": cell.name, + "from": example_value_payload(left_value), + "to": example_value_payload(right_value), + } + ) + unequal_rows.append( + { + "key": {name: value for name, value in sample_row.key_values}, + "changed_cells": cells, + } + ) + rows: RowDiffResult | None = model.row_result + return { + "unequal_rows": unequal_rows, + "from_only_keys": [dict(sample) for sample in model.left_only_key_samples], + "to_only_keys": [dict(sample) for sample in model.right_only_key_samples], + "truncated_by_count": bool( + rows is not None + and ( + rows.unequal_count > len(model.unequal_row_samples) + or rows.left_only_count > len(model.left_only_key_samples) + or rows.right_only_count > len(model.right_only_key_samples) + ) + ), + "values_truncated": values_truncated, + "values_suppressed": options.suppress_values, + "max_value_length": options.max_value_length, + } + + def _coverage_payload(*, coverage: RowDiffCoverage | None) -> dict[str, object] | None: if coverage is None: return None diff --git a/src/sqlbuild/cli/commands/_helpers/diff/output.py b/src/sqlbuild/cli/commands/_helpers/diff/output.py index e35593c298..b235670339 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/output.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/output.py @@ -17,6 +17,13 @@ RowDiffSampleRow, SchemaDiffResult, ) +from sqlbuild.cli.commands._helpers.diff.evidence import ( + display_example_value, + render_example_pair, + render_key_value, +) +from sqlbuild.cli.commands.constants import DEFAULT_DIFF_MAX_VALUE_LENGTH +from sqlbuild.cli.commands.models import DiffExampleRenderOptions from sqlbuild.executor.diff.constants import DIFF_INPUT_KIND_QUERY from sqlbuild.executor.diff.models import DiffExecutionResult, ModelDiffResult @@ -28,6 +35,9 @@ _RICH_ADDED_STYLE: str = "green" _RICH_REMOVED_STYLE: str = "red" _RICH_MUTED_STYLE: str = "dim" +_DEFAULT_EXAMPLE_RENDER_OPTIONS: DiffExampleRenderOptions = DiffExampleRenderOptions( + max_value_length=DEFAULT_DIFF_MAX_VALUE_LENGTH +) def render_diff_output( @@ -40,6 +50,8 @@ def render_diff_output( verbose: bool, max_column_examples: int, max_row_only_examples: int, + example_render_options: DiffExampleRenderOptions = _DEFAULT_EXAMPLE_RENDER_OPTIONS, + outcome: str | None = None, ) -> str: """Render a concise Rich terminal summary for a diff result.""" @@ -65,6 +77,8 @@ def render_diff_output( console.print( Text(f"selected {input_label}: {len(result.model_results):,}", justify="center") ) + resolved_outcome: str = outcome or ("findings" if has_diff_failures(result) else "pass") + console.print(Text(f"outcome: {resolved_outcome}", justify="center")) console.print() model_result: ModelDiffResult for index, model_result in enumerate(result.model_results): @@ -81,6 +95,7 @@ def render_diff_output( verbose=verbose, max_column_examples=max_column_examples, max_row_only_examples=max_row_only_examples, + example_render_options=example_render_options, ) return capture.get().rstrip() @@ -115,6 +130,7 @@ def _render_model_result( verbose: bool, max_column_examples: int, max_row_only_examples: int, + example_render_options: DiffExampleRenderOptions, ) -> None: console.print( _render_overview( @@ -150,12 +166,14 @@ def _render_model_result( model_result=model_result, verbose=verbose, max_column_examples=max_column_examples, + example_render_options=example_render_options, ), ) if verbose and model_result.unequal_row_samples: example_content: RenderableType | None = _render_examples( model_result=model_result, max_column_examples=max_column_examples, + example_render_options=example_render_options, ) if example_content is not None: _print_section( @@ -481,6 +499,7 @@ def _render_changed_columns( model_result: ModelDiffResult, verbose: bool, max_column_examples: int, + example_render_options: DiffExampleRenderOptions, ) -> RenderableType: row_result: RowDiffResult | None = model_result.row_result if row_result is None: @@ -529,6 +548,7 @@ def _render_changed_columns( model_result=model_result, max_column_examples=max_column_examples, visible_column_names=tuple(column.name for column in visible_columns), + example_render_options=example_render_options, ) if example_content is not None: return Group( @@ -557,16 +577,31 @@ def _render_examples( model_result: ModelDiffResult, max_column_examples: int, visible_column_names: tuple[str, ...] | None = None, + example_render_options: DiffExampleRenderOptions, ) -> RenderableType | None: blocks: list[RenderableType] = [] grouped_examples: dict[str, list[str]] = {} sample_row: RowDiffSampleRow for sample_row in model_result.unequal_row_samples: - key_label: str = ", ".join(f"{key}={value}" for key, value in sample_row.key_values) + key_label: str = ", ".join( + f"{key}={render_key_value(value)}" for key, value in sample_row.key_values + ) cell: RowDiffSampleCell for cell in sample_row.changed_cells: + left_value, right_value = render_example_pair( + left_value=cell.left_value, + right_value=cell.right_value, + options=example_render_options, + ) + difference_label: str = ( + f" (first difference at character {left_value.first_difference + 1:,})" + if (left_value.truncated or right_value.truncated) + and left_value.first_difference is not None + else "" + ) grouped_examples.setdefault(cell.name, []).append( - f"{key_label} | {cell.left_value} -> {cell.right_value}" + f"{key_label} | {display_example_value(left_value)} -> " + f"{display_example_value(right_value)}{difference_label}" ) column_name: str for column_name in sorted(grouped_examples): @@ -577,7 +612,7 @@ def _render_examples( visible_examples: list[str] = all_examples[:max_column_examples] example: str for example in visible_examples: - blocks.append(f" - {example}") + blocks.append(Text(f" - {example}")) if len(all_examples) > len(visible_examples): blocks.append( Text( @@ -601,7 +636,9 @@ def _render_side_only_samples( ] sample: tuple[tuple[str, object], ...] for sample in visible_samples: - blocks.append(" - " + " | ".join(f"{key}={value}" for key, value in sample)) + blocks.append( + Text(" - " + " | ".join(f"{key}={render_key_value(value)}" for key, value in sample)) + ) if len(key_samples) > len(visible_samples): truncation_message: str = ( f" showing {len(visible_samples):,} of {len(key_samples):,} {side_label} only rows" diff --git a/src/sqlbuild/cli/commands/_helpers/diff/outputs.py b/src/sqlbuild/cli/commands/_helpers/diff/outputs.py index 24a63961ae..63cc070018 100644 --- a/src/sqlbuild/cli/commands/_helpers/diff/outputs.py +++ b/src/sqlbuild/cli/commands/_helpers/diff/outputs.py @@ -2,11 +2,16 @@ from __future__ import annotations -from sqlbuild.cli.commands._helpers.diff.json_output import write_diff_json_output +from sqlbuild.cli.commands._helpers.diff.json_output import ( + render_diff_json_output, + write_diff_json_output, +) from sqlbuild.cli.commands._helpers.diff.output import has_diff_failures, render_diff_output from sqlbuild.cli.commands._helpers.diff.virtual_output import format_virtual_diff_header +from sqlbuild.cli.commands.constants import DEFAULT_DIFF_MAX_VALUE_LENGTH from sqlbuild.cli.commands.models import ( DiffCommandRequest, + DiffExampleRenderOptions, DirectDiffPreparation, VirtualDiffPreparation, VirtualDiffRunOutcome, @@ -20,27 +25,47 @@ def write_direct_diff_output( request: DiffCommandRequest, preparation: DirectDiffPreparation, result: DiffExecutionResult, + phase_seconds: dict[str, float] | None = None, + outcome: str | None = None, ) -> None: """Write direct diff output.""" - print( - render_diff_output( - result=result, - from_label=preparation.from_target, - to_label=preparation.to_target, - mode_label=_mode_label(request=request), - use_color=not request.no_color and supports_color(), - verbose=request.verbose, - max_column_examples=preparation.effective_max_column_examples, - max_row_only_examples=preparation.effective_max_row_only_examples, - ) - ) + example_render_options: DiffExampleRenderOptions = _example_render_options(request=request) write_diff_json_output( path=request.json_output_path, result=result, from_label=preparation.from_target, to_label=preparation.to_target, + example_render_options=example_render_options, + phase_seconds=phase_seconds, + outcome=outcome, ) + if request.json_output: + print( + render_diff_json_output( + result=result, + from_label=preparation.from_target, + to_label=preparation.to_target, + example_render_options=example_render_options, + phase_seconds=phase_seconds, + outcome=outcome, + ) + ) + else: + print( + render_diff_output( + result=result, + from_label=preparation.from_target, + to_label=preparation.to_target, + mode_label=_mode_label(request=request), + use_color=not request.no_color and supports_color(), + verbose=request.verbose, + max_column_examples=preparation.effective_max_column_examples, + max_row_only_examples=preparation.effective_max_row_only_examples, + example_render_options=example_render_options, + outcome=outcome, + ) + ) def write_virtual_diff_output( @@ -51,6 +76,24 @@ def write_virtual_diff_output( ) -> None: """Write virtual diff header and optional diff body.""" + example_render_options: DiffExampleRenderOptions = _example_render_options(request=request) + write_diff_json_output( + path=request.json_output_path, + result=outcome.result, + from_label=preparation.from_virtual_environment, + to_label=preparation.to_virtual_environment, + example_render_options=example_render_options, + ) + if request.json_output: + print( + render_diff_json_output( + result=outcome.result, + from_label=preparation.from_virtual_environment, + to_label=preparation.to_virtual_environment, + example_render_options=example_render_options, + ) + ) + return header: str = format_virtual_diff_header( from_virtual_environment=preparation.from_virtual_environment, to_virtual_environment=preparation.to_virtual_environment, @@ -73,16 +116,11 @@ def write_virtual_diff_output( verbose=request.verbose, max_column_examples=preparation.effective_max_column_examples, max_row_only_examples=preparation.effective_max_row_only_examples, + example_render_options=example_render_options, ) ) else: print("No VDE ref differences in selected scope.") - write_diff_json_output( - path=request.json_output_path, - result=outcome.result, - from_label=preparation.from_virtual_environment, - to_label=preparation.to_virtual_environment, - ) def resolve_diff_exit_code(result: DiffExecutionResult) -> int: @@ -91,6 +129,18 @@ def resolve_diff_exit_code(result: DiffExecutionResult) -> int: return 1 if has_diff_failures(result) else 0 +def _example_render_options(*, request: DiffCommandRequest) -> DiffExampleRenderOptions: + max_value_length: int | None = ( + None + if request.full_example_values + else request.max_value_length or DEFAULT_DIFF_MAX_VALUE_LENGTH + ) + return DiffExampleRenderOptions( + max_value_length=max_value_length, + suppress_values=request.suppress_example_values, + ) + + def _mode_label(*, request: DiffCommandRequest) -> str: if request.schema_only: return "schema-only" diff --git a/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py b/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py index 38be21541c..51ce70d8ff 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/dispatch.py @@ -375,6 +375,7 @@ def dispatch_cli_command(*, args: CliNamespace, handlers: CliEntrypointHandlers) sample_rows=args.sample_rows, sample_seed=args.sample_seed, exhaustive=args.exhaustive, + json_output=args.json, json_output_path=args.json_output, max_models=args.max_models, max_columns=args.max_columns, @@ -386,12 +387,17 @@ def dispatch_cli_command(*, args: CliNamespace, handlers: CliEntrypointHandlers) selected_target=args.target, left_query=args.left_query, left_query_file=args.left_query_file, + left_label=args.left_label, right_query=args.right_query, right_query_file=args.right_query_file, + right_label=args.right_label, unique_key_override=tuple(args.key), unkeyed=args.unkeyed, excluded_columns_override=tuple(args.exclude_column), tolerance_overrides=tuple(args.tolerance), + max_value_length=args.max_value_length, + suppress_example_values=args.no_example_values, + full_example_values=args.full_example_values, ) ) if args.command == CliCommand.RECONCILE: diff --git a/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py b/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py index f37ffe9b2e..3e6b485ea2 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/dispatch_errors.py @@ -7,7 +7,7 @@ from sqlbuild.cli.commands._helpers.entry.errors import format_expected_error from sqlbuild.cli.commands._helpers.skills.update import maintain_sqlbuild_skills from sqlbuild.cli.commands.classes.cli_namespace import CliNamespace -from sqlbuild.cli.commands.exceptions import CliUserError +from sqlbuild.cli.commands.exceptions import CliUserError, QueryDiffExecutionError from sqlbuild.cli.commands.main.entrypoint._dispatch_with_observability import ( dispatch_with_observability, ) @@ -41,10 +41,22 @@ def dispatch_and_handle_errors( return error.code if isinstance(error.code, int) else 1 except (CliUserError, RulesError) as error: logging.getLogger("sqlbuild.cli").exception("cli user error") + effective_error: CliUserError | RulesError = error + if isinstance(error, CliUserError): + output_error: QueryDiffExecutionError | None = _write_diff_machine_error( + args=args, error=error + ) + if output_error is not None: + effective_error = output_error + outcome_status: object | None = getattr(effective_error, "status", None) + if outcome_status is not None: + print(f"Query diff outcome {outcome_status}", file=sys.stderr) print( - format_expected_error(error=error, fallback_code="C000", use_color=use_color), + format_expected_error(error=effective_error, fallback_code="C000", use_color=use_color), file=sys.stderr, ) + if isinstance(effective_error, CliUserError): + return int(getattr(effective_error, "exit_code", 1)) return 1 except LintError as error: logging.getLogger("sqlbuild.cli").exception("lint failed") @@ -76,6 +88,16 @@ def _report_skill_freshness(*, invocation: ParsedCliInvocation) -> None: CliCommand.SKILLS, }: return + if args.command == CliCommand.DIFF and any( + value is not None + for value in ( + args.left_query, + args.left_query_file, + args.right_query, + args.right_query_file, + ) + ): + return project_dir: Path = Path(args.project_dir) if args.project_dir is not None else Path.cwd() try: result: SkillMaintenanceResult = maintain_sqlbuild_skills(project_dir=project_dir) @@ -83,3 +105,41 @@ def _report_skill_freshness(*, invocation: ParsedCliInvocation) -> None: return if result.message: print(result.message, file=sys.stderr, end="") + + +def _write_diff_machine_error( + *, args: CliNamespace, error: CliUserError +) -> QueryDiffExecutionError | None: + if args.command != CliCommand.DIFF: + return None + json_stdout: bool = bool(getattr(args, "json", False)) + json_path: Path | None = getattr(args, "json_output", None) + if not json_stdout and json_path is None: + return None + from sqlbuild.cli.commands._helpers.diff.json_output import render_diff_error_json + + payload: str = render_diff_error_json( + status=str(getattr(error, "status", "execution_failed")), + code=error.code, + message=error.message, + ) + if json_path is not None: + try: + json_path.parent.mkdir(parents=True, exist_ok=True) + json_path.write_text(payload + "\n", encoding="utf-8") + except (OSError, UnicodeError) as output_error: + publication_error: QueryDiffExecutionError = QueryDiffExecutionError( + f"failed to publish query diff output: {output_error}", code="C257" + ) + if json_stdout: + print( + render_diff_error_json( + status=publication_error.status, + code=publication_error.code, + message=publication_error.message, + ) + ) + return publication_error + if json_stdout: + print(payload) + return None diff --git a/src/sqlbuild/cli/commands/_helpers/entry/parser.py b/src/sqlbuild/cli/commands/_helpers/entry/parser.py index 12294e40be..494781ac30 100644 --- a/src/sqlbuild/cli/commands/_helpers/entry/parser.py +++ b/src/sqlbuild/cli/commands/_helpers/entry/parser.py @@ -375,6 +375,10 @@ def _add_data_parsers( diff_parser.add_argument( "--right-query-file", type=Path, default=None, help="path to right-side SQL query" ) + diff_parser.add_argument("--left-label", default=None, help="display label for the left query") + diff_parser.add_argument( + "--right-label", default=None, help="display label for the right query" + ) diff_parser.add_argument( "--key", action="append", default=[], help="row identity column; repeat for composite keys" ) @@ -393,6 +397,30 @@ def _add_data_parsers( default=[], help="numeric tolerance as COLUMN:absolute=VALUE or COLUMN:relative=VALUE", ) + diff_parser.add_argument( + "--max-value-length", + type=int, + default=None, + help="maximum rendered characters per example value", + ) + diff_parser.add_argument( + "--no-example-values", + action="store_true", + default=False, + help="show example keys and counts without values", + ) + diff_parser.add_argument( + "--full-example-values", + action="store_true", + default=False, + help="explicitly render complete example values", + ) + diff_parser.add_argument( + "--json", + action="store_true", + default=False, + help="write structured diff output to stdout", + ) _ = add_execution_json_output_arg(diff_parser) _ = add_select_args(diff_parser) _ = add_vars_args(diff_parser) diff --git a/src/sqlbuild/cli/commands/classes/cli_namespace.py b/src/sqlbuild/cli/commands/classes/cli_namespace.py index 3e7dee920d..cd2257f7f4 100644 --- a/src/sqlbuild/cli/commands/classes/cli_namespace.py +++ b/src/sqlbuild/cli/commands/classes/cli_namespace.py @@ -112,10 +112,15 @@ "left_query_file": None, "right_query": None, "right_query_file": None, + "left_label": None, + "right_label": None, "key": [], "unkeyed": False, "exclude_column": [], "tolerance": [], + "max_value_length": None, + "no_example_values": False, + "full_example_values": False, "reconcile_command": None, "reconcile_model": None, "reconcile_physical_relation": None, @@ -277,12 +282,17 @@ class CliNamespace: allow_partial_diff: bool left_query: str | None left_query_file: Path | None + left_label: str | None right_query: str | None right_query_file: Path | None + right_label: str | None key: list[str] unkeyed: bool exclude_column: list[str] tolerance: list[str] + max_value_length: int | None + no_example_values: bool + full_example_values: bool reconcile_command: str | None reconcile_model: str | None reconcile_physical_relation: str | None diff --git a/src/sqlbuild/cli/commands/constants.py b/src/sqlbuild/cli/commands/constants.py index c27ac51d19..0d44f9df82 100644 --- a/src/sqlbuild/cli/commands/constants.py +++ b/src/sqlbuild/cli/commands/constants.py @@ -97,3 +97,8 @@ READ_ONLY_QUERY_ROOT_KEYS: frozenset[str] = frozenset( {"except", "intersect", "select", "union", "values"} ) +DEFAULT_DIFF_MAX_VALUE_LENGTH: int = 160 +QUERY_DIFF_INCOMPLETE_PREPARATION_CODES: frozenset[str] = frozenset( + {"C229", "C230", "C231", "C232", "C233", "C234", "C235", "C240", "C244"} +) +QUERY_DIFF_INCOMPLETE_EXECUTION_CODES: frozenset[str] = frozenset({"C237", "C238", "C239"}) diff --git a/src/sqlbuild/cli/commands/exceptions.py b/src/sqlbuild/cli/commands/exceptions.py index e74283d03e..29f3d0277f 100644 --- a/src/sqlbuild/cli/commands/exceptions.py +++ b/src/sqlbuild/cli/commands/exceptions.py @@ -13,3 +13,49 @@ def __init__(self, message: str, *, code: str | None = None, help: str | None = self.message = message self.code = code if code is not None else self.code self.help = help + + +class QueryDiffOutcomeError(CliUserError): + """Expected query-diff failure with an explicit outcome and exit code.""" + + status: str + exit_code: int + + def __init__( + self, + message: str, + *, + status: str, + exit_code: int, + code: str | None = None, + help: str | None = None, + ) -> None: + super().__init__(message, code=code, help=help) + self.status = status + self.exit_code = exit_code + + +class QueryDiffIncompleteError(QueryDiffOutcomeError): + """Safety or preflight prevented complete value evidence.""" + + def __init__(self, message: str, *, code: str | None = None, help: str | None = None) -> None: + super().__init__( + message, + status="incomplete", + exit_code=2, + code=code, + help=help, + ) + + +class QueryDiffExecutionError(QueryDiffOutcomeError): + """Setup, execution, ownership publication, or cleanup failed.""" + + def __init__(self, message: str, *, code: str | None = None, help: str | None = None) -> None: + super().__init__( + message, + status="execution_failed", + exit_code=3, + code=code, + help=help, + ) diff --git a/src/sqlbuild/cli/commands/main/entrypoint/_dispatch_with_observability.py b/src/sqlbuild/cli/commands/main/entrypoint/_dispatch_with_observability.py index 3256fd9d92..167125b7c8 100644 --- a/src/sqlbuild/cli/commands/main/entrypoint/_dispatch_with_observability.py +++ b/src/sqlbuild/cli/commands/main/entrypoint/_dispatch_with_observability.py @@ -17,7 +17,7 @@ def dispatch_with_observability(*, args: CliNamespace, handlers: CliEntrypointHandlers) -> int: """Dispatch once while publishing canonical invocation and nested run facts.""" - if _creates_project(args=args): + if _creates_project(args=args) or _is_raw_query_diff(args=args): return dispatch_cli_command(args=args, handlers=handlers) project_dir: Path = Path(args.project_dir) if args.project_dir is not None else Path.cwd() command: str = "unknown" if args.command is None else str(args.command) @@ -61,3 +61,15 @@ def _creates_project(*, args: CliNamespace) -> bool: return args.command in {CliCommand.INIT, CliCommand.PLAYGROUND} or ( args.command == CliCommand.DBT and args.dbt_command == DBT_INIT_COMMAND ) + + +def _is_raw_query_diff(*, args: CliNamespace) -> bool: + return args.command == CliCommand.DIFF and any( + value is not None + for value in ( + args.left_query, + args.left_query_file, + args.right_query, + args.right_query_file, + ) + ) diff --git a/src/sqlbuild/cli/commands/main/inspection/_diff.py b/src/sqlbuild/cli/commands/main/inspection/_diff.py index 2b9cea5ccc..d8608b0155 100644 --- a/src/sqlbuild/cli/commands/main/inspection/_diff.py +++ b/src/sqlbuild/cli/commands/main/inspection/_diff.py @@ -2,6 +2,7 @@ from __future__ import annotations +from sqlbuild.adapter.contract.exceptions import AdapterUserError from sqlbuild.cli.commands._helpers.diff.execution import ( execute_direct_diff, execute_query_diff, @@ -19,46 +20,116 @@ write_direct_diff_output, write_virtual_diff_output, ) +from sqlbuild.cli.commands.constants import ( + QUERY_DIFF_INCOMPLETE_EXECUTION_CODES, + QUERY_DIFF_INCOMPLETE_PREPARATION_CODES, +) +from sqlbuild.cli.commands.exceptions import ( + CliUserError, + QueryDiffExecutionError, + QueryDiffIncompleteError, + QueryDiffOutcomeError, +) from sqlbuild.cli.commands.models import ( DiffCommandRequest, DiffInvocation, DirectDiffPreparation, QueryDiffPreparation, + QueryDiffRunOutcome, VirtualDiffPreparation, VirtualDiffRunOutcome, ) +from sqlbuild.errors.contracts.exceptions import ExecutorInputError +from sqlbuild.executor.diff.constants import ( + ROW_DIFF_KEY_ERROR_FRAGMENT, + ROW_DIFF_KEY_ERROR_PREFIX, +) from sqlbuild.executor.diff.models import DiffExecutionResult def run_diff(request: DiffCommandRequest) -> int: """Execute the diff command.""" - invocation: DiffInvocation = resolve_diff_invocation(request=request) - if is_query_diff_request(request=request): - query_preparation: QueryDiffPreparation = prepare_query_diff( - request=request, - invocation=invocation, - ) - query_result: DiffExecutionResult = execute_query_diff( - request=request, - preparation=query_preparation, - ) - write_direct_diff_output( - request=request, - preparation=DirectDiffPreparation( - from_target="left query", - to_target="right query", - adapter=query_preparation.adapter, - left_project=None, - right_project=None, - selected_names=("raw queries",), - connection_config=query_preparation.connection_config, - effective_max_column_examples=query_preparation.effective_max_column_examples, - effective_max_row_only_examples=query_preparation.effective_max_row_only_examples, - ), - result=query_result, - ) - return resolve_diff_exit_code(query_result) + query_mode: bool = is_query_diff_request(request=request) + try: + invocation: DiffInvocation = resolve_diff_invocation(request=request) + except QueryDiffOutcomeError: + raise + except Exception as error: + if query_mode: + raise QueryDiffExecutionError(str(error), code="C252") from error + raise + if query_mode: + try: + query_preparation: QueryDiffPreparation = prepare_query_diff( + request=request, + invocation=invocation, + ) + except CliUserError as error: + error_type: type[QueryDiffOutcomeError] = ( + QueryDiffIncompleteError + if error.code in QUERY_DIFF_INCOMPLETE_PREPARATION_CODES + else QueryDiffExecutionError + ) + raise error_type(error.message, code=error.code, help=error.help) from error + except Exception as error: + raise QueryDiffExecutionError(str(error), code="C253") from error + try: + query_outcome: QueryDiffRunOutcome = execute_query_diff( + request=request, + preparation=query_preparation, + ) + except QueryDiffOutcomeError: + raise + except ExecutorInputError as error: + raise QueryDiffIncompleteError( + str(error), + code=str(getattr(error, "code", "C254")), + help=getattr(error, "help", None), + ) from error + except CliUserError as error: + error_type = ( + QueryDiffIncompleteError + if error.code in QUERY_DIFF_INCOMPLETE_EXECUTION_CODES + else QueryDiffExecutionError + ) + raise error_type(error.message, code=error.code, help=error.help) from error + except AdapterUserError as error: + error_type = ( + QueryDiffIncompleteError + if _is_query_key_preflight_error(error=error) + else QueryDiffExecutionError + ) + raise error_type(error.message, code=error.code, help=error.help) from error + except Exception as error: + raise QueryDiffExecutionError(str(error), code="C255") from error + try: + write_direct_diff_output( + request=request, + preparation=DirectDiffPreparation( + from_target=query_preparation.left_label, + to_target=query_preparation.right_label, + adapter=query_preparation.adapter, + left_project=None, + right_project=None, + selected_names=( + f"{query_preparation.left_label} vs {query_preparation.right_label}", + ), + connection_config=query_preparation.connection_config, + effective_max_column_examples=query_preparation.effective_max_column_examples, + effective_max_row_only_examples=( + query_preparation.effective_max_row_only_examples + ), + ), + result=query_outcome.result, + phase_seconds=query_outcome.phase_seconds, + outcome=query_outcome.outcome, + ) + except Exception as error: + raise QueryDiffExecutionError( + f"failed to publish query diff output: {error}", code="C257" + ) from error + return query_outcome.exit_code if invocation.is_virtual_mode: virtual_preparation: VirtualDiffPreparation = prepare_virtual_diff( request=request, @@ -90,3 +161,9 @@ def run_diff(request: DiffCommandRequest) -> int: result=result, ) return resolve_diff_exit_code(result) + + +def _is_query_key_preflight_error(*, error: AdapterUserError) -> bool: + return error.message.startswith(ROW_DIFF_KEY_ERROR_PREFIX) and ( + ROW_DIFF_KEY_ERROR_FRAGMENT in error.message + ) diff --git a/src/sqlbuild/cli/commands/models.py b/src/sqlbuild/cli/commands/models.py index 865f42cd65..5187fe087b 100644 --- a/src/sqlbuild/cli/commands/models.py +++ b/src/sqlbuild/cli/commands/models.py @@ -681,6 +681,7 @@ class DiffCommandRequest: sample_rows: int | None = None sample_seed: int | None = None exhaustive: bool = False + json_output: bool = False json_output_path: Path | None = None max_models: int | None = None max_columns: int | None = None @@ -692,12 +693,17 @@ class DiffCommandRequest: selected_target: str | None = None left_query: str | None = None left_query_file: Path | None = None + left_label: str | None = None right_query: str | None = None right_query_file: Path | None = None + right_label: str | None = None unique_key_override: tuple[str, ...] = () unkeyed: bool = False excluded_columns_override: tuple[str, ...] = () tolerance_overrides: tuple[str, ...] = () + max_value_length: int | None = None + suppress_example_values: bool = False + full_example_values: bool = False @dataclass(frozen=True) @@ -732,6 +738,8 @@ class QueryDiffPreparation: connection_config: dict[str, object] left_sql: str right_sql: str + left_label: str + right_label: str selected_target: str | None database: str | None schema: str @@ -741,6 +749,35 @@ class QueryDiffPreparation: effective_max_row_only_examples: int +@dataclass(frozen=True) +class DiffExampleRenderOptions: + """Bounded example rendering controls independent of comparison execution.""" + + max_value_length: int | None + suppress_values: bool = False + + +@dataclass(frozen=True) +class RenderedDiffExampleValue: + """One safely bounded value plus explicit information-loss metadata.""" + + text: str | None + original_length: int + truncated: bool + suppressed: bool + first_difference: int | None + + +@dataclass(frozen=True) +class QueryDiffRunOutcome: + """Raw-query result plus measured lifecycle phase durations.""" + + result: DiffExecutionResult + phase_seconds: dict[str, float] + outcome: str + exit_code: int + + @dataclass(frozen=True) class VirtualDiffPreparation: """Resolved virtual diff adapter, connection, reporters, and sample limits.""" diff --git a/src/sqlbuild/compiler/discovery/main/discover_configuration.py b/src/sqlbuild/compiler/discovery/main/discover_configuration.py new file mode 100644 index 0000000000..5dbb1abac8 --- /dev/null +++ b/src/sqlbuild/compiler/discovery/main/discover_configuration.py @@ -0,0 +1,20 @@ +"""Lightweight project configuration discovery entrypoint.""" + +from __future__ import annotations + +from pathlib import Path + +from sqlbuild.compiler.discovery._helpers.yml.project import load_local_config, load_project_config +from sqlbuild.compiler.discovery.models import DiscoveredProjectInputs +from sqlbuild.runtime.observability.classes.operation_lifecycle import OperationLifecycle + + +def discover_project_configuration(*, project_dir: Path) -> DiscoveredProjectInputs: + """Load project and local configuration without importing project extensions.""" + + with OperationLifecycle(operation_kind="project", operation_name="project_discovery"): + return DiscoveredProjectInputs( + project_dir=project_dir, + project_config=load_project_config(project_dir=project_dir), + local_config=load_local_config(project_dir=project_dir), + ) diff --git a/src/sqlbuild/executor/diff/_helpers/execution.py b/src/sqlbuild/executor/diff/_helpers/execution.py index bba65814b8..4ed4af7c97 100644 --- a/src/sqlbuild/executor/diff/_helpers/execution.py +++ b/src/sqlbuild/executor/diff/_helpers/execution.py @@ -135,7 +135,7 @@ def execute_query_diff( code="X305", ) result: ModelDiffResult = ModelDiffResult( - name="raw queries", + name=options.comparison_name or "query comparison", left_relation=left_relation, right_relation=right_relation, unique_key=unique_key, diff --git a/src/sqlbuild/executor/diff/constants.py b/src/sqlbuild/executor/diff/constants.py index 5387092604..320cbd10ed 100644 --- a/src/sqlbuild/executor/diff/constants.py +++ b/src/sqlbuild/executor/diff/constants.py @@ -4,6 +4,8 @@ ROW_DIFF_RELATIVE_TOLERANCE_KEY: str = "relative" DIFF_INPUT_KIND_MODEL: str = "model" DIFF_INPUT_KIND_QUERY: str = "query" +ROW_DIFF_KEY_ERROR_PREFIX: str = "row diff " +ROW_DIFF_KEY_ERROR_FRAGMENT: str = " unique_key values" ROW_DIFF_TOLERANCE_KEYS: frozenset[str] = frozenset( (ROW_DIFF_ABSOLUTE_TOLERANCE_KEY, ROW_DIFF_RELATIVE_TOLERANCE_KEY) ) diff --git a/src/sqlbuild/executor/diff/models.py b/src/sqlbuild/executor/diff/models.py index aa6c80b7d3..4f158ac1aa 100644 --- a/src/sqlbuild/executor/diff/models.py +++ b/src/sqlbuild/executor/diff/models.py @@ -68,6 +68,7 @@ class DiffExecutionOptions: unkeyed: bool = False excluded_columns_override: tuple[str, ...] = () tolerance_overrides: RowDiffTolerances | None = None + comparison_name: str | None = None @dataclass(frozen=True) diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py b/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py index 7f1a24082d..20ef106b9a 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/diff/_test_types.py @@ -12,6 +12,7 @@ class DiffCommandE2ETestCase: expected_exit_code: int expected_stdout_fragments: tuple[str, ...] = field(default_factory=tuple) expected_stderr_fragments: tuple[str, ...] = field(default_factory=tuple) + unexpected_stdout_fragments: tuple[str, ...] = field(default_factory=tuple) mutation_sql: tuple[str, ...] = field(default_factory=tuple) @@ -24,6 +25,18 @@ class SingleDiffE2ETestCase: expected_fragment: str +@dataclass(frozen=True) +class QueryDiffOutcomeE2ETestCase: + """One structured query-diff non-pass outcome.""" + + description: str + left_query: str + right_query: str + expected_exit_code: int + expected_status: str + expected_error_fragment: str + + @dataclass(frozen=True) class DiffKeyFailureE2ETestCase: description: str diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/diff/helpers.py b/tests/e2e/src/sqlbuild/cli/commands/main/diff/helpers.py index c3a61e6e09..b973ae1183 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/diff/helpers.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/diff/helpers.py @@ -10,6 +10,12 @@ from tests.e2e.src.sqlbuild.cli.commands.shared.helpers import prepare_inline_project, run_sqb +def reject_nonfinite_json_constant(value: str) -> object: + """Fail strict JSON parsing when a non-standard numeric token appears.""" + + raise AssertionError(f"non-standard JSON numeric constant: {value}") + + def prepare_diff_project(tmp_path: Path) -> Path: """Create a small inline project for diff e2e tests.""" diff --git a/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_diff.py b/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_diff.py index 234c6ea1bd..89f56ffe4b 100644 --- a/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_diff.py +++ b/tests/e2e/src/sqlbuild/cli/commands/main/diff/test_diff.py @@ -16,6 +16,7 @@ DiffKeyFailureE2ETestCase, DiffSamplingPrecedenceE2ETestCase, DiffSamplingSeedE2ETestCase, + QueryDiffOutcomeE2ETestCase, SingleDiffE2ETestCase, VirtualDiffE2ETestCase, ) @@ -24,6 +25,7 @@ changed_key_examples, execute_duckdb, prepare_diff_project, + reject_nonfinite_json_constant, run_sampled_diff, ) from tests.e2e.src.sqlbuild.cli.commands.main.plan.helpers import ( @@ -591,8 +593,12 @@ def test_given_diff_project_when_running_diff_then_behavior_matches_expected( "--key", "order_id", ), - expected_exit_code=1, - expected_stdout_fragments=("schema differences: 1", "added columns: 1"), + expected_exit_code=2, + expected_stdout_fragments=( + "outcome: incomplete", + "schema differences: 1", + "added columns: 1", + ), ), DiffCommandE2ETestCase( description="query tolerance suppresses numeric differences", @@ -676,9 +682,68 @@ def test_given_diff_project_when_running_diff_then_behavior_matches_expected( "--key", "order_id", ), - expected_exit_code=1, + expected_exit_code=2, expected_stderr_fragments=("contains duplicate unique_key values",), ), + DiffCommandE2ETestCase( + description="query labels and diff aware truncation keep long evidence compact", + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 1 AS order_id, '" + ("a" * 200) + "left-tail' AS note", + "--right-query", + "SELECT 1 AS order_id, '" + ("a" * 200) + "right-tail' AS note", + "--left-label", + "baseline", + "--right-label", + "candidate", + "--key", + "order_id", + ), + expected_exit_code=1, + expected_stdout_fragments=( + "baseline vs candidate", + "baseline count", + "candidate count", + "[truncated; 209 chars]", + "first difference at character 201", + ), + ), + DiffCommandE2ETestCase( + description="query examples can suppress values without suppressing keys", + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 1 AS order_id, 'left-private-value' AS note", + "--right-query", + "SELECT 1 AS order_id, 'right-private-value' AS note", + "--key", + "order_id", + "--no-example-values", + ), + expected_exit_code=1, + expected_stdout_fragments=("order_id=1", ""), + unexpected_stdout_fragments=("left-private-value", "right-private-value"), + ), + DiffCommandE2ETestCase( + description="complete example values require explicit opt in", + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 1 AS order_id, '" + ("a" * 200) + "left-tail' AS note", + "--right-query", + "SELECT 1 AS order_id, '" + ("a" * 200) + "right-tail' AS note", + "--key", + "order_id", + "--full-example-values", + ), + expected_exit_code=1, + expected_stdout_fragments=("left-tail", "right-tail"), + unexpected_stdout_fragments=("[truncated;",), + ), ], ids=lambda case: case.description, ) @@ -701,6 +766,8 @@ def test_given_raw_queries_when_running_diff_then_comparison_matches_model_diff_ assert fragment in result.stdout, result.stdout + result.stderr for fragment in test_case.expected_stderr_fragments: assert fragment in result.stderr, result.stdout + result.stderr + for fragment in test_case.unexpected_stdout_fragments: + assert fragment not in result.stdout, result.stdout + result.stderr connection: duckdb.DuckDBPyConnection = duckdb.connect(str(project_dir / "diff.duckdb")) try: artifact_row: tuple[int] | None = connection.execute( @@ -787,7 +854,7 @@ def test_given_query_files_target_and_json_output_when_running_diff_then_all_inp [ SingleDiffE2ETestCase( description="mutating CTE rejection", - expected_exit_code=1, + expected_exit_code=2, expected_fragment="must not contain data-changing or administrative statements", ) ], @@ -822,7 +889,7 @@ def test_given_mutating_cte_when_running_query_diff_then_validation_rejects_it( [ SingleDiffE2ETestCase( description="duplicate query column rejection", - expected_exit_code=1, + expected_exit_code=2, expected_fragment="left query returns duplicate column names: order_id", ) ], @@ -903,6 +970,209 @@ def test_given_unreadable_historical_fingerprint_when_running_query_diff_then_co assert "No changed columns." in result.stdout +@pytest.mark.parametrize( + "test_case", + [ + SingleDiffE2ETestCase( + description="query JSON stdout", expected_exit_code=1, expected_fragment="findings" + ) + ], + ids=lambda case: case.description, +) +def test_given_json_stdout_when_running_labeled_query_diff_then_stdout_is_one_query_native_document( + test_case: SingleDiffE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_diff_project(tmp_path) + build_both_environments(project_dir=project_dir) + + result: subprocess.CompletedProcess[str] = run_sqb( + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 'NaN'::DOUBLE AS order_id, '" + ("a" * 30) + "left-tail' AS note", + "--right-query", + "SELECT 'NaN'::DOUBLE AS order_id, '" + ("a" * 30) + "right-tail' AS note", + "--left-label", + "baseline", + "--right-label", + "candidate", + "--key", + "order_id", + "--max-value-length", + "12", + "--json", + ), + project_dir=project_dir, + ) + + assert result.returncode == test_case.expected_exit_code, result.stdout + result.stderr + payload: dict[str, object] = cast( + dict[str, object], + json.loads(result.stdout, parse_constant=reject_nonfinite_json_constant), + ) + assert payload["outcome"] == test_case.expected_fragment + assert payload["from"] == "baseline" + assert payload["to"] == "candidate" + assert payload["input_kind"] == "query" + comparisons: list[dict[str, object]] = cast( + list[dict[str, object]], payload["query_comparisons"] + ) + rows: dict[str, object] = cast(dict[str, object], comparisons[0]["rows"]) + assert rows["column_mismatches"] == [{"column": "note", "rows": 1}] + examples: dict[str, object] = cast(dict[str, object], comparisons[0]["examples"]) + assert examples["values_truncated"] is True + assert examples["max_value_length"] == 12 + unequal_rows: list[dict[str, object]] = cast(list[dict[str, object]], examples["unequal_rows"]) + example_key: dict[str, object] = cast(dict[str, object], unequal_rows[0]["key"]) + assert example_key["order_id"] == {"kind": "non_finite_number", "value": "NaN"} + assert "phase_seconds" in payload + assert "Query diff total OK" in result.stderr + assert "SQLBuild Diff Summary" not in result.stdout + + +@pytest.mark.parametrize( + "test_case", + [ + SingleDiffE2ETestCase( + description="query JSON output publication failure", + expected_exit_code=3, + expected_fragment="failed to publish query diff output", + ) + ], + ids=lambda case: case.description, +) +def test_given_unwritable_json_path_when_running_query_diff_then_stdout_reports_execution_failure( + test_case: SingleDiffE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_diff_project(tmp_path) + blocked_parent: Path = project_dir / "blocked-output" + blocked_parent.write_text("not a directory\n", encoding="utf-8") + + result: subprocess.CompletedProcess[str] = run_sqb( + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 1 AS order_id", + "--right-query", + "SELECT 1 AS order_id", + "--key", + "order_id", + "--json", + "--json-output", + str(blocked_parent / "result.json"), + ), + project_dir=project_dir, + ) + + assert result.returncode == test_case.expected_exit_code, result.stdout + result.stderr + payload: dict[str, object] = cast( + dict[str, object], + json.loads(result.stdout, parse_constant=reject_nonfinite_json_constant), + ) + assert payload["outcome"] == "execution_failed" + error: dict[str, object] = cast(dict[str, object], payload["error"]) + assert test_case.expected_fragment in str(error["message"]) + assert "Query diff outcome execution_failed" in result.stderr + assert "Traceback" not in result.stderr + + +@pytest.mark.parametrize( + "test_case", + [ + SingleDiffE2ETestCase( + description="lightweight raw query discovery", + expected_exit_code=0, + expected_fragment="No changed columns.", + ) + ], + ids=lambda case: case.description, +) +def test_given_unimportable_project_extension_when_running_raw_query_diff_then_it_is_not_loaded( + test_case: SingleDiffE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_diff_project(tmp_path) + build_both_environments(project_dir=project_dir) + sink_path: Path = project_dir / "sinks" / "broken_sink.py" + sink_path.parent.mkdir(parents=True, exist_ok=True) + sink_path.write_text("import package_that_is_not_installed\n", encoding="utf-8") + + result: subprocess.CompletedProcess[str] = run_sqb( + command=( + "--no-color", + "diff", + "--left-query", + "SELECT 1 AS order_id", + "--right-query", + "SELECT 1 AS order_id", + "--key", + "order_id", + ), + project_dir=project_dir, + ) + + assert result.returncode == test_case.expected_exit_code, result.stdout + result.stderr + assert test_case.expected_fragment in result.stdout + assert "package_that_is_not_installed" not in result.stderr + + +@pytest.mark.parametrize( + "test_case", + [ + QueryDiffOutcomeE2ETestCase( + description="duplicate keys are incomplete evidence", + left_query="SELECT * FROM (VALUES (1), (1)) AS orders(order_id)", + right_query="SELECT * FROM (VALUES (1)) AS orders(order_id)", + expected_exit_code=2, + expected_status="incomplete", + expected_error_fragment="duplicate unique_key values", + ), + QueryDiffOutcomeE2ETestCase( + description="warehouse query failures are execution failures", + left_query="SELECT order_id FROM missing_orders", + right_query="SELECT 1 AS order_id", + expected_exit_code=3, + expected_status="execution_failed", + expected_error_fragment="missing_orders", + ), + ], + ids=lambda case: case.description, +) +def test_given_query_diff_cannot_complete_when_requesting_json_then_outcome_is_machine_readable( + test_case: QueryDiffOutcomeE2ETestCase, + tmp_path: Path, +) -> None: + project_dir: Path = prepare_diff_project(tmp_path) + build_both_environments(project_dir=project_dir) + + result: subprocess.CompletedProcess[str] = run_sqb( + command=( + "--no-color", + "diff", + "--left-query", + test_case.left_query, + "--right-query", + test_case.right_query, + "--key", + "order_id", + "--json", + ), + project_dir=project_dir, + ) + + assert result.returncode == test_case.expected_exit_code, result.stdout + result.stderr + payload: dict[str, object] = cast(dict[str, object], json.loads(result.stdout)) + assert payload["status"] == test_case.expected_status + assert payload["outcome"] == test_case.expected_status + error: dict[str, object] = cast(dict[str, object], payload["error"]) + assert test_case.expected_error_fragment.lower() in str(error["message"]).lower() + assert f"Query diff outcome {test_case.expected_status}" in result.stderr + + @pytest.mark.parametrize( "test_case", [ diff --git a/tests/integration/src/sqlbuild/executor/diff/_test_types.py b/tests/integration/src/sqlbuild/executor/diff/_test_types.py index 944e1cd4ad..f6edf4088e 100644 --- a/tests/integration/src/sqlbuild/executor/diff/_test_types.py +++ b/tests/integration/src/sqlbuild/executor/diff/_test_types.py @@ -9,3 +9,13 @@ class QueryArtifactCleanupTestCase: description: str expected_fingerprint_count: int + + +@dataclass(frozen=True) +class QueryArtifactCleanupFailureTestCase: + """One mandatory current-run cleanup failure case.""" + + description: str + run_id: str + expected_error: str + expected_artifact_count: int diff --git a/tests/integration/src/sqlbuild/executor/diff/test_query_artifact_cleanup.py b/tests/integration/src/sqlbuild/executor/diff/test_query_artifact_cleanup.py index 8a57f16b61..d686ba683f 100644 --- a/tests/integration/src/sqlbuild/executor/diff/test_query_artifact_cleanup.py +++ b/tests/integration/src/sqlbuild/executor/diff/test_query_artifact_cleanup.py @@ -3,11 +3,17 @@ from __future__ import annotations from datetime import UTC, datetime, timedelta +from pathlib import Path +from typing import Any import duckdb import pytest +from sqlbuild.adapter.contract.classes.statement_recorder import StatementRecorder +from sqlbuild.adapter.contract.exceptions import AdapterUserError from sqlbuild.adapters.duckdb.classes.duckdb_adapter import DuckDbAdapter +from sqlbuild.cli.commands._helpers.diff.execution import execute_query_diff +from sqlbuild.cli.commands.models import DiffCommandRequest, QueryDiffPreparation from sqlbuild.compiler.compile.models import CompiledProject from sqlbuild.compiler.fingerprints.constants import NODE_TYPE_QUERY_DIFF_ARTIFACT from sqlbuild.executor.diff.classes.query_artifact_lifecycle import QueryDiffArtifactLifecycle @@ -20,10 +26,24 @@ JanitorPlan, ) from tests.integration.src.sqlbuild.executor.diff._test_types import ( + QueryArtifactCleanupFailureTestCase, QueryArtifactCleanupTestCase, ) +class _CleanupFailingDuckDbAdapter(DuckDbAdapter): + def drop( + self, + *, + connection: Any, + destination: str, + if_exists: bool = True, + statement_recorder: StatementRecorder, + ) -> None: + del connection, destination, if_exists, statement_recorder + raise AdapterUserError("simulated query artifact cleanup failure") + + @pytest.mark.parametrize( "test_case", [ @@ -117,3 +137,61 @@ def test_given_expired_owned_and_untracked_artifacts_when_janitor_runs_then_only assert fingerprint_row == (test_case.expected_fingerprint_count,) finally: connection.close() + + +@pytest.mark.parametrize( + "test_case", + [ + QueryArtifactCleanupFailureTestCase( + description="current-run artifacts remain after mandatory cleanup fails", + run_id="20260918T142303Z_c1d2e3f4a5b6", + expected_error="simulated query artifact cleanup failure", + expected_artifact_count=2, + ) + ], + ids=lambda case: case.description, +) +def test_given_current_artifact_cleanup_failure_when_executing_query_diff_then_failure_is_mandatory( + test_case: QueryArtifactCleanupFailureTestCase, + tmp_path: Path, +) -> None: + database_path: Path = tmp_path / "query_diff.duckdb" + adapter: DuckDbAdapter = _CleanupFailingDuckDbAdapter() + request: DiffCommandRequest = DiffCommandRequest( + project_dir=None, + no_color=True, + no_sql_validation=False, + from_name=None, + to_name=None, + full=False, + schema_only=False, + bounded=None, + unique_key_override=("order_id",), + ) + preparation: QueryDiffPreparation = QueryDiffPreparation( + adapter=adapter, + connection_config={"database": str(database_path)}, + left_sql="SELECT 1 AS order_id", + right_sql="SELECT 1 AS order_id", + left_label="baseline", + right_label="candidate", + selected_target="dev", + database=None, + schema="main", + run_id=test_case.run_id, + artifact_ttl="24h", + effective_max_column_examples=3, + effective_max_row_only_examples=3, + ) + + with pytest.raises(AdapterUserError, match=test_case.expected_error): + execute_query_diff(request=request, preparation=preparation) + + connection: duckdb.DuckDBPyConnection = duckdb.connect(str(database_path)) + try: + artifact_count_row: tuple[int] | None = connection.execute( + "SELECT COUNT(*) FROM duckdb_tables() WHERE table_name LIKE '__sqlbuild_query_diff_%'" + ).fetchone() + assert artifact_count_row == (test_case.expected_artifact_count,) + finally: + connection.close() diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py index 5e85779b4a..91a2c73400 100644 --- a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/_test_types.py @@ -1,5 +1,6 @@ from dataclasses import dataclass +from sqlbuild.cli.commands.models import DiffExampleRenderOptions from sqlbuild.executor.diff.models import DiffExecutionResult @@ -30,3 +31,16 @@ class RenderVirtualDiffHeaderTestCase: expected_fragments: tuple[str, ...] expected_color_fragments: tuple[str, ...] = () unexpected_fragments: tuple[str, ...] = () + + +@dataclass(frozen=True) +class RenderDiffEvidenceTestCase: + description: str + left_value: object + right_value: object + options: DiffExampleRenderOptions + expected_left_fragment: str + expected_right_fragment: str + expected_first_difference: int | None + expected_truncated: bool + expected_suppressed: bool diff --git a/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_evidence.py b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_evidence.py new file mode 100644 index 0000000000..c4de2b5e67 --- /dev/null +++ b/tests/unit/src/sqlbuild/cli/commands/_helpers/diff/test_evidence.py @@ -0,0 +1,74 @@ +"""Tests for bounded diff example evidence.""" + +from __future__ import annotations + +import pytest + +from sqlbuild.cli.commands._helpers.diff.evidence import ( + display_example_value, + render_example_pair, +) +from sqlbuild.cli.commands.models import DiffExampleRenderOptions, RenderedDiffExampleValue +from tests.unit.src.sqlbuild.cli.commands._helpers.diff._test_types import ( + RenderDiffEvidenceTestCase, +) + + +@pytest.mark.parametrize( + "test_case", + [ + RenderDiffEvidenceTestCase( + description="late text difference keeps nearby context", + left_value=("a" * 200) + "left-tail", + right_value=("a" * 200) + "right-tail", + options=DiffExampleRenderOptions(max_value_length=30), + expected_left_fragment="left-tail", + expected_right_fragment="right-tail", + expected_first_difference=200, + expected_truncated=True, + expected_suppressed=False, + ), + RenderDiffEvidenceTestCase( + description="suppression retains lengths without values", + left_value="left-private-value", + right_value="right-private-value", + options=DiffExampleRenderOptions(max_value_length=30, suppress_values=True), + expected_left_fragment="18 chars", + expected_right_fragment="19 chars", + expected_first_difference=0, + expected_truncated=False, + expected_suppressed=True, + ), + RenderDiffEvidenceTestCase( + description="explicit unlimited rendering preserves complete values", + left_value=("a" * 200) + "left-tail", + right_value=("a" * 200) + "right-tail", + options=DiffExampleRenderOptions(max_value_length=None), + expected_left_fragment=("a" * 200) + "left-tail", + expected_right_fragment=("a" * 200) + "right-tail", + expected_first_difference=200, + expected_truncated=False, + expected_suppressed=False, + ), + ], + ids=lambda case: case.description, +) +def test_given_unequal_values_when_rendering_evidence_then_bounds_are_explicit( + test_case: RenderDiffEvidenceTestCase, +) -> None: + left: RenderedDiffExampleValue + right: RenderedDiffExampleValue + left, right = render_example_pair( + left_value=test_case.left_value, + right_value=test_case.right_value, + options=test_case.options, + ) + + assert test_case.expected_left_fragment in display_example_value(left) + assert test_case.expected_right_fragment in display_example_value(right) + assert left.first_difference == test_case.expected_first_difference + assert right.first_difference == test_case.expected_first_difference + assert left.truncated is test_case.expected_truncated + assert right.truncated is test_case.expected_truncated + assert left.suppressed is test_case.expected_suppressed + assert right.suppressed is test_case.expected_suppressed