Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 47 additions & 3 deletions docs/query-diffs.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ from a UTF-8 file, and choose keyed or unkeyed comparison explicitly:
sqb diff \
--left-query 'SELECT order_id, amount_cents FROM analytics.orders_before' \
--right-query 'SELECT order_id, amount_cents FROM analytics.orders_after' \
--left-label before \
--right-label after \
--key order_id
```

Expand All @@ -56,6 +58,9 @@ Query mode performs a full comparison when no comparison mode flag is present. `
also available. `--bounded` is model-specific; filter both raw queries directly when a bounded
query comparison is needed.

Raw-query discovery loads only project configuration, target configuration, and the selected
adapter. It does not import unrelated project sinks, hooks, loaders, or providers.

### Sampling, tolerances, and exclusions

Keyed query comparisons support deterministic union-key sampling and the same bounded mismatch
Expand Down Expand Up @@ -85,6 +90,31 @@ sqb diff \
`--exhaustive` disables inherited or requested sampling. Unkeyed comparison is always exhaustive
and exact, so it does not accept sampling or numeric tolerances.

### Bounded example values

Comparison remains exact regardless of how examples are rendered. By default, long values are
limited to 160 characters and show context around the first differing character. Every shortened
value is marked as truncated and includes its original length.

Control evidence independently from the number of collected examples:

```bash
# Increase the displayed context without changing comparison scope.
sqb diff --left-query-file before.sql --right-query-file after.sql \
--key order_id --max-value-length 500

# Retain keys and aggregate counts but suppress example values.
sqb diff --left-query-file before.sql --right-query-file after.sql \
--key order_id --no-example-values

# Explicitly opt in to complete values. --verbose alone remains bounded.
sqb diff --left-query-file before.sql --right-query-file after.sql \
--key order_id --full-example-values
```

Ordinary keys are kept complete. Exceptionally long keys are shortened with an explicit length and
stable digest so an operator can still identify them without flooding terminal output.

## Stable query results and cleanup

SQLBuild materializes each raw query once in the selected target's default schema before comparing
Expand Down Expand Up @@ -115,6 +145,20 @@ error.

## Structured output and exit behavior

Use `--json-output PATH` in either mode. Query results include `input_kind = "query"`; model results
use `input_kind = "model"`. Exit code `0` means no differences were found. Exit code `1` means
differences were found or the comparison could not execute successfully.
Use `--json-output PATH` in either mode, or `--json` to emit one JSON document to stdout while all
lifecycle progress remains on stderr. Query results include `input_kind = "query"`, named left and
right inputs, `query_comparisons`, sparse nonzero `column_mismatches`, lifecycle timings, and
explicit example truncation metadata. Existing `models` output remains available for compatibility.

Raw-query automation distinguishes complete findings from incomplete evidence and execution
failure:

| Exit | Outcome |
|---:|---|
| `0` | Complete comparison with no differences |
| `1` | Complete comparison with findings |
| `2` | Safety, schema, or key preflight prevented complete value evidence |
| `3` | Setup, connection, materialization, comparison execution, ownership, or cleanup failed |

Human output and JSON use the same outcome. Lifecycle messages include elapsed connection,
reconciliation, inspection, materialization, comparison, cleanup, and total durations.
54 changes: 48 additions & 6 deletions src/sqlbuild/.agents/skills/sqlbuild/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -7780,6 +7780,8 @@ SQLBuild templates such as `ref()` are not expanded.
sqb diff \
--left-query 'SELECT order_id, amount_cents FROM analytics.orders_before' \
--right-query 'SELECT order_id, amount_cents FROM analytics.orders_after' \
--left-label before \
--right-label after \
--key order_id

sqb diff \
Expand All @@ -7793,6 +7795,9 @@ sqb diff \
Query mode defaults to a full comparison and also supports `--schema-only`. It does not accept
`--bounded`; put the desired filter in both SQL inputs instead.

Raw-query mode loads configuration and the selected adapter without importing unrelated project
sinks, hooks, loaders, or providers.

### Comparison modes

Model mode requires exactly one mode. Raw-query mode defaults to full when no mode is supplied:
Expand Down Expand Up @@ -7979,19 +7984,47 @@ sqb diff prod:dev --full --select customer_status_snapshot --max-column-examples
Example limits only control diagnostic values printed after comparison. They are separate from
`row_diff_sample_rows`, which controls how many unique keys receive the wide value comparison.

Long values default to 160 rendered characters. SQLBuild shows context around the first difference,
marks truncation, and reports the original length without changing exact comparison counts. Control
value evidence independently from example counts:

```bash
sqb diff \
--left-query-file queries/orders_before.sql \
--right-query-file queries/orders_after.sql \
--key order_id \
--max-value-length 500

sqb diff \
--left-query-file queries/orders_before.sql \
--right-query-file queries/orders_after.sql \
--key order_id \
--no-example-values

sqb diff \
--left-query-file queries/orders_before.sql \
--right-query-file queries/orders_after.sql \
--key order_id \
--full-example-values
```

`--verbose` increases the example count but does not remove the value-length bound.

### Structured output

Use `--json-output PATH` to write a stable structured result while retaining the normal terminal
summary:
summary, or `--json` to write one JSON document to stdout with progress on stderr:

```bash
sqb diff prod:dev --bounded 14d --sample-rows 50000 --select order_lines --json-output diff.json
sqb diff --left-query-file before.sql --right-query-file after.sql --key order_id --json
```

Each model records `schema_only`, `exhaustive`, or `sampled` comparison scope; requested and observed
cursor coverage; bounded population and compared key counts; seed and configured limit; row result
counts; and changed-column counts. The top-level status distinguishes `no_differences_found` from
`differences_found`.
counts; and changed-column counts. Query output additionally includes named inputs,
`query_comparisons`, sparse nonzero `column_mismatches`, lifecycle timings, and explicit evidence
truncation metadata. Existing `models` output remains for compatibility.

### Invocation safety limits

Expand Down Expand Up @@ -8022,9 +8055,10 @@ sqb diff prod:dev --full --select tag:acceptance

### Exit codes

`sqb diff` returns exit code `0` when no differences are found, and `1` when any input has schema or
row differences or the comparison cannot execute successfully. This makes it usable in CI pipelines
as a validation gate.
Complete comparisons return `0` when no differences are found and `1` for findings. Raw-query mode
returns `2` when safety, schema, or key preflight prevents complete value evidence, and `3` for
setup, connection, materialization, comparison execution, ownership publication, or cleanup
failure. Human and JSON output carry the same outcome.

## Execution Observability

Expand Down Expand Up @@ -14541,17 +14575,23 @@ and falls back to a full row comparison when no cursor is configured.
| `--sample-seed` | Override the deterministic sample seed |
| `--exhaustive` | Disable inherited sampling for this invocation |
| `--json-output` | Write structured comparison scope, coverage, and results to a JSON file |
| `--json` | Write one structured JSON document to stdout; progress remains on stderr |
| `--max-models` | Fail when the selected scope contains more models than this limit |
| `--max-columns` | Fail when either side of a model has more columns than this limit |
| `--left-query` | Literal read-only SQL for the left side |
| `--left-query-file` | UTF-8 SQL file for the left side |
| `--right-query` | Literal read-only SQL for the right side |
| `--right-query-file` | UTF-8 SQL file for the right side |
| `--left-label` | Display and structured-output label for the left query |
| `--right-label` | Display and structured-output label for the right query |
| `--target` | Target connection for raw-query mode; defaults to the active target |
| `--key` | Row identity column; repeat for a composite key or to override model configuration |
| `--unkeyed` | Compare exact full-row multiplicities without a key |
| `--exclude-column` | Column to omit from row comparison; repeat as needed |
| `--tolerance` | Per-column numeric rule such as `amount:absolute=0.01`; repeat as needed |
| `--max-value-length` | Maximum rendered characters per example value (default: 160) |
| `--no-example-values` | Retain example keys and counts while suppressing values |
| `--full-example-values` | Explicitly opt in to complete example values |
| `--no-sql-analysis` | Disable compile-time SQL analysis (`--no-sql-validation` is an alias) |
| `--select`, `-s` | Select specific models to diff (required in v1) |
| `--exclude` | Exclude specific models from diffing |
Expand All @@ -14578,6 +14618,8 @@ sqb diff prod:dev --bounded 14d --exhaustive --select order_lines
sqb diff \
--left-query-file queries/orders_before.sql \
--right-query-file queries/orders_after.sql \
--left-label before \
--right-label after \
--key account_id \
--key order_id
```
Expand Down
108 changes: 108 additions & 0 deletions src/sqlbuild/cli/commands/_helpers/diff/evidence.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
"""Bounded, diff-aware rendering for row-diff evidence."""

from __future__ import annotations

import hashlib

from sqlbuild.cli.commands.models import DiffExampleRenderOptions, RenderedDiffExampleValue

_MAX_RENDERED_KEY_LENGTH: int = 256


def render_example_pair(
*,
left_value: object,
right_value: object,
options: DiffExampleRenderOptions,
) -> tuple[RenderedDiffExampleValue, RenderedDiffExampleValue]:
"""Render two unequal values around their first differing character."""

left_text: str = str(left_value)
right_text: str = str(right_value)
first_difference: int | None = _first_difference(left=left_text, right=right_text)
return (
_render_value(text=left_text, first_difference=first_difference, options=options),
_render_value(text=right_text, first_difference=first_difference, options=options),
)


def display_example_value(value: RenderedDiffExampleValue) -> str:
"""Render one bounded value for terminal output."""

if value.suppressed:
return f"<value suppressed; {value.original_length:,} chars>"
rendered: str = value.text or ""
if value.truncated:
return f"{rendered} [truncated; {value.original_length:,} chars]"
return rendered


def render_key_value(value: object) -> str:
"""Keep ordinary keys complete and identify exceptionally long shortened keys."""

text: str = str(value)
if len(text) <= _MAX_RENDERED_KEY_LENGTH:
return text
digest: str = hashlib.sha256(text.encode("utf-8")).hexdigest()[:12]
return f"{text[:_MAX_RENDERED_KEY_LENGTH]}… [truncated; {len(text):,} chars; sha256:{digest}]"


def example_value_payload(value: RenderedDiffExampleValue) -> dict[str, object]:
"""Build stable structured evidence for one rendered value."""

return {
"value": value.text,
"original_length": value.original_length,
"truncated": value.truncated,
"suppressed": value.suppressed,
"first_difference": value.first_difference,
}


def _render_value(
*,
text: str,
first_difference: int | None,
options: DiffExampleRenderOptions,
) -> RenderedDiffExampleValue:
original_length: int = len(text)
if options.suppress_values:
return RenderedDiffExampleValue(
text=None,
original_length=original_length,
truncated=False,
suppressed=True,
first_difference=first_difference,
)
limit: int | None = options.max_value_length
if limit is None or original_length <= limit:
return RenderedDiffExampleValue(
text=text,
original_length=original_length,
truncated=False,
suppressed=False,
first_difference=first_difference,
)
difference: int = first_difference or 0
context_before: int = min(limit // 3, max(limit - 1, 0))
start: int = min(max(difference - context_before, 0), original_length - limit)
end: int = start + limit
fragment: str = text[start:end]
rendered: str = f"{'…' if start else ''}{fragment}{'…' if end < original_length else ''}"
return RenderedDiffExampleValue(
text=rendered,
original_length=original_length,
truncated=True,
suppressed=False,
first_difference=first_difference,
)


def _first_difference(*, left: str, right: str) -> int | None:
index: int
for index, (left_char, right_char) in enumerate(zip(left, right, strict=False)):
if left_char != right_char:
return index
if len(left) != len(right):
return min(len(left), len(right))
return None
Loading
Loading