Repository navigation
feat!: harden Recall, unify query output and fix macro slot qualification - #1263
Merged
Merged
Conversation
…rve targets A target whose schema or database is `preserve` names no location, but a standalone `sqb audit` created its result history table in a literal `preserve` schema. It now stores history where a build stores it, and a history write that fails prints a warning with the warehouse error before the audit summary.
`sqb query` and `sqb recall query` share one text renderer. By default it prints a table with each column's warehouse type when the result fits the terminal (or 120 characters when redirected) and one record per row otherwise, so no column or value is cut. The footer says how to see more rows and, above 50 rows, suggests aggregating or limiting the query. BREAKING CHANGE: `--format table` and `--format csv` are removed from `sqb query` and `sqb recall query`/`schema`; they fail with a message naming what to use instead. Use the default output, `--format long`, or `--json`.
…nstead The bundled skill tells agents to keep the default output for anything a person reads, to aggregate, select fewer columns or add a LIMIT when a result is too long, never to switch format to save space, and to say why when they do change it.
Recall records the selectors a command ran with, including those read from each `--select-file`, whose paths are kept in `selector_files`. `selection_count` counts them and `selection_truncated` marks a list cut at 64 KiB.
Each statement gets its own reference to its SQL text, and the SQL of a statement that fails or is cancelled is queued again at its terminal event's priority, so it survives a full queue. `statements.sql_available` checks the statement's own invocation, and `sql_unavailable_reason` says why SQL is missing.
Violation audits that warn or fail keep a bounded sample of the rows they returned, as measurement audits already did. Recall stores the evidence as an artifact named by the audit result's `sample_digest`, with `evidence_count` and `evidence_truncated` on `audit_results`. `[recall] audit_evidence` (`warn-and-error`, `error` or `off`) chooses which outcomes keep it.
…ion status Snowflake cost history reads have `purpose = 'cost_telemetry'`, a read that timed out and was retried in smaller windows ends with outcome `handled`, and each invocation records how far cost attribution got in `cost_status`.
SIGTERM stops a command the way Ctrl-C does: Recall records it as `cancelled` with `signal_number`, its real end time and its flushed output, also when it arrives during project discovery, and `sqb` exits with code 143. Runs that died without an end have `ended_at_estimated = 1`.
SQBRSQL028 counts a plain slot read such as `@weekly.total` as unqualified, though it expands to `weekly.total`, and an aliased read as qualified. A macro's SQL is judged in its macro file rather than again at each call, so a step written only in plain reads passes, and a caller's own columns are judged without the macro output. In a macro file the help points at reading slots through an alias. Generated SQL is unchanged.
A compile whose Rules fail still writes each model's expanded SQL to `target/compiled/models/`, so SQL that macros generated can be read. A model with Rule errors starts with a header marking it not buildable and listing its findings. The exit code, the manifest, the DAG file and the other artifacts are unchanged.
Move helpers and test fixtures to where the layering and test-shape rules expect them, route query output through the shared terminal helpers, and record the new native crossing budgets.
Expect DuckDB result column types, the long query format, the compiled SQL a Rules failure now writes, and keep the statement lifecycle host entry as a dead-code root.
…rupted setup fails
…native handler A native SIGTERM handler sets a flag; a watcher cancels every running statement and interrupts the main thread the way Ctrl+C does. A statement error after SIGTERM becomes the termination, so the run ends cancelled with exit 143 instead of a failed model. A second SIGTERM ends the process at once.
…raceback Map a termination at the top of the CLI entry after its watcher has finished, so SIGTERM during argument parsing, Recall start-up or the Recall wait at exit ends the same way as during dispatch.
…nts exactly once Cancelled runs record exit code 143, or 130 for Ctrl+C. The Recall sink notes which of the invocation's own events it recorded, so an interrupt during the start event no longer records a second start. SIGTERM in a scenario run cancels its statements as Ctrl+C does.
Replace the exemption of SQL macro output with the ticket's rule: a plain slot read counts as unqualified, an aliased read as qualified, and any other macro output as the macro writes it. A query the macro writes whole is judged once, in the macro file; a query mixing caller and macro columns is judged at the call, so an expression or column-list macro's bare output beside a hand-qualified column fails again.
…ence keeps them The invocation's audit_evidence setting now gates violation sampling itself: off collects none and error collects only for error outcomes, so those rows reach neither the warehouse audit history nor the audit_completed event. Measurement evidence is unchanged.
A compile whose Rules fail now removes an earlier target/manifest.json, so target/ no longer pairs new compiled SQL with a manifest from a previous compile.
Read the compile graph's project and write the audit warning with the summary through fewer Python calls, reuse the stdout terminal and Decimal checks the query output already has, and set the budgets to the new counts.
kvlonge
enabled auto-merge (squash)
October 28, 2026 02:31
A compile whose Rules fail now leaves only this compile's model files under target/compiled/, so a model removed since the last compile no longer keeps its compiled file and a warm compile matches an uncached one.
…qualification style In a caller's query, a column a macro call reads through the relation it was passed, such as o.created_at from @src.created_at, no longer counts as unqualified. The slot decides how that read expands, so a hand-qualified caller using such a macro passes again, while a macro's bare column beside hand-qualified columns still fails.
sql_available and sql_unavailable_reason now share one read of the per-invocation SQL references, so each source table is read once per lookup.
…tired models A compile whose Rules fail writes each model's SQL and removes only the compiled file of a model the project no longer has, using a record of the model files compiles wrote. Audits, tests and other artifacts stay as the last passing compile left them, so a failed compile publishes nothing else and a later delta or warm compile matches a full one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Recall needs to be trustworthy on shared production stores: cancelled runs, selections, audit evidence, failed-statement SQL and cost telemetry were missing or misreported. Query output hid data, SQBRSQL028 rejected consistent macro slot reads, and
sqb auditwrote into a literalpreserveschema on preserve targets.Changes
cancelledwith its real end and exit code; lifecycle events are written once.recall.audit_evidence(warn-and-errordefault,error,off) with row and size caps.sqb queryandsqb recall queryshare one renderer: a box when it fits, long form otherwise, never hiding data.failure/rule-custom-finding,failure/rule-unused-cteandfailure/rule-unused-cte-after-variable-and-macronow include each model's compiled SQL, with a not-buildable header on the model that fails Rules.BREAKING CHANGE:
--format csvand--format tableare removed fromsqb queryandsqb recall query. Use the default output,--format long, or--json.Verification
make check-ciafteruv sync --all-extras;fensu check0 faults.