Skip to content

fix: bind subqueries outside DuckDB lambdas and keep strict divergence refusals SQ0008 - #92

Merged
kvlonge merged 6 commits into
mainfrom
fix/launch-translation-errors
Nov 6, 2026
Merged

kvlonge merged 6 commits into
mainfrom
fix/launch-translation-errors

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Why

The most common incremental filter, WHERE t >= DATEADD(DAY, -3, (SELECT MAX(t) FROM events)), emitted DuckDB SQL that DuckDB can't bind. The DATEADD emission wraps its operand in a lambda, and DuckDB refuses subqueries inside lambdas. The statement was then refused as SQ0002 with "no verified rule converts", spanning the whole statement. Separately, under --strict, numeric_range refusals reported SQ0003 with schema help, when every other refused divergence reports SQ0008 with policy help.

Changes

  • Conversion records each subquery value (scalar subquery, EXISTS, IN) it emits. Before a statement is emitted, a value that ends up inside a DuckDB lambda is bound once outside the lambda and referenced from inside it, so the lookback converts and binds.
  • The value is hoisted only when that can't change the result. These cases refuse as SQ0005, with the span on the subquery:
    • a subquery that reads the lambda's parameter;
    • a volatile subquery, one that calls a non-deterministic built-in or a project function not declared immutable, because hoisting could change how many times it is evaluated compared with Snowflake.
  • Every emitted statement already passes DuckDB's bind check (check_printed) before it is returned, and any rule whose SQL fails to bind is refused rather than emitted. That stays unchanged.
  • Refusals caused by a refused divergence keep that category through the outer conditions, so strict numeric_range reports SQ0008 with policy help.
  • test_composition compares time growth between depths instead of asserting an absolute wall-clock bound, which failed on loaded runners.

Verification

  • New tests: DuckDB lambda-binding scanner tests; CLI convert cases for the lookback and both SQ0005 refusals; a strict numeric_range SQ0008 help case; a value test running the lookback on DuckDB. The strict-mode analysis tests now expect DivergenceRefused.
  • New case file cases/temporal/subquery_operands.toml, recorded on Snowflake. Both cases match, and the translation baseline gains exactly those 2 entries.
  • make check-ci RECORDER=0 and fensu check pass, and the hygiene scan is clean. The gate on the exact head is recorded in a PR comment.

… SQ0008

WIP backup, not gated. Built and spot-checked locally only; the CLI and
corpus tests are not yet run, and the branch still needs a merge of main
(fensu reports conversion/constants.rs over its line limit until then).

- A subquery value a rule wraps in a DuckDB lambda is bound once outside
  every lambda. A subquery reading the lambda's parameter, or one that may
  call a volatile function, refuses as SQ0005 at the subquery instead.
- Refused divergences re-wrapped as type conditions keep SQ0008, so strict
  numeric_range refusals get policy help.
- test_composition compares time growth between depths, not an absolute
  wall-clock bound.
…rectly

Run rustfmt over the new lambda-binding code, and have the composition
test assert against its expected_max_time_ratio field directly.
…ergences

The subquery operand cases end in ORDER BY, so they declare an ordered
result. Strict-mode refusals of numeric_range, date_text_spelling and
session_timezone_utc now report a refused divergence, so the analysis
tests expect that category. The lookback value test reads its count as
text.
Snowflake results for the two cases in cases/temporal/subquery_operands.toml at 3287352.
The two recorded subquery operand cases return Snowflake's rows, so the
translation baseline gains them as matching. No other entry moves.

kvlonge commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Gate for landing at exactly a1fb85b2bf9e25ec943ad65f93a78d35ce3f6090 (on top of main 4f52a5bf; fast-forward):

  • make check-merge RECORDER=0: exit 0

  • make verify-recorded: exit 0. 8,470 cases: 6,780 passed, 994 diverged as expected, 660 refused, 36 typed, 0 pending, 0 unproven, 0 failed; 220 of 220 equivalences and rules verified.

  • Translation measure against main: 0 regressions, and 2 other changes, both new statements recorded as matching:

    • temporal/subquery_operands.toml#dateadd_subquery_lookback
    • temporal/subquery_operands.toml#dateadd_subquery_nested

    The committed baseline holds the measure.

  • Evidence for the two new cases: recorded run d182-a (2 recorded, 0 failures), imported as fa2b6a1.

  • Hosted CI (Website) is green on this head, and the public-repository hygiene scan over the full tree is clean.

  • Behaviour notes:

    • A subquery that reads a lambda's parameter, or that calls a volatile function, refuses as SQ0005 instead of being hoisted out of the lambda.
    • Strict numeric_range refusals report SQ0008.

@kvlonge
kvlonge merged commit a1fb85b into main Nov 6, 2026
1 check passed
@kvlonge
kvlonge deleted the fix/launch-translation-errors branch November 9, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant