Skip to content

BINARY: the type, the VARCHAR→BINARY cast and byte-wise text functions (D284) - #129

Closed
kvlonge wants to merge 18 commits into
mainfrom
feat/varchar-binary-cast
Closed

kvlonge wants to merge 18 commits into
mainfrom
feat/varchar-binary-cast

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

D284: BINARY as one PR, in separate commits. Every rule rests on a recorded receipt; values no receipt reaches raise or stay refused.

Commits

  1. refactor: move the recorded time zone names into a time_zones domain. conversion/constants.rs is at 2,000 lines on main. Its 601-line table of recorded zone names moves to time_zones/constants.rs, behind a recorded_zone entry. No behaviour change.
  2. BINARY type and the VARCHAR→BINARY cast (receipts: evidence c061da6, cases d92cf53).
    • SnowflakeType::Binary, held in DuckDB as BLOB. BINARY, BINARY(n) and VARBINARY resolve as cast targets.
    • CAST, :: and TRY_CAST read text as hex under the pinned BINARY_INPUT_FORMAT = HEX. Digits of either case decode, '' gives empty bytes, and NULL stays NULL (cast t006/t007, :: t006).
    • Odd-length or non-hex text raises 100115/22000 (cast t008/t009); TRY_CAST gives NULL (try-cast t006).
    • More bytes than BINARY(n) holds raise 100145/22000; Snowflake does not truncate (cast t010). TRY_CAST to BINARY(n) is refused, since nothing records it.
  3. Byte-wise rules, grouped by family, each from the form's t00x_binary receipt:
    • rule.byte_length: LENGTH, LEN and OCTET_LENGTH count bytes, typed NUMBER(18,0).
    • rule.byte_slice: LEFT, RIGHT, SUBSTR and SUBSTRING count bytes.
    • rule.byte_pad: LPAD and RPAD with BINARY padding.
    • rule.byte_search: POSITION, POSITION … IN and CHARINDEX match whole bytes, typed NUMBER(9,0).
    • rule.byte_concat: CONCAT, || and CONCAT_WS of BINARY parts.
    • rule.text_octet_length: OCTET_LENGTH of VARCHAR counts UTF-8 bytes (its t001/t004 receipts).
    • Fail-closed edges: a negative length, a start below 1, empty padding or an empty search raise instead of guessing. BINARY beside another type, LPAD/RPAD of BINARY without padding, and CONCAT_WS of a single BINARY part are refused.
  4. coverage: re-record the translation results baseline.

Results (gated locally at 29f1095; GitHub Actions is blocked by billing)

  • sqlquack verify --require-verified --strict-errors: 11,807 cases, 7,805 passed, 1,031 diverged as expected, 2,934 refused, 0 failed; 227 of 227 equivalences and rules verified.
  • syntax-coverage results --against main's baseline: 0 regressions, 28 other changes. The baseline is re-recorded.
    • Today's figure: 89 → 91. data-types-text.binary and OCTET_LENGTH move to matching. ENCRYPT_RAW moves from unmeasured to refused, since its BINARY input column now types and its case refuses on the function itself.
    • New statements: cast-as t006/t007 matching, t008/t009/t010 matching_error (100115, 100115, 100145); :: t006 and try-cast t006 matching.
    • 19 _binary/OCTET_LENGTH statements go from refused to matching.
  • Under Bind the translation figure to its obligations (D272) #123's obligation loader (every cited case must match), measured on these statements: all 12 targeted forms meet every obligation: CONCAT_WS, CONCAT, CONCAT ||, LEFT, RIGHT, LEN, LENGTH, LPAD, RPAD, POSITION … IN, SUBSTR and SUBSTRING. So does OCTET_LENGTH. POSITION(s, t) and CHARINDEX stay unmet, on their start-position cases only.
  • cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings (sqlquack-python excluded locally), crate tests and fensu check (0 faults) pass.

Left refused (no receipt-backed rule here)

TO_BINARY, TRY_TO_BINARY, HEX_DECODE_BINARY, TRY_HEX_DECODE_BINARY, HEX_ENCODE of BINARY, BIT_LENGTH, BASE64_, MD5/SHA1/SHA2_BINARY, BITAND/BITOR/BITXOR/BITNOT/BITSHIFT of BINARY, INSERT of BINARY, TO_CHAR/TO_VARCHAR of BINARY with a format, COMPRESS and the WKB functions. Each still refuses.

Follow-ups

CAST, :: and TRY_CAST of hex text in mixed case, empty text, NULL,
odd-length and non-hex text, and BINARY(2) overflow, to record before a
VARCHAR-to-BINARY cast rule is built.
Snowflake results for cast.cast-as t006-t010, cast.double-colon t006 and
try-cast t006 at e675d17.
conversion/constants.rs is at 2,000 lines on main, and the BINARY rules
add to it. The 601-line table of time zone names Snowflake was recorded
accepting now lives in time_zones/constants.rs. No behaviour change.
SnowflakeType gains BINARY, held in DuckDB as BLOB. CAST, :: and
TRY_CAST read text as hex under the pinned BINARY_INPUT_FORMAT HEX:
digits of either case decode, empty text gives empty bytes and NULL
stays NULL. Odd-length or non-hex text raises 100115, or is NULL under
TRY_CAST. More bytes than BINARY(n) holds raise 100145 instead of being
truncated; TRY_CAST to BINARY(n) is refused, since nothing records it.
…ation of BINARY

Each family reads BINARY in bytes, as the t00x_binary receipts record:
LENGTH, LEN and OCTET_LENGTH count bytes; LEFT, RIGHT, SUBSTR and
SUBSTRING slice them; LPAD and RPAD pad with BINARY padding; POSITION
and CHARINDEX match whole bytes; CONCAT, || and CONCAT_WS join BINARY
parts. Values no BINARY receipt reaches (negative lengths, starts below
1, empty padding or search) raise instead of guessing, and BINARY beside
another type stays refused. OCTET_LENGTH of VARCHAR counts UTF-8 bytes,
as its receipts record.
BINARY as a data type and OCTET_LENGTH move to matching, and the seven
new BINARY cast cases match Snowflake (three by raising the same error).
ENCRYPT_RAW moves from unmeasured to refused, since its BINARY input
column now types and its case refuses on the function. Nothing regresses.
CAST, :: and TRY_CAST to BINARY under hex input: case-insensitive hex,
empty text and NULL, odd-length and non-hex text, and a value longer than
the declared BINARY length. Where the pages are silent (odd-length hex,
BINARY as a TRY_CAST target) the obligation says the receipt decides.
kvlonge added a commit that referenced this pull request Nov 6, 2026
…ew round 1)

A value already BINARY cast to BINARY(n) fell through to the same-type
cast and lost the length check Snowflake applies (100145). No recording
shows that path, so it now refuses; hex text to BINARY(n) keeps its
recorded check. BINARY(0) and lengths over 67108864 refuse as targets.
The odd-length case description no longer cites a page that says
nothing about odd digit counts.
…nd 1)

A value already BINARY cast to BINARY(n) now goes through the same
length check as hex text, raising 100145 for more bytes than the target
holds instead of returning them all. TRY_CAST of BINARY to BINARY(n)
stays refused. The same-type cast rule declares the BINARY failure.
Records whether a value already BINARY raises 100145 when cast to a
shorter BINARY(n), as hex text does, before #129 relies on it.
kvlonge added a commit that referenced this pull request Nov 6, 2026
Snowflake results for cast.cast-as t011 and t012 at 57820de; a value longer
than n raises 100145/22000 rather than truncating.
A BINARY value that fits BINARY(2), with its length and NULL, and one that
is a byte too long, which raises as the cast of hex text does.
Snowflake results for cast.double-colon t007 and t008 at 5b4dead; as with
CAST, a value longer than n raises 100145/22000 rather than truncating.
The same two cases as CAST: a BINARY value that fits BINARY(2), with its
length and NULL, and one a byte too long, which raises.

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Closing: SQLQuack lands through local-gate trains, not PR merges (D94), so open PRs only add clutter (D628/D630). The branch is kept. Any work here that isn't on main yet is tracked by the train owner and lands through a train.

@kvlonge kvlonge closed this Nov 8, 2026
@kvlonge
kvlonge deleted the feat/varchar-binary-cast branch November 9, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant