Skip to content

D250: POWER(2, n), TO_CHAR MON/MMMM (fixes MMMM), FLATTEN with PATH - #116

Merged
kvlonge merged 16 commits into
mainfrom
fix/d250-launch-blockers
Nov 6, 2026
Merged

kvlonge merged 16 commits into
mainfrom
fix/d250-launch-blockers

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

D250: three launch blockers from corpus round 3 (dbt_utils.date_spine, dbt_date.get_date_dimension, FLATTEN over VARIANT), each converted from Snowflake receipts. A wrong answer TO_CHAR gave for MMMM is fixed too.

What changes

  • POWER(2, n) of a whole NUMBER exponent converts as pow(2, n). A double holds every whole power of two from 2^-1074 to 2^1023 exactly, so there is no rounding question, unlike other bases. Fractional exponents and other bases still refuse.
  • TO_CHAR MON and MMMM write the English month abbreviation and full name in title case (Jan, January). They convert only when written in upper case, as recorded; mon or Mmmm refuse.
  • Fixes a wrong answer: on main, TO_CHAR(d, 'MMMM') converts to strftime(d, '%m%m'), giving '0909' instead of 'September', because the shorter MM matched both halves. The tokenizer now refuses any format where two adjacent fields spell one longer run of a letter (MMMMMM, DDDD, SSSS, YYYYYYYY).
    • A new test walks every documented format element (YYYY … UUUU, HH12, AM/PM, A/P, FF0–FF9, TZH…). Each converts as recorded or refuses.
    • TO_DATE matches whole recorded formats, and TO_TIMESTAMP's reader rejects a repeated element, so neither had the double match.
  • LATERAL FLATTEN with PATH of literal dot-separated plain keys flattens the value at that path:
    • PATH is prefixed with the path: x.y[0], x.y.b, and x.y['a c'] for a key that isn't a plain name.
    • A missing path, a scalar, a JSON null or an empty array at the path gives no rows, and keys match by case.
    • PATH with RECURSIVE or OUTER, and other path spellings, still refuse.

Evidence

  • Receipts, recorded by lane A:
  • Replay: pow.power-of-two (5), to-char.month-names (3 of 4; the fourth refuses on the ',' separator), flatten.path (3), flatten.toml t002_path and flatten_recursive flatten_path_option all pass. The older DD-MON-YYYY cases (to-char t002_date_format) now pass too. 0 failed.
  • The dbt repros convert and run on DuckDB: 3 * power(2, 1) gives 6.0, and lateral flatten(input => e.payload, path => 'items') gives the item rows.
  • Full verify: 11,607 cases, 7,650 passed, 0 failed, 220 of 220 verified. Translation baseline against main e9f002d: 0 regressions and 17 changes. 15 statements now match, and 2 new statements refuse: month-names t003 on its ',' separator, and the REGEXP_SUBSTR probe, which Convert REGEXP_SUBSTR with a computed position, occurrence or group #112's computed rule handles.
  • The new format-walk test passes. On main it would fail at MMMM and DDDD.

The D94 gate runs on the landing head when it's this PR's turn.

POWER(2, n) for whole exponents from -52 to 64, dbt_utils.date_spine's
sum of a * POWER(2, k), the double's range edges, overflow and NULL.
TO_CHAR(date, 'MON') and 'MMMM' for every month, beside other fields, on
a timestamp, and for NULL.
The recorded PATH cases reach arrays only. These record an object under
the path (KEY, VALUE and the prefixed PATH, a key with a space), PATH
keys matched by case, and a scalar, JSON null or empty array at the
path.
FLATTEN(input => v, path => 'x.y') flattens the value at v:x.y, as the
recordings show: the same INDEX and VALUE rows, PATH prefixed with the
path ('x.y[0]'), and no rows where the path is missing. The path must be
a literal of plain dot-separated keys; PATH with RECURSIVE or OUTER
stays refused, unrecorded.

flatten.toml#t002_path and flatten_recursive.toml#flatten_path_option
now pass; dbt's lateral flatten(input => e.payload, path => 'items')
converts.
dbt_utils.date_spine sums a * POWER(2, k) for each bit, and POWER of a
base other than 10 refused. A double holds every whole power of two from
2^-1074 to 2^1023 exactly, so DuckDB's pow(2, n) gives the value Snowflake
records with no rounding question, unlike other bases. POWER(2, n) of a
fractional exponent stays refused.

Its cases are pow.power-of-two.toml, pending Snowflake receipts.
… end

Recorded from test/d250-regexp-position 4b84bcb (fix/d250-launch-blockers 7b4d9d2 plus the
REGEXP_SUBSTR probe): 10 cases, utc-v1, no failures. POWER(2, 1024) returns inf rather than an
error; POWER(2, -1075) is 0. TO_CHAR 'MMMM' gives the full English month name, 'MON' the
three-letter one. REGEXP_SUBSTR('abc', 'x*', n) gives '' at 3 and 4 and NULL at 5.
'x*' on 'abc' at positions 3, 4 and 5, for #112's position handling.
…ar or null

Recorded from fix/d250-launch-blockers 16d072b (3 cases, utc-v1, no failures). Under an object,
PATH is prefixed with the input path and a key with a space is bracketed: x.y['a c'], x.y.b.
…wice

TO_CHAR(date, 'MMMM') converted as MM twice ('0909' for September),
where Snowflake writes the month's full name: the format table had no
MMMM, and the shorter MM matched both halves. MMMM and MON are now
fields, writing the English month name and abbreviation in title case
as recorded for every month (receipts d66ba9a); written in another
letter case they refuse, unrecorded.

The tokenizer also refuses a format where two adjacent fields spell one
longer run of a letter (MMMMMM, DDDD, SSSS, YYYYYYYY), so an element
missing from the table cannot silently match as shorter ones. A test
walks every documented format element: each converts as recorded or
refuses. TO_DATE matches whole recorded formats and TO_TIMESTAMP's
reader rejects a repeated element, so neither had the double match.
Against main's baseline (e9f002d): 0 regressions. 15 statements match:
pow.power-of-two (5), to-char.month-names t001, t002, t004, flatten.path
(3), flatten.toml t002_path, flatten_recursive flatten_path_option and
to-char/to-varchar t002_date_format. Two new statements refuse
(month-names t003 on its ',' separator, the REGEXP_SUBSTR empty-match
probe). verify-recorded: 11607 cases, 7650 passed, 0 failed, 220 of 220
verified.
…er PATH

Review: the conditions named the exact range only; 2^1024 = inf and
2^-1075 = 0 are recorded (pow.power-of-two t004, t003). FLATTEN's THIS
under PATH has no receipt yet, so a case records it.
kvlonge added a commit that referenced this pull request Nov 6, 2026
This branch converts POWER(2, n), so the row pinning another base's
refusal named a base that now converts. It names base 3.
month-names t007 and its result from evidence/d250-followup 07fb230:
Snowflake writes MONMM as 'September' and MMMMDD as 'September05'.
Snowflake writes MONMM as the full month name, not MON then MM, so the
converted 'Sep09' was a wrong answer. A field right after MON now
refuses. MMMM before a field stays converted, as t007 records
(MMMMDD is September05). The format walk pins MONMM, MONDD and MMMMDD,
and the translation baseline records t007 as refused.
kvlonge added a commit that referenced this pull request Nov 6, 2026
Bare FF directly after MON converted ('MONFF' gave 'Sep123'), because
the FF branch ran before the MON check. Adjacency is now one table of
recorded followers: nothing after MON, only DD after MMMM (MMMMDD is
September05), and no month name directly after another field. MMMMYYYY,
MMMMHH24, MMMMFF and DDMON refuse until a recording shows them. The
format walk pins MONFF, MMMMYYYY, MMMMFF, DDMON and MON-MM.
kvlonge added a commit that referenced this pull request Nov 6, 2026
kvlonge added a commit that referenced this pull request Nov 6, 2026
@kvlonge
kvlonge merged commit 4b6278d into main Nov 6, 2026
1 of 2 checks passed
@kvlonge
kvlonge deleted the fix/d250-launch-blockers branch November 7, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant