feat(core)!: support offset_expr on window bounds (spec v0.102.0) - #1156
feat(core)!: support offset_expr on window bounds (spec v0.102.0)#1156anasik wants to merge 5 commits into
Conversation
|
Can you check |
check now, @nielspardon |
nielspardon
left a comment
There was a problem hiding this comment.
Thanks for the copy-on-write fix — that part looks good.
The main thing left is on the producer side, in isthmus WindowBoundConverter. It still turns every Calcite offset into a long, so RANGE BETWEEN 1.5 PRECEDING silently becomes 1 PRECEDING (DECIMAL is in SqlTypeName.EXACT_TYPES), and it can never emit an offset_expr for a decimal or interval offset. Since the deprecated offset field is an int64 and has to stay one, I'd build the bound from the offset's own type: integral literals become an i64 offset_expr, which the writer still mirrors into offset; everything else sets only offset_expr. The same place should map a zero offset to WindowBound.CURRENT_ROW, since the spec says zero isn't an offset. One caveat for RANGE: an integral offset needs the ordering column's type, not a blanket i64, or add(T, D) -> T isn't defined.
Three body-level ones: please add an offset expression to the RelSamples ConsistentPartitionWindow sample so the shared round-trip tests cover one; the BREAKING CHANGE: footer should also mention the removed public BoundConverter.convert(WindowBound) and that plans with bounds_type unset no longer parse; and ExpressionRexConverter.isRowMode (:760) still rejects UNSPECIFIED even when both bounds are unbounded, which checkBoundsType now allows.
The rest is inline. Suggestions are written against 62e2aa9e but not compiled — worth a local build before committing them.
nielspardon
left a comment
There was a problem hiding this comment.
This looks good — the producer side is better than what I suggested; keying the offset type off the ordering expression is the right call, and WindowBoundConverterTest covers the cases individually.
Three follow-ups, none of which I'd block on — see inline.
Two notes outside the diff: the bullet "Update isthmus and Spark conversions, since both only support literal offsets in their own window frame APIs" isn't true for isthmus any more, and the body becomes the commit message. And WindowBoundConverter.toWindowBound is public and gained three parameters — probably fine to leave out of the footer given it's effectively isthmus-internal, but worth a thought.
Preceding/Followingnow preferoffset_exprover the deprecatedoffsetwhen set, per the spec's precedence ruleBOUNDS_TYPE_UNSPECIFIEDwhen a bound isCurrentRow,Preceding,orFollowing; still allowed when both bounds areUnbounded(needed for Spark'sUnspecifiedFrame)negate(decimal<P,S>)viaFunctionMappings'sUNARY_MINUS→negatemapping, soSELECT -decimal_colmay now resolve a different function; untested by any isthmus testCloses #1143
BREAKING CHANGE:
WindowBound.Preceding/Following.offset()now returnsExpressioninstead oflong.ExpressionProtoConverter.BoundConverter's publicconvert(WindowBound)is now an instance method (reachable viaExpressionProtoConverter.toProto(WindowBound)) instead of a static utility. A plan withbounds_typeunset (BOUNDS_TYPE_UNSPECIFIED) alongside a bound that isn't Unbounded no longer parses; it must setbounds_typetoROWSorRANGE, or useUnboundedon both sides.