Skip to content

fix(parquet/internal/encoding): don't panic decoding zero BYTE_STREAM_SPLIT values without SIMD - #1579

Open
zeroshade wants to merge 1 commit into
apache:mainfrom
zeroshade:bss-decode-empty
Open

zeroshade wants to merge 1 commit into
apache:mainfrom
zeroshade:bss-decode-empty

Conversation

@zeroshade

Copy link
Copy Markdown
Member

Rationale for this change

ByteStreamSplitDecoder.Decode for 4- and 8-byte types panics with index out of range [0] with length 0 when there is nothing left to decode, but only when the pure Go implementation is used (-tags noasm, or a CPU without AVX2/NEON). The pure Go functions take &out[0] of an empty slice. The AVX2 and NEON wrappers already return early for zero values, so the default build on amd64 and arm64 returns 0, nil.

A column reader hits this when a page holds fewer values than its header claims and it asks the decoder for the rest.

What changes are included in this PR?

The pure Go width-4 and width-8 decode functions (little- and big-endian) return early for zero values, like the SIMD wrappers.

Are these changes tested?

Yes, TestByteStreamSplitDecodeExhausted decodes past the end of a page for int32 and int64. It panics without the fix under -tags noasm.

Are there any user-facing changes?

No API changes. Decoding an exhausted BYTE_STREAM_SPLIT page no longer panics in builds without SIMD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant