[adriangb]

Which issue does this PR close?

• Relates to Improve Performance (Polars 2.0 Call to Action) datafusion#26169 (DataFusion PR: perf: replace the routed hash join dynamic filter with one global filter (InList or bloom filter) datafusion#26442)
• Related: Add Sbbf::estimated_fpp to expose the bloom filter FPP estimate #11088 (Sbbf::estimated_fpp, merged), Add bloom filter folding to automatically size SBBF filters #9628 (folding, merged)

Rationale for this change

DataFusion builds a parquet Sbbf as the membership test in the dynamic filter of a hash join. Each build partition fills its own filter in parallel with hashes it has already computed, the filters are merged into one, the result is folded to the target fpp, and every probe row is checked against it.

The public API cannot do this today:

Need	Today
Insert or check precomputed hashes in bulk	insert_hash / check_hash are private, and per hash
Merge filters built independently (one per thread or partition)	Not possible
Check many hashes fast	check exits early on a branch the CPU cannot predict

Measured in DataFusion on 16M probe rows: checking with a per-row branchy check takes ~104 ms, the branch-free check ~21 ms.

What changes are included in this PR?

All additions are on parquet::bloom_filter::Sbbf:

API	What
insert_hashes(&mut self, &[u64])	Insert precomputed 64-bit hashes
check_hashes(&self, &[u64]) -> BooleanBuffer (feature arrow)	Branch-free bulk check: tests all 8 words of the block with no early exit
union(&mut self, &Sbbf) -> Result<()>	OR another filter into this one. other must have the same number of blocks or a multiple of it; a larger other is folded on the fly (same grouping as fold_to_target_fpp), so fold(a ∪ b) == fold(a) ∪ fold(b)

The hashes can come from any well-mixed 64-bit hash. check / insert keep using the spec hash; the docs say the two only agree when the caller uses the spec hash.

An earlier version of this PR added hash_value, public insert_hash / check_hash and Arrow array APIs (hash_array, insert_array, check_array, supports_data_type). DataFusion no longer needs them (it reuses its own join hashes), so they are removed.

Are these changes tested?

• Branch-free check equals check_hash.
• check_hashes equals check_hash for each hash.
• Union of per-chunk filters is bit-for-bit equal to one filter of all hashes; union then fold equals fold then union; a larger filter is folded on the fly; a smaller filter cannot be unioned into a larger one.

Are there any user-facing changes?

New public API only. No breaking changes.

🤖 Generated with Claude Code

[mbutrovich]

Can we get some extra review cycles on #11549 and land that first, since it already brings in Sbbf?

[adriangb]

Thanks @mbutrovich . I reviewed #11549 and plan to restack this work on it once it merges (experiment for that in #11685)

[mbutrovich]

Thanks @mbutrovich . I reviewed #11549 and plan to restack this work on it once it merges (experiment for that in #11685)

Sounds good, thanks for the review pass! I was excited to see someone with another use for it so wanted to make sure it lands with what you need.

[etseidl]

@adriangb do you want to dust this one off? I think we're close to the point of pulling bloom filters into a new crate.

[adriangb]

I've rebased. I can't mark this as ready / non-draft because I've hit my PR limit.