Skip to content

Move asap_sketchlib to current main - #43

Merged
zzylol merged 2 commits into
mainfrom
chore/sketchlib-main
Oct 21, 2026
Merged

zzylol merged 2 commits into
mainfrom
chore/sketchlib-main

Conversation

@zzylol

@zzylol zzylol commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Why

The crate pinned asap_sketchlib at 3fc3442, 11 commits behind main. Upstream has since made ASAPv1 decoding strict, unified the DDSketch encoding, renamed the sketch types and methods (#209), and removed the legacy serialization formats. Staying on the old pin would keep the demo on wire formats and names that no longer exist upstream.

What

  • Moves the pin to asap_sketchlib main (60101c6).
  • Adapts the sketch wrappers and the merge bench to upstream's renames. No sketch logic is reimplemented or vendored; upstream has everything the crate needs.
  • Updates the merge-bench byte counts for the two sketches whose payloads changed.

How

Renames applied in src/sketches/{cms,cms_heap,countsketch,ddsketch,hll,kll,mod}.rs and src/bin/asap_merge_bench.rs:

  • CountMin → CountMinSketch, Count → CountSketch, CMSHeap → CountMinSketchWithHeap, HeapItem → OwnedDataInput, DDSketch → DdSketch, KLL → Kll, ErtlMLE → Ertl.
  • insert / insert_many → update / update_weighted.
  • serialize_to_bytes / deserialize_from_bytes → to_asapv1 / from_asapv1.
  • KLL init_kll / init_kll_with_seed → with_k / with_k_and_seed (the default m is still 8).

The crate was already ASAPv1-only, so removing the legacy formats changes nothing here. The lockfile no longer pulls in prost or serde-big-array.

Before

  • asap_sketchlib 3fc3442 with the old type and method names.

After

  • asap_sketchlib 60101c6. Two payloads change: DDSketch always writes its negative store and zero count (701 → 704 bytes), and the Count-Min-with-heap metadata key k is now heap_size (8,766 → 8,774 bytes). Count-Min, Count Sketch and HLL sizes are unchanged.
  • Decoding is stricter (trailing bytes, non-float64 float fields and out-of-order heap entries are rejected), so every stage of a deployment must run the same build. The demo builds one image for all stages.
  • docs/results/merge-bench.md and .csv carry the new byte counts; the timings are still from the original run.

Verification

  • cargo fmt --check, cargo clippy --all-targets --features otap-engine,stream-benchmark -- -D warnings and cargo test --features otap-engine,stream-benchmark pass on this head (99 + 26 + 11 + 6 + 1 tests). No golden or wire-format tests needed changing.
  • cargo build --release --features collector --bin asap-collector builds. CI runs the scaled acceptance on this head.

@zzylol zzylol added the enhancement New feature or request label Oct 20, 2026
@zzylol
zzylol force-pushed the chore/sketchlib-main branch from d927134 to 17030ed Compare October 20, 2026 23:38
@zzylol
zzylol force-pushed the feat/cardinality-tradeoff-demo branch from 67d557c to ba4d60b Compare October 20, 2026 23:38
@zzylol
zzylol force-pushed the chore/sketchlib-main branch from 17030ed to f3c4b28 Compare October 20, 2026 23:38
@zzylol
zzylol force-pushed the feat/cardinality-tradeoff-demo branch 2 times, most recently from fc4afc4 to d997227 Compare October 20, 2026 23:39
@zzylol
zzylol force-pushed the chore/sketchlib-main branch 2 times, most recently from 5d672e8 to 57ff866 Compare October 20, 2026 23:55
@zzylol
zzylol force-pushed the feat/cardinality-tradeoff-demo branch from d997227 to 2eebd74 Compare October 20, 2026 23:55
@zzylol
zzylol force-pushed the chore/sketchlib-main branch from 57ff866 to ededb1d Compare October 21, 2026 00:02
Bump the asap_sketchlib pin from 3fc3442 to 60101c6 and adapt the
sketch wrappers and the merge bench to upstream's renames:

- CountMin -> CountMinSketch, Count -> CountSketch,
  CMSHeap -> CountMinSketchWithHeap, HeapItem -> OwnedDataInput,
  DDSketch -> DdSketch, KLL -> Kll, ErtlMLE -> Ertl
- insert / insert_many -> update / update_weighted
- serialize_to_bytes / deserialize_from_bytes -> to_asapv1 / from_asapv1
- KLL init_kll / init_kll_with_seed -> with_k / with_k_and_seed

The crate already used ASAPv1 only, so no legacy format is involved.
Upstream's ASAPv1 changes alter two payloads: DDSketch always writes
its negative store and zero count (+3 bytes here), and the
Count-Min-with-heap metadata key `k` is now `heap_size` (+8 bytes).
The merge-bench byte counts for those two sketches are updated.
@zzylol
zzylol force-pushed the chore/sketchlib-main branch from ededb1d to b0cf1ac Compare October 21, 2026 00:17
@zzylol
zzylol changed the base branch from feat/cardinality-tradeoff-demo to main October 21, 2026 00:17
@zzylol
zzylol merged commit af8bb26 into main Oct 21, 2026
2 checks passed
@zzylol
zzylol deleted the chore/sketchlib-main branch October 21, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant