Skip to content

Remove redundant microbenchmarks - #11179

Merged
connortsui20 merged 1 commit into
developfrom
ct/stoic-rubin-t4f27x
Nov 3, 2026
Merged

connortsui20 merged 1 commit into
developfrom
ct/stoic-rubin-t4f27x

Conversation

@connortsui20

@connortsui20 connortsui20 commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Summary

Remove third-party timing baselines, old decode-first comparison paths, and one exact duplicate. Keep benchmarks for distinct Vortex operations and representations.

Changes

  • Reduce the integer Widen target from 46 cases to 17. Keep the operations on Widen arrays and measure decoding separately, instead of repeating every operation on canonical and explicitly decoded inputs.
  • Remove the expanded-index take baselines, decode-then-compare baselines, and the benchmark that manually reifies constant run lengths. Keep the maintained entry points and their distinct input representations.
  • Remove patterns_i128(Random), which duplicates random_i128(0.5) with the same values and mask.
  • Remove Arrow and std timing baselines and their unused dependencies. Keep the row_encode --output-sizes comparison and the documented local allocation baselines.
  • Update the affected sharding estimates in scripts/codspeed_bench_counts.json, preserving the new decimal Widen target. These estimates are not a fresh inventory of every simulation benchmark.

The restored integer arithmetic, Widen slice/chunked, bitpacked Lt, and mask iteration cases remain. The exclusions from #11176 also remain in effect.

Validation

The original revision reported targeted Clippy and simulation builds, shard-planner tests, spelling checks, and pinned-nightly formatting. Those checks have not been rerun after rebasing onto current develop. No local checks were run during this cleanup. The pre-commit and pre-push hooks were bypassed as requested. CI validation is pending.

@connortsui20

Copy link
Copy Markdown
Member Author

I brought back a few

Keep the maintained Vortex entry points while removing third-party timing
baselines, explicit decode-first comparison paths, and an exact duplicate.
Measure integer Widen decoding separately instead of repeating every
operation over canonical and explicitly decoded inputs.

Update the affected benchmark sharding estimates and drop unused benchmark
dependencies. Preserve distinct operations, the decimal Widen target, and
the CodSpeed exclusions already on develop.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@codspeed

codspeed Bot commented Nov 3, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 4.24%

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 3 improved benchmarks
❌ 5 regressed benchmarks
✅ 1740 untouched benchmarks
🆕 17 new benchmarks
⏩ 611 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation bitwise_and_vortex_buffer[128] 4.1 µs 5.8 µs -29.56%
❌ CPU Simulation patched_take_10_stratified 55.1 µs 69.4 µs -20.68%
❌ CPU Simulation random_i64[0.95] 129.1 µs 155.5 µs -16.99%
❌ CPU Simulation primitive_i64_vortex 72.3 µs 86.1 µs -16.01%
❌ CPU Simulation fragmented_runs[7] 111.1 µs 126.5 µs -12.15%
⚡ CPU Simulation patterns_i128[Runs] 99.2 µs 74.3 µs +33.59%
⚡ CPU Simulation random_i128[0.5] 123.1 µs 95.4 µs +29.1%
⚡ CPU Simulation random_i128[0.8] 149.1 µs 124.5 µs +19.82%
🆕 CPU Simulation add_canonical N/A 929.3 µs N/A
🆕 CPU Simulation add_constant N/A 533.8 µs N/A
🆕 CPU Simulation add_widen N/A 598.8 µs N/A
🆕 CPU Simulation between_in_range N/A 136.4 µs N/A
🆕 CPU Simulation cast_f64 N/A 349.6 µs N/A
🆕 CPU Simulation chunked N/A 1.5 ms N/A
🆕 CPU Simulation compare_in_range N/A 161.5 µs N/A
🆕 CPU Simulation decode N/A 342.8 µs N/A
🆕 CPU Simulation fill_null_out_of_range N/A 1.2 ms N/A
🆕 CPU Simulation filter N/A 325 µs N/A
🆕 CPU Simulation interleave N/A 1.3 ms N/A
🆕 CPU Simulation min_max N/A 130.3 µs N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/stoic-rubin-t4f27x (3cd9a97) with develop (24cd0ea)

Open in CodSpeed

Footnotes

  1. 611 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@connortsui20
connortsui20 merged commit 5eb35ab into develop Nov 3, 2026
99 of 101 checks passed
@connortsui20
connortsui20 deleted the ct/stoic-rubin-t4f27x branch November 3, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants