Skip to content

Reduce ClickHouse lookup memory use and scan time - #966

Merged
Intelli merged 4 commits into
PlayPro:masterfrom
Telesphoreo:optimize-clickhouse-lookups
Sep 3, 2026
Merged

Reduce ClickHouse lookup memory use and scan time#966
Intelli merged 4 commits into
PlayPro:masterfrom
Telesphoreo:optimize-clickhouse-lookups

Conversation

@Telesphoreo

Copy link
Copy Markdown
Contributor

Summary

ClickHouse compatibility views converted binary strings into Array(Int8) values one byte at a time. A rollback lookup could allocate several gigabytes before it returned any rows. The views also wrapped location columns in if expressions, which caused spatial lookups to scan more data than needed.

This PR upgrades the ClickHouse JDBC driver to 0.10.0 and enables its binary string support. CoreProtect can now read binary columns as raw bytes without building per-byte arrays in ClickHouse.

Location inspectors now read event_data FINAL directly. They filter the stored family, wid, x, and z columns, which match the start of the table sorting key. Compatibility views also project stored location columns without if wrappers.

Union queries now use numeric tbl values. This prevents ResultSet.getInt("tbl") from failing with the newer JDBC driver.

Testing

  • Built with mvn -DskipTests package.
  • Checked block inspection on a live ClickHouse database.
  • Checked a one-day rollback lookup with a 50-block radius on the same database.

Fixes #965

@Intelli
Intelli merged commit fa7ed2a into PlayPro:master Sep 3, 2026
3 checks passed
@Telesphoreo
Telesphoreo deleted the optimize-clickhouse-lookups branch September 4, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize ClickHouse location lookups

2 participants