[jerry-024]

Summary

Allow DiskANN v1 compact indexes to store and search vectors above 1024 dimensions, including 2560-dimensional vectors.

Changes

• Remove the global 1024-dimensional validation and the compact layout's one-page raw-vector limit. Keep the interleaved layout's 4 KiB record bound.
• Process F16 raw vectors in fixed-size stack chunks for L2, inner product, and cosine reranking, avoiding per-candidate allocations for wider vectors.
• Cover 2560-dimensional F32/F16 build, read, and search paths, and update the storage-format and user documentation.

Testing

• cargo test -p paimon-vindex-core diskann_ --lib --quiet (174 passed)
• cargo fmt --all -- --check
• git diff --check

Notes

• The separate DiskANN performance benchmark has been launched, but its results have not yet been reviewed.
• Older readers that still enforce the 1024-dimensional v1 check will reject newly written indexes above 1024 dimensions. The interleaved layout remains bound by its 4 KiB page.

[JingsongLi]

+1