Repository navigation
Cache snapshot memory hashes and add checked saving - #2079
Conversation
Make save reuse content-addressed blobs without rereading them and add checked_save for full hashing and existing-blob verification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
There was a problem hiding this comment.
🟡 Changes recommended
An Unreleased changelog entry is missing to explain that save trusts same-length blobs and callers needing verification and corruption repair should use checked_save.
1 open issue
What changed in this PR
This PR reduces repeated hashing during snapshot persistence by caching memory-layer digests and trusting existing blobs during ordinary saves. Snapshot::checked_save provides explicit verification and repair under the documented immutable-store assumption.
Changes:
- Cache digests on shared memory layers and seed them during
checked_load. - Let ordinary saves reuse same-length regular blobs without reading their contents. Checked saves recompute hashes and atomically replace mismatched blobs.
- Document the trust boundary and add regression tests for hash reuse and corruption repair.
| File | Description |
|---|---|
src/hyperlight_host/src/sandbox/snapshot/memory/mod.rs |
Stores a digest cache on each shared memory blob. |
src/hyperlight_host/src/sandbox/snapshot/file/mod.rs |
Implements checked saving, cached hashes, and cache seeding during verified loads. |
src/hyperlight_host/src/sandbox/snapshot/file/fsutil.rs |
Supports trusted blob reuse or verified atomic replacement. |
src/hyperlight_host/src/sandbox/snapshot/file/digest.rs |
Tracks hashed bytes in test builds. |
src/hyperlight_host/src/sandbox/snapshot/file_tests.rs |
Tests hash reuse, checked-save hashing, corruption repair, and new destinations. |
docs/snapshot-oci-format.md |
Explains immutable blob reuse and verification guarantees. |
Review effort: Balanced
Explain trusted blob reuse in save and direct callers needing verification and corruption repair to checked_save. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
This comment has been minimized.
This comment has been minimized.
Preserve both immutable-blob and durability documentation after the sync removal merged upstream. Clarify that checked_save does not add file or directory synchronization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Jorge Prendes <jorge.prendes@gmail.com>
Benchmark ResultsMeasured commit: kvm / amd (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
kvm / intel (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
mshv3 / amd (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
mshv3 / intel (Linux) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
hyperv-ws2025 / amd (Windows) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
hyperv-ws2025 / intel (Windows) (➖ stable)No benchmark improved or regressed. Benchmark Resultsfunction_call_codec
payload_allocation
sandboxes
slot_pool
snapshot_files
virtq_readonly
virtq_readwrite
Reported by |

Summary
Avoid repeatedly hashing immutable snapshot memory layers and rereading existing content-addressed blobs during ordinary saves, while retaining a fully verifying save option.
checked_loadverification.Snapshot::savetrust existing regular blob files with the expected content-addressed name and length, without reading their contents.Snapshot::checked_saveto recompute memory-layer hashes and verify existing blobs before reuse, atomically replacing mismatched contents.Behavior and trust boundary
Ordinary
saveno longer detects or repairs same-length corruption in existing memory blobs. Callers must keep loaded and published memory blobs immutable and outside writable guest mounts. Usechecked_savefor save-time verification orchecked_loadto verify persisted contents. Neither method protects against concurrent mutation of loaded backing files.The OCI format, tag semantics, and metadata/transport write paths are unchanged. There is no backing-file cache. This PR is independent of #2078 and does not remove synchronous file flushes.
Validation
cargo test -p hyperlight-host --locked --lib save -- --test-threads=1— 24 tests passed, including VM-backed round trips, cache reuse, full checked-save hashing, and corrupted-blob repair.cargo clippy -p hyperlight-host --locked --lib -- -D warningscargo +nightly fmt --all -- --checkgit diff --checkNo isolated latency improvement is claimed; tests verify the reduction in bytes hashed.