Repository navigation
Track retirement of loader_scratch_vm_allocate.patch (iOS loader scratch stays in the footprint) #45
Description
Problem
On iOS 18, after v0.3.0-5 moved ggml host buffers to vm_allocate (#44), about 240 MiB of an SD-Turbo model's footprint still stays in libmalloc's cache after dispose (llamadart #2004, plan #2011). It is held by the per-thread scratch vectors in ModelLoader::load_tensors (third_party/stable-diffusion.cpp/src/model_loader.cpp, read_buffer, convert_buffer, zip_entry_buffer) and by convert_tensor's buf.
Fix
patches/loader_scratch_vm_allocate.patch backs those buffers with vm_allocate/vm_deallocate on Apple; other platforms keep std::vector. Test: tests/test_loader_scratch_runtime.py.
Patch-specific maintainer exception
Like torch_zip_entry_lookup.patch (#24), no upstream report is filed: the maintainer has not authorized one. This issue tracks retirement until an upstream report is authorized; retire the patch when upstream's loader no longer takes tensor-sized scratch from the heap on Apple platforms.
Acceptance
- PR adding the patch and test merged.
- Verified on iOS 18 after a release.
- Patch retired or an upstream report authorized.
Activity
Carry loader_scratch_vm_allocate.patch: Apple loader scratch from vm_…
leehack commentedon Nov 15, 2026
Fixed by #46 (merged as 0ec6c1f), which ships in
v0.3.0-6. On-device iOS 18 confirmation will come from the llamadart A/B.Non-blocking audit P3s, kept here:
patches/README.md: the new row (and the earlierintrinsic_tiny_decoder_policyrow) comes after a blank line, so GitHub renders it as plain text, not as part of the table.vm_deallocatefromTensorScratch::release()(mutation M5), and none proves the memory goes back to the OS. The probe only checks thatoperator newis avoided. Only the manual macOSphys_footprinttable covers this.zip_entry_bufferhas no PyTorch-zip fixture (mutation M3 survives).model_manager.cpp:974(load_tensor_parts) andload_float_tensor'sloaded_data(model_loader.cpp:1486) still allocate tensor-sized buffers on the heap.