[leehack]

Closes #45. For llamadart #2004, plan #2011.

Root cause

#44 moved ggml host buffers to vm_allocate, but on iOS 18 about 240 MiB stayed in libmalloc's cache after dispose. It is held by ModelLoader::load_tensors's per-thread scratch vectors (read_buffer, convert_buffer, zip_entry_buffer, model_loader.cpp:1171-1173) and convert_tensor's buf (:151). An interposer on v0.3.0-5 saw 28 to 56 MiB blocks from those threads (every one returned into load_tensors(...)::$_0).

Change

• patches/loader_scratch_vm_allocate.patch: on Apple, TensorScratch over vm_allocate/vm_deallocate (grows to max(size, 2*cap), keeps no contents, throws std::bad_alloc); other platforms keep std::vector<uint8_t>. Replaces the three loader vectors and convert_tensor's buffer. The zip_entry_buffer hunk stays.
• CMakeLists.txt carry line, patches/README.md row (patch-specific maintainer exception, retirement tracked in Track retirement of loader_scratch_vm_allocate.patch (iOS loader scratch stays in the footprint) #45; nothing posted upstream), README.md count of carried fixes (five to six).
• Tests tests/test_loader_scratch_runtime.py and tests/native/loader_scratch_runtime_test.cpp; macOS smoke step in validate.yml (build job, after "Split graph evaluation", SD_REQUIRE_RUNTIME=1; GitHub's runners have no Metal, so the Metal case skips there).

Footprint (SD-Turbo q8_0, M4 Max, macOS 26.7.1, phys_footprint MiB, 3 load/decode/dispose cycles, 6 threads)

Build	Loaded	Peak	5 s after dispose
v0.3.0-5	2194 / 2135 / 2185	2695 / 2621 / 2627	56 / 583 / 589
patched	1680 / 1658 / 1663	2126 / 2105 / 2107	54 / 65 / 68

Failing first

macos-arm64 built from unpatched main with the new tests, SD_REQUIRE_RUNTIME=1 SD_REQUIRE_METAL_GENERATION=1: 4 failures (largest loader-thread operator new: 147456 B CPU/F32, 2056192 B Metal/F32, same for BF16 convolutions). With the patch: OK, 184 B in all four cases.

Mutations (largest loader-thread operator new, bytes; threshold 16384)

Mutation	F32 CPU	F32 MTL0	BF16 CPU	BF16 MTL0	Result
M1 read_buffer to vector	fail	fail	fail	fail	killed (4 failures)
M2 convert_buffer to vector	pass	fail	pass	fail	killed on Metal only (2 failures)
M3 zip_entry_buffer to vector	pass	pass	pass	pass	survives: there is no PyTorch-zip fixture
M4 convert_tensor buf to vector	pass	pass	fail	fail	killed (2 failures)

M3 survives because only a PyTorch ZIP checkpoint reaches zip_entry_buffer; the hunk is kept for parity with the other scratch buffers and is untested.

Validation (this Mac, M-series, Metal)

• test_loader_scratch_runtime.py 2 OK; test_fp8_runtime.py 9 OK; test_graph_split_runtime.py 2 OK; test_runtime_api.py 1 OK.
• SD_REQUIRE_METAL_GENERATION=1 python3 -m unittest discover -s tests: 121 tests OK, 8 skipped (e.g. Linux and Dart-gated ones).
• validate_artifacts.py ok for macos-arm64 and ios-arm64. The patched ios-arm64 dylib imports _vm_allocate and _vm_deallocate, and not _posix_memalign or ___cxa_atexit.
• Not run: on-device iOS 18 footprint (the footprint table is macOS); needs device evidence after release per AGENTS.md.