Fix LlamaLanguageModel build against current llama.swift - #193
Conversation
The open-ended dependency range resolves llama.swift to releases wrapping current llama.cpp builds, where the Llama trait no longer compiles: llama_sampler_init_penalties regained its leading n_vocab parameter, and llama_model_params replaced use_mmap and use_mlock with a llama_load_mode enum. Pass the vocabulary size at all three penalties call sites and set load_mode to LLAMA_LOAD_MODE_MMAP, matching the previous mmap-only behavior. Verified against llama.swift 2.10549.0 with the full live test suite.
|
Merging now. Thanks, @james-333i! |
|
Hi @james-333i. Thank you for all of these. Porting the fixes back from your fork is exactly the kind of help this project needed, and I'm sorry they sat for a week without CI even running. I've merged #199, #201, #202, and #204 after verifying them locally. #200 picked up a conflict with #199 in |
The macOS legs build with the MLX, Llama, and CoreML traits and take 6-8 minutes on a cold cache. The first green run on main after #193/#207 had one job cancelled at the 10-minute limit while saving the SwiftPM cache, after Lint, Build, and Test had all passed. Because the save is cut off, the cache never populates and every run stays cold.
2 participants
Current llama.swift releases changed the signature of
llama_sampler_init_penalties, which takes the vocabulary size as its first argument again. This updates the calls accordingly so the Llama trait builds. The other Llama PRs I am submitting include this commit as their base, so merging this one first collapses them to single-purpose diffs.