[ericcurtin]

What this PR does

Adds llmman (https://github.com/llmmanorg/llmman) as an alternative backend for the top-level models element, selected explicitly with COMPOSE_MODEL_PROVIDER=llmman (shell or .env). It is also used automatically when COMPOSE_MODEL_PROVIDER is unset, the docker model plugin is not installed and llmman is on PATH (debug log only). Docker Model Runner always wins when its plugin is installed, and an explicit docker-model-runner never falls back. Any other value is an error.

Related issue

No linked issue yet: this is a proposal and I'm aware CONTRIBUTING.md asks for a status/approved issue for AI-assisted PRs. Happy to open an issue and discuss the design there if maintainers prefer.

Changes made

• modelAPI becomes a small internal interface (Check, ListModels, HasModel, PullModel, ConfigureModel, SetModelVariables, Close). The existing code is dmrModelAPI, with the same commands and errors. The model/endpoint variable injection is shared by both providers.
• New pkg/compose/model_llmman.go, driving the llmman CLI with the project environment:
  • llmman ps verifies the daemon first (never starts it; actionable error otherwise).
  • llmman list --format finds models already stored, and llmman itself resolves references, so only missing models are pulled with llmman pull.
  • context_size and runtime_flags have no per-model equivalent in llmman, so they are ignored with a warning event.
  • Services get <NAME>_MODEL and <NAME>_URL (or model_var / endpoint_var), where the URL is the OpenAI base URL built from LLMMAN_HOST (default http://host.docker.internal:17434/v1).
• Dry-run only runs the read-only ps and list commands.
• Usage notes are in the doc comment of model_llmman.go; there is no hand-written models doc to update.

Testing done

• Unit tests pass: go test ./pkg/... (new fake-llmman tests plus the unchanged Docker Model Runner dry-run test)
• E2E tests pass: not run
• Linting passes on ./pkg/compose/... with golangci-lint (3 remaining nolintlint reports are in files this PR does not touch)
• Code formatted: gofmt, gofumpt/gci through golangci-lint
• Manually tested: real llmman 0.1.567 and a Docker engine on Linux with a throwaway daemon. Pulled docker.io/ai/smollm2:135M-Q2_K and hf.co/unsloth/SmolLM2-135M-Instruct-GGUF:Q2_K through docker compose create, re-ran with nothing to pull, checked the container env vars, --dry-run, daemon down, missing binary and a bad provider value. Not tested: a container calling the daemon, inference, Docker Desktop, API-key auth or TLS.

AI tool used (if applicable)

AI Tool: opencode with anthropic/claude-sonnet-5-5 (high variant), supervised by me. The code, tests and this description were drafted by the agent, and I have reviewed the change (AI_AGENT_DISCLOSURE.md removed from the commit).

Additional context

Behaviour worth knowing

• llmman resolves ai/<name> as a Hugging Face repo, not Docker Hub, unlike Docker Model Runner. Use <name> or docker.io/ai/<name>; a failed pull of an ai/ reference says so.
• llmman binds 127.0.0.1 by default. On a Linux engine, containers need the daemon bound on a reachable address (0.0.0.0 needs LLMMAN_API_KEYS or LLMMAN_AUTH=off) and extra_hosts: host.docker.internal:host-gateway. Documented, not automated.
• ensureModels is touched (Check/HasModel hooks, a local err in the goroutine instead of the shared outer one); Docker Model Runner behaviour is the same. This may conflict with fix(models): check the model listing error and keep pull errors per goroutine #14284.

[docker-agent]

Assessment: 🟡 NEEDS ATTENTION

Lower-confidence findings (not posted inline)

• [low] pkg/compose/model_llmman.go:212 — scanner.Err() not checked after stdout-scanning loop in PullModel (confidence: weak 38/100)
• [low] pkg/compose/model_llmman.go:161 — llmman list <model> --format flag after positional arg may be unsupported by some CLI frameworks (confidence: weak 34/100)
• [low] pkg/compose/model.go:57 — ListModels error silently ignored; could mask backend failures (confidence: weak 38/100)

[codecov]

Codecov Report

❌ Patch coverage is 98.41270% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/compose/model.go	96.72%	2 Missing ⚠️
pkg/compose/model_llmman.go	99.21%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[ndeloof]

IMHO compose is not the right place to select a model runtime. Docker Model Runner has been design to abstract this away, so that model reference is the only thing user has to provide. Better request DMR to support this alternative runtime

[ericcurtin]

IMHO compose is not the right place to select a model runtime. Docker Model Runner has been design to abstract this away, so that model reference is the only thing user has to provide. Better request DMR to support this alternative runtime

That's not how this works. llmman is a replacement for DMR, like a DMR 2.0, DMR is not an llmman wrapper.