From 5f5a90ab8fab0e81c42c5c1a736212e471cec311 Mon Sep 17 00:00:00 2001
From: Karthik <karthiksoma95@gmail.com>
Date: Wed, 28 Oct 2026 23:42:21 -0400
Subject: [PATCH] docs: explain shared Llama instance concurrency

---
 README.md | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/README.md b/README.md
index c02df9eed..934f6abde 100644
--- a/README.md
+++ b/README.md
@@ -344,6 +344,44 @@ By default `llama-cpp-python` generates completions in an OpenAI compatible form
 
 Text completion is available through the [`__call__`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__call__) and [`create_completion`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.create_completion) methods of the [`Llama`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama) class.
 
+### Sharing a model between threads
+
+A `Llama` instance holds mutable inference state and is not safe for concurrent inference calls.
+Overlapping calls on the same instance can crash the process in native code; a Python `try`/`except` cannot catch a native abort.
+If multiple threads share an instance, use the same lock around every inference operation on it:
+
+```python
+from threading import Lock
+from llama_cpp import Llama
+
+llm = Llama(model_path="path/to/embedding-model.gguf", embedding=True)
+model_lock = Lock()
+
+def embed_text(text):
+    with model_lock:
+        return llm.embed(text)
+```
+
+This serializes calls; it does not perform parallel inference.
+For a text-generation model, hold the lock while consuming and closing the stream, not only while creating it:
+
+```python
+from contextlib import closing
+from threading import Lock
+from llama_cpp import Llama
+
+llm = Llama(model_path="path/to/generation-model.gguf")
+model_lock = Lock()
+
+with model_lock:
+    with closing(llm.create_completion("Hello", stream=True)) as stream:
+        for chunk in stream:
+            print(chunk["choices"][0]["text"], end="", flush=True)
+```
+
+Use this pattern in synchronous worker threads.
+Do not hold a blocking `threading.Lock` across `await` calls in an async application.
+
 ### Pulling models from Hugging Face Hub
 
 You can download `Llama` models in `gguf` format directly from Hugging Face using the [`from_pretrained`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.from_pretrained) method.
@@ -654,6 +692,8 @@ llama = Llama(
 
 ### Embeddings
 
+If multiple threads share your embedding model, follow [Sharing a model between threads](#sharing-a-model-between-threads).
+
 To generate text embeddings use [`create_embedding`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.create_embedding) or [`embed`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.embed). Note that you must pass `embedding=True` to the constructor upon model creation for these to work properly.
 
 ```python