From bf8021af249a052950fdc9d8cb1fbc6b3b8b8561 Mon Sep 17 00:00:00 2001 From: 1bcMax Date: Wed, 18 Nov 2026 22:04:59 -0600 Subject: [PATCH] =?UTF-8?q?docs(chat):=20Claude=20prompt=20caching=20on=20?= =?UTF-8?q?chat=20completions=20=E2=80=94=20caller=20cache=5Fcontrol=20and?= =?UTF-8?q?=20pricing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The page said cache_control blocks in message content were honored; until BlockRunAI/blockrun#1132 they were dropped before reaching Claude. Describes the part-level marker, prompt_cache, the limit of 4, and how exact and upto price cache writes and reads. Merge after blockrun#1132 is live. --- docs/api-reference/chat-completions.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/api-reference/chat-completions.md b/docs/api-reference/chat-completions.md index f553a8d..389c232 100644 --- a/docs/api-reference/chat-completions.md +++ b/docs/api-reference/chat-completions.md @@ -145,8 +145,12 @@ The model-swap headers above are listed in `Access-Control-Expose-Headers` whene - **Claude native `/v1/messages`**: `usage.input_tokens`, `usage.output_tokens`, `usage.cache_creation_input_tokens`, `usage.cache_read_input_tokens` - **OpenAI-compat `/v1/chat/completions`**: `usage.prompt_tokens_details.cached_tokens` (reads), `usage.prompt_tokens_details.cached_creation_tokens` (writes), `usage.completion_tokens_details.reasoning_tokens` (reasoning) -**Enabling prompt caching on Anthropic models:** -Pass `"prompt_cache": true` in the request body, or embed `cache_control` blocks in your message content directly — both are honored. +**Enabling prompt caching on Anthropic models** (every Claude model except the Fable family): +- Put `"cache_control": {"type": "ephemeral"}` on the content part that ends the prefix you resend: a `text` part of a system, user, assistant or tool message, or an `image_url` part. Claude caches the prompt up to that part. At most 4 per request (more is a 400 before payment); `ttl` is ignored (5-minute cache); message-level and tool-definition markers are ignored. +- Or pass `"prompt_cache": true` and the gateway places breakpoints for you (end of the system prompt and the last turns). It adds none when your request carries its own markers. +- Prefixes shorter than the model's minimum cacheable length are simply not cached. + +Pricing: a request that writes a cache (your markers, or `prompt_cache`) is quoted with the whole estimated prompt at the model's cache-write rate, because the payment is signed before anyone knows whether the cache will be written or read. With `upto` (non-streaming Claude), the settlement then charges cache reads at the cache-read rate and cache writes at the cache-write rate, on the share of the prompt the upstream reports for each, plus the 5% platform fee. With `exact` (every stream), the quoted amount is what you pay. :::note{title="No phantom identity tokens in your billed input"} BlockRun does **not** prepend a hidden identity/system directive to your prompt by default. The `input_tokens` (`prompt_tokens`) you are billed for reflect exactly the messages you sent — there are no extra phantom input tokens added by the gateway.