[NVolcz]

normalize_messages removes reasoning_content from every assistant message. When a model reasons, calls a tool and goes on from the result, its reasoning is gone from the prompt by the time it reads the result. The Gemma 4 docs say:

If a single model turn involves function or tool calls, thoughts must NOT be removed between the function calls.

Before, for a request that sends the reasoning back with the tool call:

<|turn>model
<|tool_call>call:find_person{name:<|"|>Alice<|"|>}<tool_call|><|tool_response>response:find_person{...}<tool_response|>

After:

<|turn>model
<|channel>thought
...the model's reasoning...
<channel|><|tool_call>call:find_person{name:<|"|>Alice<|"|>}<tool_call|><|tool_response>response:find_person{...}<tool_response|>

Reasoning is now kept for the assistant messages after the last user message and removed from earlier ones, as before. The Gemma 4 and Qwen3 templates render it only for those messages. Clients that send it as reasoning or thinking instead of reasoning_content have it moved to reasoning_content, which the templates read.

The prompt is longer now, and the request after a tool call prefills the reasoning again. The engine already holds the reasoning it generated, but the prompt cache reuses only saved prompt prefixes, and the template also writes a newline before <channel|> that the model did not generate. In a two-step tool chain with thinking on (gemma4-it:e4b, about 240 tokens of reasoning):

Request	Before	After
After the first tool call	176 tokens, 147 cached, prefill 0.81 s	412 tokens, 147 cached, prefill 0.97 s
After the second tool call	213 tokens, 147 cached, prefill 0.82 s	449 tokens, 412 cached, prefill 0.81 s

Each tool step pays once for its reasoning. The next request can then reuse the longer prompt, because it is now large enough to be saved as a prefix.

Once a new user message follows, the template drops the earlier turn's reasoning again, so the new prompt differs from the cached one from the first tool call on. With prefix slots (the default on NPU) the follow-up in the test above reused the same 147 tokens as before. Where prefix slots are off, on OGA backends and for Gemma 4 conversations with images or audio, the cache is cleared on any difference, so that request prefills the whole conversation again; this is not measured.

The docs now tell clients to send the reasoning back, and test_tool_calls.reasoning_kept_within_turn checks it with prompt_tokens.

How to reproduce

Send a tool call with its result, once without and once with reasoning_content on the tool call, and compare prompt_tokens:

for reasoning in '' ', "reasoning_content": "The user wants the weather in Paris, so I call get_weather."'; do
curl -s http://localhost:52625/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "gemma4-it:e4b", "max_tokens": 1,
  "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object",
    "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
  "messages": [
    {"role": "user", "content": "What is the weather in Paris?"},
    {"role": "assistant", "content": null, "tool_calls": [{"id": "call_1", "type": "function",
      "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}}]'"$reasoning"'},
    {"role": "tool", "tool_call_id": "call_1", "content": "{\"temperature_c\": 23}"}
  ]}' | jq '.usage.prompt_tokens'
done

On main both requests report the same count, because the reasoning is removed. On this branch the second one is larger by the reasoning's tokens.

I found this while comparing the prompts flm renders with Google's template, using a small debug branch that logs every prompt to a file (log-prompt, FLM_LOG_PROMPT=<file>). It also shows the rendered prompt directly, if you want to see the reasoning in it.