Repository navigation
fix(server-utils): Count Google GenAI tool-use prompt tokens as input tokens - #25834
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
… tokens Gemini reports the tokens of built-in tool results (Google Search, URL context, code execution) in usageMetadata.toolUsePromptTokenCount, apart from promptTokenCount, while totalTokenCount includes both. gen_ai.usage.input_tokens only took promptTokenCount, so input + output did not add up to the total. Add the tool-use prompt tokens to the input count for responses and streams. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
pcbeingused333
requested review from
andreiborza and
isaacs
and removed request for
a team
October 28, 2026 15:41
Member
|
Is there an issue that was raised before you opened this PR? Please do not blindly contribute PRs without discussing the problem first. Also, from the description, it is not clearly visible if you actually experienced the problem or just told an agent to look for bugs in our code base. See our contributing guidelines.
That's fine but we expect from you that you 1. write the description yourself and 2. fully understand the code change. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gemini reports the tokens of built-in tool results fed back to the model (Google Search, URL context, code execution) in
usageMetadata.toolUsePromptTokenCount, apart frompromptTokenCount.totalTokenCountincludes both ("Total token count for prompt, response candidates, and tool-use prompts (if present)" intypes.ts).addResponseAttributesand the stream'shandleResponseMetadataonly usedpromptTokenCountforgen_ai.usage.input_tokens, so with those toolsinput_tokens + output_tokensdid not add up togen_ai.usage.total_tokens.The tool-use prompt tokens are now added to the input count, for a response and for a stream (the two counts are tracked separately, so a chunk that repeats only one of them keeps the other). Responses without tool-use tokens are unchanged.
Tests in
google-genai-streaming.test.tscover a response and a stream withtoolUsePromptTokenCount(input 140, total 160) and a response without it. The two with tool-use tokens fail ondevelop.This change was made with Claude Code.