Repository navigation
docs(models): output budget passed through as sent, no reasoning floor - #200
Merged
Merged
Conversation
…loor The platform no longer raises a deepseek-v4-flash output budget below 16384 while reasoning is on (owner decision 2026-10-19). max_tokens, max_completion_tokens and max_output_tokens are now used exactly as sent on every model and route. - deepseek-v4-flash card (en + es): drop the floor and the "prompt within 16384 tokens of the window is rejected" sentence; say reasoning counts inside the budget, how a reasoning-exhausted answer looks (empty content, finish_reason "length", reasoning_tokens == completion_tokens) and the remedies (reasoning off, lower effort, generous budget plus streaming). - Controlling reasoning (en + es): same guidance, linking the existing long-outputs streaming section instead of duplicating it. - examples (en + es): replace the "max_tokens >= 300" advice. - OpenAPI max_tokens description: pass-through contract. - Tests pin the new contract and assert the floor wording is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QA follow-up on the output-budget pass-through docs: - The budget is used as sent on every model except glm5.3-flash, where a value above 131072 is lowered to 131072 (its upstream limit). Said in the reasoning section (en + es) and the OpenAPI max_tokens description, whose generic-400 rule now names that exception. - mimo-v2.6-flash card (en + es): replace "max_tokens ≥ 300" with "reasoning counts inside max_tokens"; the floor regex now also catches ≥ / &ge;. - Reasoning location scoped per route (reasoning_content on Chat Completions, a reasoning output item on Responses). - examples.md paragraph under the mimo example made model-generic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured 2026-10-19 on deepseek-v4-flash (dense prompt, budget 16384): none reasons 0 tokens; minimal 13k and still hits length; low 11k; high spends all 16384 with empty content. So: - reasoning off is the reliable fix, named per route (reasoning_effort on Chat Completions, reasoning.effort on Responses); GLM cannot turn it off (minimal is its shortest). - a lower effort is a hint on deepseek-v4-flash and qwen3.8-flash, not a cap; only qwen3.6/gemma4 cap it, on Chat Completions. - finish_reason "length" and completion_tokens_details are scoped to Chat Completions; on Responses only the reasoning output item is promised. Applied to the reasoning section and deepseek-v4-flash card (en + es) and the OpenAPI max_tokens description, with tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…soning off
Non-blocking review follow-ups:
- Responses signal measured live: status "incomplete" with
incomplete_details.reason "max_output_tokens", only a reasoning output
item, output_tokens_details.reasoning_tokens == output_tokens. In the
reasoning section; the deepseek-v4-flash card says it briefly.
- Bold lead "Empty answer with finish_reason "length"?" (en + es).
- Note that with reasoning off the answer can still stop at the budget.
- OpenAPI max_output_tokens: pass-through, glm5.3-flash exception, points
to max_tokens.
- Neutral wording for the glm5.3-flash ceiling ("the model's maximum").
- Drop the now-duplicated "its tokens count toward max_tokens".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nan-devops#902 lowers any deepseek-v4-flash output budget above 393216 to 393216 (DeepSeek's own maximum), on max_tokens, max_completion_tokens and max_output_tokens. The pass-through contract now names both per-model ceilings (131072 on glm5.3-flash, 393216 on deepseek-v4-flash) in the reasoning section (en + es), the deepseek-v4-flash card and the OpenAPI max_tokens / max_output_tokens descriptions. Tests updated first; the Responses-signals test is renamed to what it checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolved into one text per surface: - deepseek-v4-flash card (en + es): #200's wording (both ceilings, per-route signals, remedies) plus #202's non-streamed x-nan-truncated-in-reasoning header on the /chat/completions signal. #202's voseo phrasing and "raise the budget, or stream" are superseded. - Controlling reasoning: #200's paragraph keeps #202's three-field rule and header (scoped to non-streamed Chat Completions, per nan-devops#901, which reads `choices`); #201's token-budget subsection is linked from the "effort is a hint" sentence. - OpenAPI max_tokens: #200's text plus the header. max_output_tokens: #202 claimed finish_reason and the header on Responses; replaced by the Responses signal (status incomplete, incomplete_details) and the ceilings. - Tests: union of both suites, adjusted to the final text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The description pointed Responses users at max_tokens' remedy, which uses
the Chat Completions form (reasoning_effort). Responses needs
reasoning: {"effort": "none"}; say it in place, with the GLM caveat.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Builds on #201 (reasoning token budget) and #202 (budget used as sent,
x-nan-truncated-in-reasoningheader), merged into this branch with one coherent text per surface (details below).The docs stop describing the
deepseek-v4-flashoutput-budget floor (a budget below 16384 silently raised while reasoning is on) and document the new contract: the output budget (max_tokens/max_completion_tokenson/v1/chat/completions,max_output_tokenson/v1/responses) is used exactly as sent on every model, except above the model's maximum on two models, where it is lowered to that maximum: 131072 onglm5.3-flash, 393216 ondeepseek-v4-flash(nan-devops#902).models.mdx): floor text removed, including the sentence saying a prompt within 16384 tokens of the window is rejected even with a smallmax_tokens. Now says reasoning counts inside the budget, how a reasoning-exhausted answer looks on each route, and the remedies.finish_reason: "length"?":content,finish_reason: "length", reasoning inmessage.reasoning_content,usage.completion_tokens_details.reasoning_tokens == completion_tokens.x-nan-truncated-in-reasoning: true(nan-devops#901; it readschoices, so it is not promised on Responses).status: "incomplete",incomplete_details.reason: "max_output_tokens", only areasoningoutput item,usage.output_tokens_details.reasoning_tokens == output_tokens.reasoning_effort: "none"on Chat Completions,reasoning: {"effort": "none"}on Responses); GLM cannot turn it off (minimalis its shortest). A lower effort is a hint, not a cap, ondeepseek-v4-flashandqwen3.8-flash(measured:minimalstill hit the limit on a dense prompt); onlyqwen3.6/gemma4cap it, on Chat Completions; links docs(models): document the reasoning token budget #201's token-budget subsection for a hard cap. With reasoning off the answer can still stop at the budget, so the budget must fit the answer. To keep the reasoning: a generous budget plus streaming, linking the existing "Long outputs: stream them" section (120 s / 524) instead of duplicating it.max_tokens ≥ 300" replaced by "reasoning counts insidemax_tokens".max_tokens ≥ 300" advice with model-generic guidance.src/data/openapi.json):max_tokensdescribes the pass-through contract, both per-model ceilings (including in the generic-400 rule) and the remedies;max_output_tokensstates the ceilings and the Responses signal (docs: the output budget is used as sent; document the reason-empty signal #202's claim offinish_reasonand the header there was Chat-only and is replaced).Why
Owner decision 2026-10-19: remove the reasoning output-budget floor. See the internal triage note.
Tests
src/tests/lib/docsReasoning.test.ts: suite "output budget: passed through as sent, no floor (en + es)" asserts, in both locales: no floor wording or≥ 300/≥ 300advice anywhere on models or examples; both per-model ceilings (glm5.3-flash131072,deepseek-v4-flash393216); the Chat Completions and Responses signals scoped per route (nofinish_reason/completion_tokens_detailspromised for Responses);noneas the reliable fix on both routes, effort as a hint, the GLM caveat; the bold lead; the reasoning-off budget note; the mimo card; model-generic examples; link anchors exist. The specmax_tokenstest asserts the pass-through text.src/lib/openapiSpec.test.ts: floor test replaced by a pass-through test (specmax_tokensandmax_output_tokens, both cards, negative asserts on the floor wording).npm test: 87 files, 2516 tests passed.npm run build: complete, exit 0.Merge with #201/#202
/chat/completionssignal; docs: the output budget is used as sent; document the reason-empty signal #202's card phrasing (voseo, "we never raise it", "Raise the budget, or stream") superseded by one phrasing.max_tokens: docs(models): output budget passed through as sent, no reasoning floor #200 text + header.max_output_tokens: Responses signal instead of the Chat-only header/finish_reasonclaim.Deploy notes
max_tokens=256with reasoning on returns 256 tokens.max_completion_tokenstomax_tokenswhenmax_tokensis absent (nan-devops#902; DeepSeek's API ignoresmax_completion_tokens): deployed and verified live,deepseek-v4-flashwithmax_completion_tokens=256returns 256.deepseek-v4-flash393216 output ceiling (nan-devops#902): deployed and verified live,max_completion_tokens=1000000is served (lowered), not rejected.main.🤖 Generated with Claude Code