Skip to content

docs(models): output budget passed through as sent, no reasoning floor - #200

Merged
sre-helmcode merged 7 commits into
mainfrom
docs/drop-output-floor
Oct 19, 2026
Merged

sre-helmcode merged 7 commits into
mainfrom
docs/drop-output-floor

Conversation

@sre-helmcode

@sre-helmcode sre-helmcode commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Builds on #201 (reasoning token budget) and #202 (budget used as sent, x-nan-truncated-in-reasoning header), merged into this branch with one coherent text per surface (details below).

The docs stop describing the deepseek-v4-flash output-budget floor (a budget below 16384 silently raised while reasoning is on) and document the new contract: the output budget (max_tokens / max_completion_tokens on /v1/chat/completions, max_output_tokens on /v1/responses) is used exactly as sent on every model, except above the model's maximum on two models, where it is lowered to that maximum: 131072 on glm5.3-flash, 393216 on deepseek-v4-flash (nan-devops#902).

  • deepseek-v4-flash card (EN + ES, models.mdx): floor text removed, including the sentence saying a prompt within 16384 tokens of the window is rejected even with a small max_tokens. Now says reasoning counts inside the budget, how a reasoning-exhausted answer looks on each route, and the remedies.
  • Controlling reasoning section (EN + ES): a paragraph led by "Empty answer with finish_reason: "length"?":
    • Chat Completions signal: empty content, finish_reason: "length", reasoning in message.reasoning_content, usage.completion_tokens_details.reasoning_tokens == completion_tokens.
    • The three Chat Completions facts together are the rule (from docs: the output budget is used as sent; document the reason-empty signal #202); non-streamed Chat Completions calls also carry x-nan-truncated-in-reasoning: true (nan-devops#901; it reads choices, so it is not promised on Responses).
    • Responses signal (measured live): status: "incomplete", incomplete_details.reason: "max_output_tokens", only a reasoning output item, usage.output_tokens_details.reasoning_tokens == output_tokens.
    • Reliable fix: reasoning off (reasoning_effort: "none" on Chat Completions, reasoning: {"effort": "none"} on Responses); GLM cannot turn it off (minimal is its shortest). A lower effort is a hint, not a cap, on deepseek-v4-flash and qwen3.8-flash (measured: minimal still hit the limit on a dense prompt); only qwen3.6 / gemma4 cap it, on Chat Completions; links docs(models): document the reasoning token budget #201's token-budget subsection for a hard cap. With reasoning off the answer can still stop at the budget, so the budget must fit the answer. To keep the reasoning: a generous budget plus streaming, linking the existing "Long outputs: stream them" section (120 s / 524) instead of duplicating it.
  • mimo-v2.6-flash card (EN + ES): "recommended max_tokens ≥ 300" replaced by "reasoning counts inside max_tokens".
  • examples.md (EN + ES): replaced the "max_tokens ≥ 300" advice with model-generic guidance.
  • OpenAPI (src/data/openapi.json): max_tokens describes the pass-through contract, both per-model ceilings (including in the generic-400 rule) and the remedies; max_output_tokens states the ceilings and the Responses signal (docs: the output budget is used as sent; document the reason-empty signal #202's claim of finish_reason and the header there was Chat-only and is replaced).

Why

Owner decision 2026-10-19: remove the reasoning output-budget floor. See the internal triage note.

Tests

  • Union of this PR's and docs(models): document the reasoning token budget #201/docs: the output budget is used as sent; document the reason-empty signal #202's asserts, adjusted to the merged text (header asserts kept; voseo/"never raise" wording superseded).
  • src/tests/lib/docsReasoning.test.ts: suite "output budget: passed through as sent, no floor (en + es)" asserts, in both locales: no floor wording or ≥ 300 / ≥ 300 advice anywhere on models or examples; both per-model ceilings (glm5.3-flash 131072, deepseek-v4-flash 393216); the Chat Completions and Responses signals scoped per route (no finish_reason / completion_tokens_details promised for Responses); none as the reliable fix on both routes, effort as a hint, the GLM caveat; the bold lead; the reasoning-off budget note; the mimo card; model-generic examples; link anchors exist. The spec max_tokens test asserts the pass-through text.
  • src/lib/openapiSpec.test.ts: floor test replaced by a pass-through test (spec max_tokens and max_output_tokens, both cards, negative asserts on the floor wording).
  • Tests were written first and failed against the previous text on each iteration.
  • npm test: 87 files, 2516 tests passed. npm run build: complete, exit 0.

Merge with #201/#202

Deploy notes

  • All prerequisites are live in prod (2026-10-19), so this can merge:
    1. Floor removal (nan-devops#898): deployed and verified live, max_tokens=256 with reasoning on returns 256 tokens.
    2. Copying max_completion_tokens to max_tokens when max_tokens is absent (nan-devops#902; DeepSeek's API ignores max_completion_tokens): deployed and verified live, deepseek-v4-flash with max_completion_tokens=256 returns 256.
    3. The deepseek-v4-flash 393216 output ceiling (nan-devops#902): deployed and verified live, max_completion_tokens=1000000 is served (lowered), not rejected.
  • No VERSION file in this repo; deploy runs on push to main.

🤖 Generated with Claude Code

barckcode and others added 7 commits October 19, 2026 12:06
…loor

The platform no longer raises a deepseek-v4-flash output budget below
16384 while reasoning is on (owner decision 2026-10-19). max_tokens,
max_completion_tokens and max_output_tokens are now used exactly as sent
on every model and route.

- deepseek-v4-flash card (en + es): drop the floor and the "prompt within
  16384 tokens of the window is rejected" sentence; say reasoning counts
  inside the budget, how a reasoning-exhausted answer looks (empty content,
  finish_reason "length", reasoning_tokens == completion_tokens) and the
  remedies (reasoning off, lower effort, generous budget plus streaming).
- Controlling reasoning (en + es): same guidance, linking the existing
  long-outputs streaming section instead of duplicating it.
- examples (en + es): replace the "max_tokens >= 300" advice.
- OpenAPI max_tokens description: pass-through contract.
- Tests pin the new contract and assert the floor wording is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QA follow-up on the output-budget pass-through docs:
- The budget is used as sent on every model except glm5.3-flash, where a
  value above 131072 is lowered to 131072 (its upstream limit). Said in the
  reasoning section (en + es) and the OpenAPI max_tokens description, whose
  generic-400 rule now names that exception.
- mimo-v2.6-flash card (en + es): replace "max_tokens &ge; 300" with
  "reasoning counts inside max_tokens"; the floor regex now also catches
  &ge; / &amp;ge;.
- Reasoning location scoped per route (reasoning_content on Chat
  Completions, a reasoning output item on Responses).
- examples.md paragraph under the mimo example made model-generic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured 2026-10-19 on deepseek-v4-flash (dense prompt, budget 16384):
none reasons 0 tokens; minimal 13k and still hits length; low 11k; high
spends all 16384 with empty content. So:
- reasoning off is the reliable fix, named per route (reasoning_effort on
  Chat Completions, reasoning.effort on Responses); GLM cannot turn it off
  (minimal is its shortest).
- a lower effort is a hint on deepseek-v4-flash and qwen3.8-flash, not a
  cap; only qwen3.6/gemma4 cap it, on Chat Completions.
- finish_reason "length" and completion_tokens_details are scoped to Chat
  Completions; on Responses only the reasoning output item is promised.
Applied to the reasoning section and deepseek-v4-flash card (en + es) and
the OpenAPI max_tokens description, with tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…soning off

Non-blocking review follow-ups:
- Responses signal measured live: status "incomplete" with
  incomplete_details.reason "max_output_tokens", only a reasoning output
  item, output_tokens_details.reasoning_tokens == output_tokens. In the
  reasoning section; the deepseek-v4-flash card says it briefly.
- Bold lead "Empty answer with finish_reason "length"?" (en + es).
- Note that with reasoning off the answer can still stop at the budget.
- OpenAPI max_output_tokens: pass-through, glm5.3-flash exception, points
  to max_tokens.
- Neutral wording for the glm5.3-flash ceiling ("the model's maximum").
- Drop the now-duplicated "its tokens count toward max_tokens".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nan-devops#902 lowers any deepseek-v4-flash output budget above 393216 to
393216 (DeepSeek's own maximum), on max_tokens, max_completion_tokens and
max_output_tokens. The pass-through contract now names both per-model
ceilings (131072 on glm5.3-flash, 393216 on deepseek-v4-flash) in the
reasoning section (en + es), the deepseek-v4-flash card and the OpenAPI
max_tokens / max_output_tokens descriptions. Tests updated first; the
Responses-signals test is renamed to what it checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolved into one text per surface:
- deepseek-v4-flash card (en + es): #200's wording (both ceilings,
  per-route signals, remedies) plus #202's non-streamed
  x-nan-truncated-in-reasoning header on the /chat/completions signal.
  #202's voseo phrasing and "raise the budget, or stream" are superseded.
- Controlling reasoning: #200's paragraph keeps #202's three-field rule
  and header (scoped to non-streamed Chat Completions, per nan-devops#901,
  which reads `choices`); #201's token-budget subsection is linked from
  the "effort is a hint" sentence.
- OpenAPI max_tokens: #200's text plus the header. max_output_tokens:
  #202 claimed finish_reason and the header on Responses; replaced by the
  Responses signal (status incomplete, incomplete_details) and the
  ceilings.
- Tests: union of both suites, adjusted to the final text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The description pointed Responses users at max_tokens' remedy, which uses
the Chat Completions form (reasoning_effort). Responses needs
reasoning: {"effort": "none"}; say it in place, with the GLM caveat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sre-helmcode
sre-helmcode merged commit 1c8bfe9 into main Oct 19, 2026
1 check passed
@barckcode
barckcode deleted the docs/drop-output-floor branch October 20, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants