Skip to content

Narrow the 12288 output cap to the DashScope route - #3201

Open
HugoXK wants to merge 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/5-outbound-cap-per-provider
Open

HugoXK wants to merge 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/5-outbound-cap-per-provider

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #3198

Written-by: Dev-5
Owner: Dev-5
Bot: Dev-5
Tier: T2
Claim-Id: ddb5ff3edda7443b9be2b7b952699d10
Branch: dev/5-outbound-cap-per-provider
Base: 0f7e82121832d4cea8e26f1e9681355100432df7

What was wrong

core/src/host/models.js held one ceiling, OUTBOUND_MAX_TOKENS = 12_288, and the llm/stream listener applied it to every provider: a caller asking for more was restarted with 12288, and a model row declaring more was sent 12288. The comment above the constant says whose limit it is — DashScope's 400 (max_completion_tokens … expected a value <= 12288, #3132) — but the code had no provider test, so a route whose endpoint accepts more was silently cut.

The fix

The clamp now runs only on the route that produced that 400. The predicate is named beside the constant with its evidence: DASHSCOPE_ROUTE = 'opencsitool' — the DSH route key #3132 recorded, whose proxy answered litellm.BadRequestError: DashscopeException … expected a value <= 12288 for opencsitool/DeepSeek-V4.1-Flash-1017. On every other route the listener calls next() straight away, so the provider receives the caller's maxTokens, or DSH fills the cap the model row declares. No cap is raised or invented.

The limit of that evidence: it is a route-level test, because the provider route is what listProviders() / GenerateOptions.provider expose as { id, name } — the endpoint is fixed per route, not per model. A call with no provider key cannot be placed against the evidence, so it keeps today's clamp: a cap that is too low only shortens an answer, while a 400 loses it whole (the failure #3132 recorded).

DashScope behaviour is unchanged and still pinned: a row declaring more is asked for 12288, a caller asking for more is sent 12288, the caller's object is never written to, and the second ctx.llm.stream pass carries 12288.

Red first

On base 0f7e82121832d4cea8e26f1e9681355100432df7, with the new cases only:

cd core && TZ=UTC TMPDIR=/tmp node --import ./test/harness-resolve.mjs --test test/llm-output-cap.test.js
not ok 2 - a model row declaring more than the ceiling is left alone on a route whose endpoint accepts it
  error: 'the declared default is filled by DSH, not written here'
  expected: false
  actual: true
not ok 5 - a caller asking for more than the ceiling keeps its value on a route whose endpoint accepts it
  error: 'the caller's request reaches the provider unchanged'
  expected: 20000
  actual: 12288
# tests 9
# pass 7
# fail 2

The two existing provider cases (:51, :66) keep their assertions as the DashScope half of a split; the other four are untouched.

Verification at this tip

  • core/test/llm-output-cap.test.js — # tests 9 / # pass 9 / # fail 0
  • node scripts/ds-bot-patches.mjs check — ok 10 patches apply cleanly in order; canonical — ok patches/ds-bot/ is what generate writes
  • TZ=UTC npm run verify — exit 0; 10 suites, 0 failures in each: 3345 tests, 20, 717, 149, 172, 538, 151, 686, 37, 34; arch-docs: OK; ok 10 patches apply cleanly in order; mojibake scan 0 hits.
  • CI checks at this tip — 19 total: 16 success + 2 skipped + 1 failure; skipped are live · ubuntu · Node 24 · real model and window check, the one failure is enqueue-guard (no Verifier report names the tip ff38bd47…), the expected red while the T2 PASS is pending.

Boundaries

  • core/src/host/models.js changes only the predicate around the constant and the listener's first line; the clamp value, both branches and their comments are otherwise untouched.
  • patches/ds-bot/0001-core-fixes.patch and patches/ds-bot/files/test/llm-output-cap.test.js come from generate; nothing is hand-edited.
  • No UI surface: host request shaping only.
  • No gate files, .github/**, core/client.js, patches/ds-bot/0008 or i18n.js touched.
  • If #3195/#3197 land first, this branch rebases and re-runs generate rather than hand-merging the patch.

Checkpoint

{
  "pr": 3201,
  "claim": "Dev-5",
  "issue": 3198,
  "summary": "The 12288 output cap encoded one endpoint's 400 but was applied to every provider, silently cutting answers on routes that accept more. The llm/stream listener now clamps only on the opencsitool route that #3132 recorded, named beside the constant with that evidence, and passes every other route through so the caller's request or the model row's declared cap reaches the provider. A route with no provider key keeps the clamp, because a shorter answer is recoverable and a 400 is not. DashScope behaviour and the copy-not-mutate rule are unchanged.",
  "files": ["core/src/host/models.js", "core/test/llm-output-cap.test.js", "patches/ds-bot/0001-core-fixes.patch", "patches/ds-bot/files/test/llm-output-cap.test.js", "docs/subsystems/core-host.md", "changelog.d/3198.fixed.md"],
  "checks": { "ran": "TZ=UTC npm run verify; cd core && TZ=UTC node --import ./test/harness-resolve.mjs --test test/llm-output-cap.test.js; node scripts/ds-bot-patches.mjs check; node scripts/ds-bot-patches.mjs canonical", "result": "green", "at": "ff38bd47701c8e6b011f0f0b3b42b1ebba416bed" },
  "next": "none — ready for review"
}

The ceiling encodes one endpoint's 400: DashScope, reached through the
`opencsitool` route that ThinkFlowLab#3132 recorded, refused `max_completion_tokens`
above 12288. The listener applied it to every provider, so a call on a
route that accepts more was silently cut to 12288.

The clamp now tests that route key, named beside the constant with its
evidence; every other route takes the caller's request or its model
row's declared cap untouched, and a route with no provider key keeps
the clamp because a short answer is recoverable while a 400 is not.
@HugoXK
HugoXK marked this pull request as ready for review November 19, 2026 01:48
@HugoXK

HugoXK commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy review request — T2, one request at this tip.

  • tip ff38bd47701c8e6b011f0f0b3b42b1ebba416bed (base 0f7e82121832d4cea8e26f1e9681355100432df7), 6 files, +100/−13, draft: false, maintainer_can_modify: true.
  • checks at this tip: 22 total = 19 success + 2 skipped + 1 failure; skipped are live · ubuntu · Node 24 · real model and window check; the one failure is enqueue-guard — no Verifier report names the tip ff38bd47…, i.e. the expected red while the T2 Verdict is pending. tests-ok, merge-gate, judge, policy-sign are success.
  • what it changes: core/src/host/models.js clamped maxTokens to 12_288 on every provider although that number came from one endpoint's 400 (#3132). The clamp is now predicated on that route (opencsitool), named beside the constant with the evidence, and every other route passes through untouched. DashScope behaviour is pinned unchanged; a call with no provider key keeps the clamp (a short answer beats a lost one).
  • no UI surface: host request shaping only, so no screenshots apply.
  • red-first, the full verify reading and the boundaries are in the body.

A Verdict naming this exact tip is what the enqueue needs.

@HugoXK

HugoXK commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Independent reproduction (advisory, no request attached).

A second engineer re-ran the defect on base 0f7e8212 with a probe of their own (no reuse of this branch's cases, no fake-ctx.js), driving the llm/stream waterfall by its documented semantics:

{"label":"openai route, declares 32000, asks 20000","sentCap":12288,"providerRequests":1}
{"label":"anthropic route, declares 32000, asks nothing","sentCap":12288,"providerRequests":1}
{"label":"openai route, declares 32000, asks 8000","sentCap":8000,"providerRequests":1}
{"label":"openai route, declares 8000, asks nothing","sentCap":"(absent)","providerRequests":1}
VERDICT: the defect HOLDS — a non-DashScope route declaring 32000 is asked for 12288, and one asking 20000 is sent 12288

Two independent routes (not one named provider) hit the global clamp, a request under the ceiling passes through untouched, and each call reaches the provider once — so the reading is about the missing provider predicate, not about a particular endpoint name.

It also confirms the shape of the fix: narrowing the clamp to the route that produced the 400 leaves every other route's request alone, which is what this branch does.

@github-actions github-actions Bot added the risk:T2 Risk tier T2 (host/plugin code or an unnamed path), set by the controller from the diff label Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:T2 Risk tier T2 (host/plugin code or an unnamed path), set by the controller from the diff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

models: the 12288 output cap is applied to every provider, not to the DashScope endpoint whose 400 it encodes

1 participant