Repository navigation
Conversation
The ceiling encodes one endpoint's 400: DashScope, reached through the `opencsitool` route that ThinkFlowLab#3132 recorded, refused `max_completion_tokens` above 12288. The listener applied it to every provider, so a call on a route that accepts more was silently cut to 12288. The clamp now tests that route key, named beside the constant with its evidence; every other route takes the caller's request or its model row's declared cap untouched, and a route with no provider key keeps the clamp because a short answer is recoverable while a 400 is not.
|
@Birfy review request — T2, one request at this tip.
A Verdict naming this exact tip is what the enqueue needs. |
|
Independent reproduction (advisory, no request attached). A second engineer re-ran the defect on base Two independent routes (not one named provider) hit the global clamp, a request under the ceiling passes through untouched, and each call reaches the provider once — so the reading is about the missing provider predicate, not about a particular endpoint name. It also confirms the shape of the fix: narrowing the clamp to the route that produced the 400 leaves every other route's request alone, which is what this branch does. |
Fixes #3198
Written-by: Dev-5
Owner: Dev-5
Bot: Dev-5
Tier: T2
Claim-Id: ddb5ff3edda7443b9be2b7b952699d10
Branch:
dev/5-outbound-cap-per-providerBase:
0f7e82121832d4cea8e26f1e9681355100432df7What was wrong
core/src/host/models.jsheld one ceiling,OUTBOUND_MAX_TOKENS = 12_288, and thellm/streamlistener applied it to every provider: a caller asking for more was restarted with 12288, and a model row declaring more was sent 12288. The comment above the constant says whose limit it is — DashScope's 400 (max_completion_tokens … expected a value <= 12288, #3132) — but the code had no provider test, so a route whose endpoint accepts more was silently cut.The fix
The clamp now runs only on the route that produced that 400. The predicate is named beside the constant with its evidence:
DASHSCOPE_ROUTE = 'opencsitool'— the DSH route key #3132 recorded, whose proxy answeredlitellm.BadRequestError: DashscopeException … expected a value <= 12288foropencsitool/DeepSeek-V4.1-Flash-1017. On every other route the listener callsnext()straight away, so the provider receives the caller'smaxTokens, or DSH fills the cap the model row declares. No cap is raised or invented.The limit of that evidence: it is a route-level test, because the provider route is what
listProviders()/GenerateOptions.providerexpose as{ id, name }— the endpoint is fixed per route, not per model. A call with no provider key cannot be placed against the evidence, so it keeps today's clamp: a cap that is too low only shortens an answer, while a 400 loses it whole (the failure #3132 recorded).DashScope behaviour is unchanged and still pinned: a row declaring more is asked for 12288, a caller asking for more is sent 12288, the caller's object is never written to, and the second
ctx.llm.streampass carries 12288.Red first
On base
0f7e82121832d4cea8e26f1e9681355100432df7, with the new cases only:The two existing provider cases (
:51,:66) keep their assertions as the DashScope half of a split; the other four are untouched.Verification at this tip
core/test/llm-output-cap.test.js—# tests 9 / # pass 9 / # fail 0node scripts/ds-bot-patches.mjs check—ok 10 patches apply cleanly in order;canonical—ok patches/ds-bot/ is what generate writesTZ=UTC npm run verify— exit 0; 10 suites, 0 failures in each: 3345 tests, 20, 717, 149, 172, 538, 151, 686, 37, 34;arch-docs: OK;ok 10 patches apply cleanly in order; mojibake scan 0 hits.live · ubuntu · Node 24 · real modelandwindow check, the one failure isenqueue-guard(no Verifier report names the tip ff38bd47…), the expected red while the T2 PASS is pending.Boundaries
core/src/host/models.jschanges only the predicate around the constant and the listener's first line; the clamp value, both branches and their comments are otherwise untouched.patches/ds-bot/0001-core-fixes.patchandpatches/ds-bot/files/test/llm-output-cap.test.jscome fromgenerate; nothing is hand-edited..github/**,core/client.js,patches/ds-bot/0008ori18n.jstouched.#3195/#3197land first, this branch rebases and re-runsgeneraterather than hand-merging the patch.Checkpoint
{ "pr": 3201, "claim": "Dev-5", "issue": 3198, "summary": "The 12288 output cap encoded one endpoint's 400 but was applied to every provider, silently cutting answers on routes that accept more. The llm/stream listener now clamps only on the opencsitool route that #3132 recorded, named beside the constant with that evidence, and passes every other route through so the caller's request or the model row's declared cap reaches the provider. A route with no provider key keeps the clamp, because a shorter answer is recoverable and a 400 is not. DashScope behaviour and the copy-not-mutate rule are unchanged.", "files": ["core/src/host/models.js", "core/test/llm-output-cap.test.js", "patches/ds-bot/0001-core-fixes.patch", "patches/ds-bot/files/test/llm-output-cap.test.js", "docs/subsystems/core-host.md", "changelog.d/3198.fixed.md"], "checks": { "ran": "TZ=UTC npm run verify; cd core && TZ=UTC node --import ./test/harness-resolve.mjs --test test/llm-output-cap.test.js; node scripts/ds-bot-patches.mjs check; node scripts/ds-bot-patches.mjs canonical", "result": "green", "at": "ff38bd47701c8e6b011f0f0b3b42b1ebba416bed" }, "next": "none — ready for review" }