fix(session): stop the legacy 200k price from overriding a model's real context tier - #44223
fix(session): stop the legacy 200k price from overriding a model's real context tier#44223xyzs996 wants to merge 1 commit into
Conversation
…al tier `experimentalOver200K` is the legacy spelling of the lowest context tier, from when 200k was the only threshold in the catalog. In every models.dev entry that carries both fields — 361 of them — its prices equal the lowest context tier's exactly, and no entry carries it without `tiers`. `getUsage` treated it as a live threshold at a hardcoded 200 000, reached through a `??` that fires whenever the tier filter comes back empty — which is exactly when the request is *below* the model's real threshold. So any request between 200 001 tokens and the real threshold was priced at the high tier. The GPT-5.6 family steps at 272k, so a 250k gpt-5.6-luna request was estimated at $0.40/M instead of the documented $0.20/M. 252 tier entries in the catalog sit at a threshold other than 200k (272k, 256k, 128k, 32k, 262144, 512k) and every one of them has the same band. Now the fallback is only consulted when the model has no context tiers at all, so tiered models bill at the base price below their lowest threshold.
github-actions
Bot
added
needs:compliance
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, I found one potentially related PR: Related PRPR #42919: fix(opencode): apply long context config pricing This PR is related because:
However, PR #44223 is focused on a different cost bug in the opposite direction (charging too much in the 200k–threshold band), and makes a distinct fix to the |
github-actions
Bot
removed
needs:issue
needs:compliance
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
Bug DescriptionI am experiencing a severe performance/freeze issue with the OpenCode Agent status lifecycle. After starting a session or agent workflow, the agent status indicator shows Active for approximately ~1 minute before automatically shifting back to Inactive (indicated by a red status badge). Immediately following this state transition, the entire application interface freezes, stops responding to user interactions, and remains hung indefinitely. The only way to recover from this state is to forcefully terminate and restart the application. Steps to Reproduce
Expected Behavior
Environment & Context
Logs / Additional Notes
|
2 participants
Issue for this PR
Closes #44224
Type of change
What does this PR do?
The
??ingetUsagefires whenever the context-tier filter comes back empty — and the filter is empty precisely when the request is below the model's real threshold. Control then falls toexperimentalOver200K && contextTokens > 200_000, a different, hardcoded number, so a request between 200,001 tokens and the model's actual threshold gets the high price. For the GPT-5.6 family, which steps at 272k, a 250k request is estimated at $0.40/M instead of the documented $0.20/M.experimentalOver200Kreads like an independent, cheaper threshold, but in the data it is just the lowest context tier under an older name, from when 200k was the only number in the catalog. Against today'smodels.dev/api.json: 361 models carry bothtiersandcontext_over_200k; in all 361 thecontext_over_200kprices equal the lowest tier's exactly; 0 carry it withouttiers. So for correct catalog data that branch is only ever reached below the real threshold, where the base price is the right answer.The change: consult
experimentalOver200Konly when the model has no context tiers at all. A tiered model matching no tier is genuinely below every threshold, so it bills at base. Models carrying only the legacy field behave exactly as before — there are none in models.dev today, but hand-written and third-party model configs can still have that shape.252 tier entries in the catalog sit at a threshold other than 200k (272k, 256k, 128k, 32k, 262144, 512k), and every one has this band.
How did you verify your code works?
bun test test/session/compaction.test.ts— 56 pass, 1 skip, 0 failbun test test/session/llm.test.ts test/server/negative-tokens-regression.test.ts— 30 pass, 0 failbun run typecheck— cleanoxlinton both changed files — 0 errors, no new warnings on the changed linesOn the tests themselves:
falls back to over-200k pricing when no cost tier matchespinned the old behaviour using a model shape that does not occur in the catalog — a tier at 500k plus anexperimentalOver200Kat a lower price, i.e. the two fields disagreeing. I kept the case actually worth covering, the legacy-only model, renamed it...when the model has no cost tiers, and dropped the contradictorytiersentry; its assertion is unchanged. Addeddoes not use the 200k fallback below a higher real threshold, built fromgpt-5.6-luna's real numbers: 250k bills at base, 300k bills at the tier.Screenshots / recordings
Not a UI change.
Checklist
Two related things from #42910 I deliberately left alone, since they are separate calls: Zen's handler trips its tier on a hardcoded
200_000overinput + cacheRead + cacheWrite5m + cacheWrite1h, which both uses the wrong constant and double-counts cache-write tokens; and line 382 uses strictly-greater for every provider, while vendors disagree about the token exactly at the line (xAI's models page says a prompt that "reaches" the threshold bills high; Google's Gemini pricing page says<= 200kbills low). Same number, opposite answers, and thetierobject has no field to record which one a model means. Worth one token at most, so not worth mixing in here.