Skip to content

Latest commit

 

History

History
764 lines (559 loc) · 33 KB

thinking.md

File metadata and controls

764 lines (559 loc) · 33 KB

Thinking & Reasoning Capability Support

Operator-facing guide for thinking/reasoning support in EggPool.

1. Conceptual Model

EggPool models thinking/reasoning as a three-layer problem:

  1. Native support: a model may support extended thinking natively (e.g. Anthropic's thinking blocks, OpenAI's reasoning_content).
  2. Transcoding: EggPool may be able to translate client controls between protocols (OpenAI reasoning_effort ↔ Anthropic thinking blocks).
  3. Discovery: clients discover support through eggpool.capabilities in /v1/models.

Key invariants:

  • Unknown ≠ unsupported. A status of "unknown" means no capability data has been observed. It is explicitly not a claim that thinking is unsupported. This avoids false negatives when catalog or model-info data has not yet been populated.
  • No fabrication. EggPool does not generate hidden reasoning. It only forwards provider-exposed content. If the upstream does not return thinking blocks, the client does not receive them.
  • Protocol compatibility alone does not imply thinking support. A model reachable via the OpenAI Chat Completions-compatible protocol may still lack thinking support. Capability status is independent of protocol compatibility.

Reasoning facts are scoped to the exact provider/model pair. Source authority is operator override, explicit live provider metadata, verified provider-scoped model-info metadata, then unknown. A live source may leave a control dimension unknown while still allowing a lower source to fill it; an explicit empty reasoning_options list is a complete no-control declaration. Model-family names never supply effort defaults.

Compositional provider controls

ThinkingControlContract is the canonical provider/model contract for caller controls. It records toggle, effort, and budget independently, each as supported, unsupported, or unknown. This lets the catalog represent toggle-only, effort-only, budget-only, and combined hosts without a growing combination enum. Reasoning can be supported while all three controls are unsupported, which means fixed/no caller control.

Provider-bound entries are authoritative. Collapsed entries use conservative mixed summaries when providers differ and must not be used to infer that one provider supports the union of another provider's controls.

Status values (CapabilityStatus):

Value Meaning
supported Confirmed upstream thinking support (from provider catalog, model-info source, or manual override)
unsupported Confirmed upstream does not support thinking
unknown No data observed yet: not a claim of either support or lack
mixed Some backing providers support thinking, others do not
conflicting External sources disagree on support status; requires operator resolution via manual override

Source: rust/src/catalog/cache.rs

2. Thinking Transcoding

Thinking transcoding is enabled by default. No configuration is required. To disable it, set the feature gate in [transcoder.features]:

[transcoder.features]
thinking = false

When thinking = false, the transcoder drops thinking-related fields with a structured reasoning_content_dropped warning rather than forwarding them. Specifically:

  • OpenAI → Anthropic: reasoning_content in assistant messages is dropped.
  • Anthropic → OpenAI: thinking content blocks are dropped.
  • Streaming: thinking stream deltas are dropped.

The loss_policy setting in [transcoder] controls whether dropped fields cause a request rejection or a warning log:

  • loss_policy = "warn" (default): logs structured warnings per request but proceeds.
  • loss_policy = "reject": returns HTTP 400 when request-body translation would drop or alter thinking fields before upstream dispatch.

Example config for strict loss policy:

[transcoder]
loss_policy = "reject"

Source: rust/src/wire/adaptation.rs

3. Configuration Reference

Feature Gate

[transcoder.features]
thinking = true          # Enable thinking/reasoning transcoding (default: true)

Budget Defaults

Global effort→budget token mapping is a legacy translation-policy fallback, used only when the selected model contract does not carry a verified mapping:

[transcoder.thinking_budget_defaults]
low = 1024
medium = 4096
high = 16384

All values must be > 0. See Budget Mapping for resolution order.

Budget Resolution Policy

[transcoder]
budget_resolution_policy = "lenient"   # default
  • "lenient": preserves an unmapped effort safely by omitting the target thinking control and emitting a bounded warning; it allows budget clamping.
  • "strict": rejects unknown effort levels and clamped budgets with BudgetResolutionError (HTTP 400) before dispatch.

OpenAI effort values are model-dependent. reasoning_effort = "none" remains an explicit effort intent during routing; it is accepted only when the selected provider advertises none in its effort set. A target may map it to an explicit disable only through a verified target contract, and it must never become a positive budget or an invented toggle. Current or future values such as xhigh and max are translated only when the selected capability provides an explicit mapping; otherwise lenient mode drops the target control rather than guessing a 4096-token budget.

OpenAI Chat Completions Reasoning Fields

Controls which JSON field names EggPool emits for reasoning content in OpenAI Chat Completions-compatible responses:

[transcoder.openai_reasoning_fields]
non_stream = ["reasoning_content"]          # default
stream_delta = ["reasoning"]                # default
emit_compat_aliases = false                # default

When emit_compat_aliases = true, additional field names beyond the primary entry are emitted. Only enable when all downstream clients tolerate extra fields.

Capability Policy

Controls how requests with explicit thinking controls are routed when the candidate model's capability status is not "supported":

[transcoder.capability_policy]
unsupported_thinking = "reject"             # default: reject | warn_drop | route_best_effort
unknown_thinking = "reject"                 # default: reject | allow_with_warning | route_best_effort
mixed_collapsed_thinking = "filter"         # default: filter | reject | allow

See Routing Policy for full details.

Complete Example

[transcoder]
enabled = true
loss_policy = "warn"
prefer_native = true
budget_resolution_policy = "lenient"

[transcoder.features]
thinking = true
tools = true

[transcoder.thinking_budget_defaults]
low = 1024
medium = 4096
high = 16384

[transcoder.capability_policy]
unsupported_thinking = "reject"
unknown_thinking = "reject"
mixed_collapsed_thinking = "filter"

[transcoder.openai_reasoning_fields]
non_stream = ["reasoning_content"]
stream_delta = ["reasoning"]
emit_compat_aliases = false

4. Capability Overrides

Operators can override discovered capability data with manual TOML configuration. Overrides are applied in a 3-layer chain with increasing precedence:

  1. Defaults / discovered data (from provider catalog, model-info sources)
  2. Global overrides: [model_capabilities."<model_id>".thinking]
  3. Provider-scoped overrides: [providers.<id>.model_capabilities."<model_id>".thinking]

Provider-scoped overrides take highest precedence when the provider ID matches.

Override Fields

[model_capabilities."gpt-4o".thinking]
status = "supported"
source = "manual_override"
native_protocols = ["openai"]
budget_tokens_min = 1024
budget_tokens_max = 32768
effort_to_budget_tokens = { low = 512, medium = 2048, high = 8192 }
toggle = "unsupported"
effort = "supported"
budget = "unsupported"
notes = "Operator-confirmed thinking support via OpenAI"
Field Type Description
status string "supported", "unsupported", "unknown", "mixed", or "conflicting"
source string Defaults to "manual_override" when status is set
native_protocols list "openai" and/or "anthropic"
budget_tokens_min int Minimum budget (must be > 0)
budget_tokens_max int Maximum budget (must be > 0, ≥ min)
effort_to_budget_tokens dict Custom effort→budget mapping (e.g. { low = 512, medium = 2048 })
toggle string Host toggle support: supported, unsupported, or unknown
effort string Host effort support: supported, unsupported, or unknown
budget string Host numeric-budget support: supported, unsupported, or unknown
notes string Operator notes

The three control fields are independent of status: supported reasoning can be fixed/no-control, toggle-only, effort-only, budget-only, or any explicit combination. When status = None, the entire override is a no-op: all other fields are cleared. Existing supported_efforts, budget bounds, and explicit effort maps continue to decode into the corresponding canonical dimensions; an explicitly empty supported_efforts list means effort is unsupported.

Routing checks the exact selected provider/model contract for every requested dimension. A toggle request can use a toggle-only target; an effort request must match the target's advertised effort set; a numeric budget request must fit its advertised budget bounds. A supported reasoning model with no caller controls (for example a fixed-reasoning target) does not satisfy an explicit effort or budget request. Known mismatches are rejected or dropped locally by the provider-control policy and never count as upstream health failures.

Provider metadata normalization

The catalog reads one shared reasoning_options shape:

[
  {"type": "toggle"},
  {"type": "effort", "values": ["minimal", "low", "medium", "high", "xhigh"]},
  {"type": "budget_tokens", "min": 1024, "max": 32768}
]

The list is normalized independently by control type. An omitted field leaves all control dimensions unknown; a complete empty list marks all three unsupported while reasoning itself can remain supported. Effort labels are preserved in first-seen order (med normalizes to medium) and do not create numeric budgets. A provider/model budget map is retained only when supplied by metadata or configuration.

Provider-Scoped Example

[providers.anthropic-prod.model_capabilities."claude-sonnet-4-20250514".thinking]
status = "supported"
native_protocols = ["anthropic"]
budget_tokens_min = 1024
budget_tokens_max = 128000
effort_to_budget_tokens = { low = 1024, medium = 10000, high = 128000 }

Source: rust/src/config.rs, rust/src/catalog/cache.rs

5. /v1/models Metadata

Capabilities are exposed under eggpool.capabilities.thinking in each model object. The serialization is compact: unknown/empty values are omitted.

Provider-Scoped Entry (full shape)

When models.collapse_models = false (the default), each provider gets its own model entry:

{
  "id": "claude-sonnet-4-20250514/anthropic-prod",
  "object": "model",
  "owned_by": "anthropic-prod",
  "name": "Claude Sonnet 4",
  "eggpool": {
    "provider_id": "anthropic-prod",
    "base_model_id": "claude-sonnet-4-20250514",
    "capabilities": {
      "thinking": {
        "status": "supported",
        "source": "provider_catalog",
        "native_protocols": ["anthropic"],
        "control_contract": {
          "toggle": "unsupported",
          "effort": "supported",
          "budget": "supported",
          "accepted_efforts": ["low", "medium", "high"]
        },
        "anthropic_request_fields": ["thinking", "thinking_budget"],
        "anthropic_response_fields": ["thinking"],
        "anthropic_stream_delta_fields": ["thinking"],
        "anthropic_response_block_types": ["thinking"],
        "budget_tokens_min": 1024,
        "budget_tokens_max": 128000,
        "effort_to_budget_tokens": {"low": 1024, "medium": 10000, "high": 128000}
      }
    }
  }
}

Collapsed Entry with Mixed Status

When models.collapse_models = true, a single entry aggregates all providers:

{
  "id": "claude-sonnet-4-20250514",
  "object": "model",
  "eggpool": {
    "providers": ["anthropic-prod", "anthropic-readonly"],
    "routing_priority_max": 5,
    "capabilities": {
      "thinking": {
        "status": "mixed",
        "source": "aggregate",
        "native_protocols": ["anthropic"],
        "providers": {
          "anthropic-prod": "supported",
          "anthropic-readonly": "unsupported"
        }
      }
    }
  }
}

The providers dict in the thinking block shows per-provider status so clients can understand why the aggregate is mixed.

Source: rust/src/catalog/cache.rs

6. Routing Policy

[transcoder.capability_policy] controls how requests with explicit thinking/reasoning controls are routed when candidate models have non-"supported" status.

Per-Status Policies

unsupported_thinking: how to handle candidates whose thinking status is "unsupported":

Value Behavior
reject (default) Excludes the candidate from routing
warn_drop Routes but logs a warning
route_best_effort Ignores the status entirely

unknown_thinking: how to handle candidates whose thinking status is "unknown":

Value Behavior
reject (default) Excludes the candidate
allow_with_warning Routes but logs a warning
route_best_effort Ignores the status entirely

mixed_collapsed_thinking: how to handle collapsed-model entries with mixed provider support:

Value Behavior
filter (default) Narrows to only supported providers; if none remain, falls through to rejection
reject Excludes the model entirely
allow Ignores the status entirely

Always-Rejected Status

"conflicting" status is always rejected regardless of policy settings. Operators resolve conflicts by setting a manual override that clears the conflict (see Capability Overrides).

Error Distinction

Error HTTP Status When
CapabilityError 400 No compatible upstream found for a thinking request
ModelNotFoundError 404 Model does not exist in the catalog
ModelUnavailableError 503 Model exists but is currently unavailable (health, quota, etc.)

Source: rust/src/catalog/cache.rs, rust/src/error.rs

Provider-bound control policy

After account selection, [transcoder.provider_control_policy] applies the same policy to top-level and nested selectable controls:

unsupported_control Behavior
reject (default) Raises CapabilityError before upstream dispatch.
warn_drop Removes the unsupported control and records a warning.
map_if_known Applies only an explicit contract mapping; otherwise raises CapabilityError.

This includes top-level thinking_budget. An effort-only contract therefore rejects an unmappable budget under the default policy, while map_if_known may convert an exact budget in the contract's effort-to-budget table into its corresponding effort. The none contract rejects selectable controls under reject and map_if_known; only warn_drop removes them. Historical reasoning content is not a selectable control and remains unchanged.

7. Budget Mapping

The budget resolver translates client-side effort levels or explicit budgets into a concrete token count.

Legacy Translation-Policy Default Mapping

When no verified provider/model mapping or global config override applies, the legacy translation policy uses:

Effort Level Budget Tokens
low 1024
medium 4096
high 16384

Source: rust/src/wire/adaptation.rs

Resolution Order

The resolver evaluates these sources in order, stopping at the first match:

  1. Explicit budget_tokens (Anthropic style): validated and clamped to capability min/max.
  2. reasoning_effort via capability mapping: looks up the effort in ThinkingCapability.effort_to_budget_tokens.
  3. Global config defaults: [transcoder.thinking_budget_defaults].
  4. Hard-coded fallback: low=1024, medium=4096, high=16384.
  5. Unmapped effort: if no verified target mapping exists:
    • "lenient" mode: omits target thinking and emits an unknown_effort warning with reason = "target_mapping_unknown".
    • "strict" mode: raises BudgetResolutionError (HTTP 400).

The legacy hard-coded compatibility values apply only to low, medium, and high. The explicit none effort value produces no budget; it is not silently rewritten as a binary toggle.

Clamping

When budget_tokens_min or budget_tokens_max are known (from capability data or overrides), the resolved budget is clamped to that range:

  • Budgets below min are raised to min.
  • Budgets above max are lowered to max.

Under "strict" policy, clamped budgets cause rejection. Under "lenient" policy, clamping is silently applied with a warning.

Source: rust/src/wire/adaptation.rs

8. Client Examples

OpenAI-Style Request (reasoning_effort)

{
  "model": "minimax-m3/minimax",
  "messages": [
    {"role": "user", "content": "Solve this carefully."}
  ],
  "reasoning_effort": "medium"
}

EggPool resolves "medium" → 4096 tokens (from global defaults) and forwards as an Anthropic thinking block to the upstream provider.

Anthropic-Style Request (thinking)

{
  "model": "minimax-m3/minimax",
  "messages": [
    {"role": "user", "content": "Solve this carefully."}
  ],
  "thinking": {
    "type": "enabled",
    "budget_tokens": 4096
  }
}

EggPool receives the explicit budget, validates and clamps it against capability min/max, then forwards to the upstream provider.

Streaming Request

For streaming, include "stream": true as usual. Thinking stream deltas are translated between protocols automatically:

  • Anthropic thinking deltas → OpenAI reasoning delta field (configurable via [transcoder.openai_reasoning_fields]).
  • OpenAI reasoning deltas → Anthropic thinking deltas.

9. OpenCode Integration

eggpool configsetup opencode generates an OpenCode-compatible configuration that includes reasoning annotations for discovered models.

Discovery Logic

The generator (rust/src/operations/integrations.rs) projects each model through the provider-neutral agent projection and inspects capabilities.thinking.status:

  • "supported" → emits "reasoning": true plus variants for each validated supported_efforts entry.
  • All other statuses ("unknown", "unsupported", "mixed", "conflicting") → the reasoning field is omitted.

This means the generated config never claims reasoning support for models without confirmed upstream backing.

Provider-Scoped Model IDs

When collapse_models = false, the generator renders provider-suffixed model IDs (e.g. gpt-4o/openai) so OpenCode's model picker disambiguates providers serving the same upstream model.

Example Output

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "eggpool": {
      "npm": "@ai-sdk/openai",
      "name": "EggPool",
      "options": {
        "baseURL": "http://localhost:11300/v1",
        "apiKey": "{env:EGGPOOL_API_KEY}"
      },
      "models": {
        "claude-sonnet-4-20250514/anthropic-prod": {
          "name": "Claude Sonnet 4/anthropic-prod",
          "limit": { "context": 200000 },
          "modalities": { "input": ["text"], "output": ["text"] },
          "reasoning": true
        },
        "gpt-4o/openai": {
          "limit": { "context": 128000 },
          "modalities": { "input": ["text"], "output": ["text"] }
        }
      }
    }
  }
}

Note that gpt-4o/openai has no reasoning field: its capability status is unknown or unsupported, so the annotation is omitted. Set EGGPOOL_API_KEY in the environment that launches OpenCode.

Source: rust/src/operations/integrations.rs

10. Observability

Thinking usage is recorded as durable aggregate facts, not per-request traces. reasoning_tokens and thinking_characters accumulate in the usage_rollups table and surface in the dashboard metrics API (rust/src/operations/metrics.rs, rust/src/server/dashboard/api.rs). There is no GET /api/stats/thinking endpoint and no in-memory thinking counter: thinking decisions are classified at request time by the wire adaptation layer (rust/src/wire/) and capability-aware routing (rust/src/routing/eligibility.rs), with loss/drop reasons reported through the standard bounded adaptation warnings (e.g. reasoning_content_dropped, thinking_signature_dropped, budget_clamped, budget_rejected).

The requests table carries a thinking_trace_json column (migration 0039_thinking_observability.sql) reserved for future diagnostic use; no runtime path currently populates it, so do not query it expecting data.

Source: rust/src/operations/metrics.rs, rust/src/server/dashboard/

11. Troubleshooting

reasoning_effort is ignored

Symptoms: Client sends reasoning_effort but the upstream receives no thinking controls.

Cause: Thinking transcoding was explicitly disabled with [transcoder.features].thinking = false.

Fix: Remove the override or enable the feature:

[transcoder.features]
thinking = true

Restart the service after configuration changes.


Model listing shows unknown support

Symptoms: /v1/models returns thinking.status = "unknown" for a model you believe supports thinking.

Cause: No capability data has been populated yet. This is the default state: "unknown" means "no data observed", not "unsupported".

Fix: Wait for a catalog refresh cycle, or set a manual override:

[model_capabilities."model-id".thinking]
status = "supported"
native_protocols = ["anthropic"]

Request rejected with capability_error

Symptoms: HTTP 400 with CapabilityError when sending a thinking request.

Cause: No compatible upstream account was found. All candidates were filtered by the capability policy (e.g. unsupported_thinking = "reject" with no "supported" providers).

Fix:

  • Check the model's capability status in /v1/models.
  • Verify [transcoder.capability_policy] settings.
  • If the model does support thinking but status is wrong, set a manual override.
  • If you want best-effort routing, change unknown_thinking or unsupported_thinking to route_best_effort.

Collapsed model routes inconsistently

Symptoms: A collapsed model (collapse_models = true) sometimes routes to a thinking-capable provider, sometimes not.

Cause: The aggregate status is "mixed": some providers support thinking, others do not. The default mixed_collapsed_thinking = "filter" narrows to supported providers, but if the supported provider is unavailable, the request may fall through.

Fix:

  • Check per-provider status in the providers field of the collapsed model's capabilities.
  • Set provider-scoped overrides to resolve inconsistencies.
  • Consider using collapse_models = false to get explicit provider-scoped routing.

Streaming reasoning deltas are missing

Symptoms: Non-streaming responses include reasoning content, but streaming responses do not.

Cause: Thinking transcoding is likely disabled for streaming. The feature gate affects both streaming and non-streaming paths.

Fix: Remove the disabling override or ensure [transcoder.features] thinking = true is set. Check [transcoder.openai_reasoning_fields] to verify the streaming delta field name matches your client's expectations (default: "reasoning").


Provider rejects an apparently supported budget

Symptoms: HTTP 400 from upstream with a budget-related error, even though the client sent a seemingly valid budget.

Cause: The resolved budget exceeds the provider's actual limits. The capability's budget_tokens_max may be incorrect, or the provider may have lower limits than configured.

Fix:

  • Check the provider's actual limits (documentation or API response).
  • Adjust budget_tokens_max in the capability override:
    [model_capabilities."model-id".thinking]
    budget_tokens_max = 8192
  • Switch to budget_resolution_policy = "strict" to catch clamping before dispatch.

12. Closing-Pass Hardening

This section records the semantic hardening decisions for thinking/reasoning handling (Phase A–G). The phase narrative below uses the names of the retired Python implementation in which the hardening was first made (RequestCoordinator.* helpers, CapabilityError/BudgetResolutionError classes, thinking_trace fields); the current Rust runtime implements the same semantics in rust/src/catalog/cache.rs, rust/src/wire/, rust/src/routing/eligibility.rs, and rust/src/coordinator/. Read the phase text for intended behavior, and the Rust modules for the current implementation: do not treat the Python method names as callable code.

Phase A: Missing Capability Metadata Is unknown

Routing now treats a catalog entry with no capabilities.thinking block as semantically equivalent to an explicit status = "unknown". Previously, missing metadata would silently fall through to "supported", masking misconfiguration.

The catalog cache and routing gate in rust/src/catalog/cache.rs and rust/src/coordinator/ are the single source of truth for this classification.

Operator impact:

  • Models with unconfigured thinking capability now participate in the unknown_thinking policy evaluation (default: reject).
  • Add a manual override to opt in:
    [model_capabilities."<model-id>".thinking]
    status = "supported"
    source = "manual_override"

Phase B: BudgetResolutionError Is a CapabilityError

BudgetResolutionError (raised when strict policy rejects an unknown effort level or a clamped budget) is now a subclass of CapabilityError. The proxy layer's existing except CapabilityError handler automatically renders it as HTTP 400 without any manual mapping code. The error carries rich kwargs (model_id, requested_budget_tokens, resolved_budget_tokens, budget_resolution_policy, reason, provider_id) for diagnostic logging.

Phase C: Per-Provider Budget Recompute at Dispatch

After route selection but before upstream dispatch, RequestCoordinator._recompute_thinking_budget_for_selected_provider() re-runs resolve_thinking_budget() against the selected provider's capability (not the collapsed model-level one). This means:

  • Provider-scoped [providers.<id>.model_capabilities."<model-id>".thinking] overrides take effect at dispatch time, not only at preflight translation.
  • Strict rejections here flow through the same CapabilityError → HTTP 400 path as Phase B.
  • The selected capability is recorded on the request trace under thinking_trace.capability_status / thinking_trace.capability_source.

Phase D: Trace Decisions + Rejection Attribution

The transcoding trace now records a single string decision field with one of:

Value Meaning
passthrough No thinking-related warnings (native or no thinking controls)
transcoded A thinking-related warning present (e.g. budget resolution input)
dropped A thinking field was dropped (reasoning_content_dropped, thinking_signature_dropped, anthropic_top_level_thinking_dropped)
clamped budget_clamped warning present
rejected budget_rejected warning present (strict policy)

Rejections from capability-aware routing are attributed with the relevant thinking status (unknown vs unsupported) so the rejection counter and the operator-facing reason distinguish them.

Phase E: Top-Level reasoning_content Detection

classify_thinking_request() now detects top-level reasoning_content on assistant messages (string or list). Clients that attach thinking text alongside content without going through reasoning_content content-blocks are now correctly classified as thinking-required.

Phase F: supports_tools Removed from ModelCapabilities

The vestigial supports_tools: True field has been removed from model_capabilities_to_dict(). Tool support is owned by transcoder features ([transcoder.features] tools = true), not by ModelCapabilities. Tests pin the removal.

Phase G: Explicit anthropic_top_level_thinking_dropped Kind

When the Anthropic-style top-level thinking block is dropped during Anthropic→OpenAI transcoding (no verified mapping), the warning now uses an explicit kind anthropic_top_level_thinking_dropped instead of the generic dropped_field bucket. Operators can configure loss_policy = "reject" per-subsystem and have this drop attributed accurately to the thinking trace.

Phase H: Final Provider Budget Cleanup

This final cleanup pass closes two semantic gaps left after Phase C:

Original Client Intent Is Preserved Through Translation

The post-selection recompute previously forwarded the already-translated Anthropic thinking.budget_tokens value as both requested_effort and requested_budget_tokens to resolve_thinking_budget(). Because the resolver prioritises requested_budget_tokens over requested_effort, the selected provider's effort_to_budget_tokens mapping was silently bypassed for OpenAI reasoning_effort clients.

The new _extract_original_thinking_budget_inputs() helper parses context.original_body (not the provider-bound payload) and returns either (effort, None) for OpenAI clients or (None, budget) for Anthropic clients. The recompute now passes only the original client intent, so:

  • OpenAI reasoning_effort = "high" against a provider whose effort_to_budget_tokens.high = 32768 produces an Anthropic body with thinking.budget_tokens = 32768 even when global defaults are 16384.
  • Anthropic thinking.budget_tokens = 50000 against a provider whose budget_tokens_max = 8192 is still clamped/validated against the selected provider's min/max.

Post-Selection Capability Rejection Cleans Up State

Phase C noted that strict rejections propagate as CapabilityError, but did not handle the durable-state side effects already in flight by the time the recompute runs (request row, attempt row, reservation, active request count, health slot). The new _finalize_selected_capability_rejection() helper runs on rejection and:

  • Finalizes the attempt row with release_reason = "capability_rejected", retry_category = "never", status_code = 400.
  • Releases the reservation durably and removes it from the in-memory quota estimator.
  • Decrements the router's active request count for the selected account.
  • Releases the health-manager probe slot.
  • Marks thinking_trace.decision = "rejected" and stamps provider_id on the trace.
  • Finalizes the request row as client_error with thinking_trace_json persisted.
  • Increments the thinking-metrics rejected counter with the rejection reason (strict_clamp / unknown_effort_strict / capability_rejected).
  • Does not record a health failure: this is a client-validation outcome, not an account health signal.

The streaming and non-streaming dispatch paths share identical cleanup semantics via _apply_selected_provider_transcode_adjustments().

Phase I: Final Polish

This small polish pass hardens the trace metadata and the no-health-penalty guarantee after Phase H:

  • upstream_fields is always populated when the recompute writes. _recompute_thinking_budget_for_selected_provider() treats an empty list the same as None when deciding whether to populate thinking_trace.upstream_fields = ["thinking"]. A pre-populated non-empty list is preserved verbatim so future paths can stack additional upstream fields without being clobbered. The preflight translation already populates the field, so this is a defensive change for non-preflight paths (synthetic requests, future providers, alternate code paths).
  • No-health-penalty guarantee is test-pinned. The strict-rejection cleanup tests now assert HealthManager.is_account_healthy() stays True and the underlying AccountHealth fields (consecutive_failures, health_state, disabled_models, disabled_until, disabled_reason, cooldown_until) stay at their default values after both the non-streaming and streaming cleanup paths, and through a double invocation. Capability rejection remains a client-validation outcome and never records an upstream health signal.

13. Tests for Closing-Pass Behavior

The closing pass adds regression coverage in the native Rust suite (run serially with --test-threads=1):

  • rust/tests/wire_adaptation.rs: capability/routing classification phases (thinking-status extraction, top-level reasoning_content detection, warning classification)
  • rust/tests/wire_codecs.rs + rust/tests/wire_qualification.rs: budget-resolution error shape (CapabilityError semantics) and explicit dropped-thinking-kind coverage
  • rust/tests/routing_domain.rs (+ routing_domain_d008.rs): selected-provider effort mapping and clamp validation
  • rust/tests/coordinator_c007.rs / coordinator_c008.rs: strict-rejection cleanup invariants, streaming parity, idempotency, upstream_fields population + preservation, and the no-health-penalty regression (capability rejection stays a client-validation outcome and never records an upstream health signal)