Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,29 @@ All notable changes to blockrun-llm will be documented in this file.

## Unreleased

### Fixed
- **`router_core` re-synced to upstream `@blockrun/router-core` `273c2af`**
(was `8773a99`), which includes the 2026-11 router audit. All 88
cross-language snapshot decisions match the TypeScript engine.
- Chinese debug, refactor, extraction and proof prompts are no longer
classified as chat. Japanese prompts no longer get the Mandarin
extraction band.
- With tools attached, a bare language name ("java coffee prices online")
is no longer routed to the code-agent pool.
- Tool-name verbs, airline reservation ids and the multi-step signal no
longer fire on look-alike text ("add" in "address", "CANCEL", a sentence
ending in a number). Full-width `?` now counts as a question.
- An unpriced candidate no longer outranks priced ones on cost. NaN prices
and malformed `model_performance` entries no longer leak NaN into costs
or the ranking. `calculate_model_cost` reports no savings for an
unpriced model.
- A one-character or non-numeric `prompt_truncation_chars` no longer makes
the router scan the whole prompt.
- Capability snapshot synced to the 2026-11-08 catalog. `moonshot/kimi-k3`,
which is still routed and served, is retained. DeepSeek chat, reasoner and
V4.1 Flash stay text-only until an image probe confirms the catalog's new
vision tag.

## 1.21.5 — 2026-11-07

### Fixed
Expand Down
2 changes: 1 addition & 1 deletion blockrun_llm/router_core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Router Core — deterministic, constraint-first model routing.

Python port of `@blockrun/router-core <https://github.com/BlockRunAI/router-core>`_
(upstream commit ``8773a99``), the same routing engine the TypeScript SDK and
(upstream commit ``273c2af``), the same routing engine the TypeScript SDK and
the BlockRun gateway use. The package is deliberately product-neutral: task
classification, hard capability filtering, portfolio scoring, ordered
fallbacks, and routing configuration. It contains no wallet, gateway client,
Expand Down
2 changes: 1 addition & 1 deletion blockrun_llm/router_core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Default Routing Config

Python port of ``@blockrun/router-core`` ``config.ts`` (upstream commit
``8773a99``; the file is unchanged since ``e38958b``, 2026-09-29, so it is the
``273c2af``; the file is unchanged since ``e38958b``, 2026-09-29, so it is the
same table the TypeScript SDK bundles). V3.5: every tier rung is a public
catalog id.

Expand Down
95 changes: 64 additions & 31 deletions blockrun_llm/router_core/model_capabilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
product catalog is temporarily unavailable, without importing product code.

GENERATED upstream by ``scripts/sync-model-capabilities.mjs`` from the public
catalog (GET https://blockrun.ai/api/v1/models) on 2026-10-20; ``supports_tools``
catalog (GET https://blockrun.ai/api/v1/models) on 2026-11-08; ``supports_tools``
comes from a live function-calling probe. Re-sync from ``model-capabilities.ts``
rather than editing by hand — a hand edit is lost on the next sync.
"""
Expand All @@ -35,9 +35,9 @@
"supports_vision": True,
},
# override: The public catalog's `categories` omit "vision" for this Anthropic model even
# though the gateway accepts image input for it (the prior hand-maintained snapshot had
# it, and Anthropic's model card lists it). Without this the vision filter would silently
# drop it — reported against the catalog; remove once the categories carry vision.
# though the gateway accepts image input for it (the prior hand-maintained snapshot had it,
# and Anthropic's model card lists it). Without this the vision filter would silently drop it
# — reported against the catalog; remove once the categories carry vision.
"anthropic/claude-haiku-4.5": {
"context_window": 200_000,
"max_output_tokens": 64_000,
Expand Down Expand Up @@ -81,9 +81,9 @@
"supports_vision": True,
},
# override: The public catalog's `categories` omit "vision" for this Anthropic model even
# though the gateway accepts image input for it (the prior hand-maintained snapshot had
# it, and Anthropic's model card lists it). Without this the vision filter would silently
# drop it — reported against the catalog; remove once the categories carry vision.
# though the gateway accepts image input for it (the prior hand-maintained snapshot had it,
# and Anthropic's model card lists it). Without this the vision filter would silently drop it
# — reported against the catalog; remove once the categories carry vision.
"anthropic/claude-sonnet-4.6": {
"context_window": 1_000_000,
"max_output_tokens": 128_000,
Expand All @@ -109,12 +109,24 @@
"supports_tools": False,
"supports_vision": False,
},
# override: The 2026-11-08 catalog began tagging deepseek/deepseek-chat "vision" (it was text-
# only through 2026-10-20), and DeepSeek still ships image input as a separate experimental id
# (deepseek/deepseek-v4-flash-vision-exp). No image probe has confirmed the tag. Routing an
# image turn to a model that silently drops the image returns a confident 200 the caller
# cannot branch on, so this fails closed until a correctly sized image probe passes on both
# chains; then remove this entry.
"deepseek/deepseek-chat": {
"context_window": 1_048_576,
"max_output_tokens": 65_536,
"supports_tools": True,
"supports_vision": False,
},
# override: The 2026-11-08 catalog began tagging deepseek/deepseek-reasoner "vision" (it was
# text-only through 2026-10-20), and DeepSeek still ships image input as a separate
# experimental id (deepseek/deepseek-v4-flash-vision-exp). No image probe has confirmed the
# tag. Routing an image turn to a model that silently drops the image returns a confident 200
# the caller cannot branch on, so this fails closed until a correctly sized image probe passes
# on both chains; then remove this entry.
"deepseek/deepseek-reasoner": {
"context_window": 1_048_576,
"max_output_tokens": 65_536,
Expand All @@ -133,7 +145,12 @@
"supports_tools": True,
"supports_vision": False,
},
# supportsTools: not probed — fails closed
# override: The 2026-11-08 catalog began tagging deepseek/deepseek-v4.1-flash "vision" (it was
# text-only through 2026-10-20), and DeepSeek still ships image input as a separate
# experimental id (deepseek/deepseek-v4-flash-vision-exp). No image probe has confirmed the
# tag. Routing an image turn to a model that silently drops the image returns a confident 200
# the caller cannot branch on, so this fails closed until a correctly sized image probe passes
# on both chains; then remove this entry.
"deepseek/deepseek-v4.1-flash": {
"context_window": 1_048_576,
"max_output_tokens": 65_536,
Expand Down Expand Up @@ -200,6 +217,7 @@
"supports_tools": True,
"supports_vision": True,
},
# supportsTools: not probed — fails closed
"google/gemini-4-argon": {
"context_window": 1_048_576,
"max_output_tokens": 65_536,
Expand All @@ -225,47 +243,69 @@
"supports_tools": False,
"supports_vision": True,
},
# absent from the 2026-11-08 listing but still routed — previous entry retained; probe before
# removing
"moonshot/kimi-k3": {
"context_window": 1_048_576,
"max_output_tokens": 65_536,
"supports_tools": True,
"supports_vision": True,
},
# supportsTools: not probed — fails closed
"nvidia/llama-3.2-11b-vision": {
"context_window": 128_000,
"max_output_tokens": 16_384,
"moonshot/kimi-k3.1-pro": {
"context_window": 1_048_576,
"max_output_tokens": 65_536,
"supports_tools": False,
"supports_vision": True,
},
# supportsTools: not probed — fails closed
"nvidia/minimax-m3.1-flash": {
"context_window": 1_048_576,
"max_output_tokens": 16_384,
"supports_tools": False,
"supports_vision": False,
},
# supportsTools: not probed — fails closed
"nvidia/muse-glimmer-30b": {
"context_window": 131_072,
"max_output_tokens": 16_384,
"supports_tools": False,
"supports_vision": False,
},
# override: The catalog tags this model "vision", but a correctly sized probe does not
# hold up (ClawRouter, 2026-08-31): a 64x64 solid-red PNG was named correctly 1 of 4
# times on Base, and on Solana the image was silently dropped and a text model answered
# "white". An HTTP 200 with a confident wrong answer gives the caller nothing to branch
# on, so image turns must not be routed here. It is ecoTiers.SIMPLE.fallback[0] since
# nemotron-3-nano-30b was delisted (2026-09-08); remove once a probe of this size comes
# back right on both chains.
# override: The catalog tags this model "vision", but a correctly sized probe does not hold up
# (ClawRouter, 2026-08-31): a 64x64 solid-red PNG was named correctly 1 of 4 times on Base,
# and on Solana the image was silently dropped and a text model answered "white". An HTTP 200
# with a confident wrong answer gives the caller nothing to branch on, so image turns must not
# be routed here. It is ecoTiers.SIMPLE.fallback[0] since nemotron-3-nano-30b was delisted
# (2026-09-08); remove once a probe of this size comes back right on both chains.
"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning": {
"context_window": 256_000,
"max_output_tokens": 16_384,
"supports_tools": False,
"supports_vision": False,
},
# supportsTools: not probed — fails closed
"nvidia/nemotron-3-super-120b": {
"context_window": 131_072,
"max_output_tokens": 16_384,
"supports_tools": False,
"supports_vision": True,
},
# supportsTools: not probed — fails closed
"nvidia/nemotron-3-ultra-550b": {
"context_window": 1_000_000,
"max_output_tokens": 16_384,
"supports_tools": False,
"supports_vision": False,
},
# supportsTools: not probed — fails closed
"nvidia/nemotron-3.5-bolt": {
"context_window": 131_072,
"max_output_tokens": 16_384,
"supports_tools": False,
"supports_vision": True,
},
# supportsTools: not probed — fails closed
"nvidia/nemotron-3.5-lightning": {
"context_window": 1_000_000,
"max_output_tokens": 16_384,
Expand All @@ -290,12 +330,6 @@
"supports_tools": True,
"supports_vision": False,
},
"openai/gpt-4.1-nano": {
"context_window": 128_000,
"max_output_tokens": 32_768,
"supports_tools": True,
"supports_vision": False,
},
"openai/gpt-4o": {
"context_window": 128_000,
"max_output_tokens": 16_384,
Expand Down Expand Up @@ -333,12 +367,12 @@
"supports_tools": False,
"supports_vision": True,
},
# supportsTools: gateway unavailable at probe time — fails closed; override: 2026-08-29
# probe: every request (6 plain + 3 tool attempts) returned a gateway 500, so the probe
# measured an incident, not the model. Codex's function calling is established by the
# 2026-07 Terminal-Bench / tau2 calibration trajectories in portfolio.ts. Hosts observing
# the 500s should drop it with RouterOptions.unavailableModels rather than this snapshot
# claiming the model cannot call tools.
# supportsTools: gateway unavailable at probe time — fails closed; override: 2026-08-29 probe:
# every request (6 plain + 3 tool attempts) returned a gateway 500, so the probe measured an
# incident, not the model. Codex's function calling is established by the 2026-07 Terminal-
# Bench / tau2 calibration trajectories in portfolio.ts. Hosts observing the 500s should drop
# it with RouterOptions.unavailableModels rather than this snapshot claiming the model cannot
# call tools.
"openai/gpt-5.3-codex": {
"context_window": 400_000,
"max_output_tokens": 128_000,
Expand Down Expand Up @@ -437,7 +471,6 @@
"supports_tools": True,
"supports_vision": True,
},
# supportsTools: not probed — fails closed
"openai/gpt-6.1-sol": {
"context_window": 1_050_000,
"max_output_tokens": 128_000,
Expand Down
Loading
Loading