EggPool supports configurable effective context limits for individual models on individual providers. This lets operators advertise a smaller context window than the provider physically supports.
Some models support very large context windows (e.g., 1M tokens), but the cost and latency at that scale can be prohibitive. An operator may want to cap the advertised context window so that OpenCode's compaction machinery triggers earlier, keeping requests in a more efficient regime.
Apply to all providers unless overridden per-provider:
[model_overrides."model-id"]
max_context_tokens = 200000
max_input_tokens = 180000
max_output_tokens = 16384
enforce_context_limit = trueOverride global limits for a specific provider:
[providers.opencode-go.model_overrides."MiniMax-M3"]
max_context_tokens = 220000
max_output_tokens = 16384
enforce_context_limit = trueEach field is resolved independently:
- Provider-specific override
- Global override
- Upstream-reported metadata
- Unknown (None)
max_input_tokensmust not exceedmax_context_tokensmax_output_tokensmust not exceedmax_context_tokens
Override keys match the base model ID, not the provider-suffixed exposed ID:
# Correct: base model ID
[providers.opencode-go.model_overrides."MiniMax-M3"]
# Wrong: provider-suffixed ID
[providers.opencode-go.model_overrides."MiniMax-M3/opencode-go"]When the same model is served by multiple providers:
- Unsuffixed (
MiniMax-M3): Uses the conservative minimum across all providers - Provider-suffixed (
MiniMax-M3/opencode-go): Uses that provider's exact limit
Generate an OpenCode configuration with model limits:
eggpool configsetup opencode > opencode-config.jsonThis produces a JSON file with explicit limit.context, limit.input, and limit.output values for each model. OpenCode uses these to trigger compaction before exceeding the configured window.
When enforce_context_limit = true (the default), EggPool rejects requests that exceed the configured context limit. This is a defensive guardrail, not the primary compaction mechanism. The primary mechanism is OpenCode's native compaction driven by the advertised limits.
Enforcement returns HTTP 400 with protocol-appropriate error envelopes for OpenAI and Anthropic endpoints.
For translated requests, tool-schema allowance is included in the context estimate mathematically. It is not appended to the encoded provider body, so the upstream payload remains unchanged.
model_capabilities and model_overrides are live-reloadable: edit the
config and run eggpool rehash to apply new limits without a restart
(see docs/live-config-rehash.md). A restart is only needed when the same
edit also touches restart-required fields.
The public /v1/models response includes configured effective limits under the
namespaced eggpool.limits object when the catalog knows them:
{
"id": "MiniMax-M3/opencode-go",
"object": "model",
"eggpool": {
"limits": {
"context": 220000,
"output": 16384
}
}
}The dashboard Models table links each model ID to its model-info detail page.
That detail page renders the effective and externally observed context/output
limits when model-info enrichment has populated them. The table itself remains
focused on routing availability, usage, cost, and pricing exactness so the
default Models view stays scannable. Pricing exactness follows the same
request-cost labels used elsewhere in EggPool: provider_reported,
exact, derived, partial, estimated, and unknown.
Per-user request, token, microdollar, and concurrent request budgets are
separate from these model context/output limits. Operators manage rules with
eggpool users budget set (--enforcement hard|estimated, default
estimated); daily and monthly periods use UTC calendar windows. A hold is
reserved atomically before provider dispatch, and retries extend the same
logical request hold. User usage distinguishes provider-reported cost, local
estimates, and conservative reservation fallback.
Each rule is explicitly hard or estimated, visible in the dashboard,
GET /api/user-budget, and the stored audit history. Pre-existing rules are
estimated and stay that way until an administrator explicitly promotes them;
nothing is silently upgraded to a guarantee.
estimated(default): token holds use a bounded request-size estimate and microdollar holds use EggPool's estimated pricing projection, including fallback rates when no model price is known. These are operational approximations, not an exact charge or invoice for flat-rate providers, and not a hard ceiling.hard: token rules admit only requests carrying an explicit caller output cap compatible with the request surface, and reserve a conservative worst-case bound (admission input bound plus that cap, covering the concrete attempt plus selector overhead; retries extend per attempt after re-checking). USD rules additionally require an operator-configured conservative tariff ([providers.<id>.budget] max_microdollars_per_million_tokens); unpriced providers fail closed, an explicit zero tariff means verified free, and estimator fallbacks are never a hard price. Requests with unbounded output, an unknown tariff, or arithmetic overflow are denied with HTTP 429 and machine codehard_budget_not_enforceablebefore upstream submission. The response advises callers to include a positive output cap supported by the API and operators to configure a tariff when hard USD rules apply.budget_exceededidentifies an exhausted rule. Observed provider usage above a hard hold is settled visibly, never clamped.
Operators can optionally cap bursty inference traffic before EggPool reads the request body or enters durable budget reservation. The rule is process-local, uses a monotonic 60-second token bucket, and defaults to disabled:
[admission_fairness]
requests_per_minute_per_principal = 120
global_requests_per_minute = 1200
max_tracked_principals = 4096Each principal's bucket is keyed by its authenticated user ID, so multiple agent credentials for one user share the same allowance. The global bucket, when configured, intersects the per-user rule. Rates are bounded to 1–60,000 requests per minute; tracked principal state is bounded to 1–65,536 entries. If the state table is full of recently active principals, a new principal is rejected with 429 until stale state can be reclaimed; EggPool does not evict a live bucket to grant extra traffic. Limiter state resets on restart, while daily/monthly budgets remain durable and authoritative.
Throttles return HTTP 429, Retry-After, machine code
short_window_rate_limited, and X-EggPool-Admission-Scope (principal,
global, or principal_state_capacity). Admitted responses expose remaining
short-window allowance in X-EggPool-Admission-Remaining, plus the scope and
60-second window. Malformed or oversized requests consume an admission token
because the decision is made before body collection. No-rule installations
take the existing admission path without allocating principal state.
The supported request fields used as output bounds are:
| API surface | Explicit output bound fields |
|---|---|
| Chat Completions | max_completion_tokens, max_tokens |
| Responses | max_output_tokens, max_completion_tokens, max_tokens |
| Compact Responses | max_output_tokens, max_completion_tokens, max_tokens |
| Anthropic Messages | max_tokens |
Synthetic loopback captures of Codex CLI 0.162.1 and OpenCode 1.18.35
confirmed that behavior depends on client model configuration: Codex CLI sent
a streaming Responses request without an output cap, while OpenCode sent
max_output_tokens: 1024 from its configured limit.output. The sanitized
capture record is in
rust/tests/fixtures/on_prem_budget_agent_compatibility.json.
An unbounded request continues to fail before dispatch under hard token rules.
For OpenCode, configure a model output limit so its Responses request carries
the bound. EggPool does not inject or infer a cap for any client.
See the API reference for the
authenticated /api/user-budget view.
One optional instance-wide ceiling is intersected with every per-user rule:
an admitted call must satisfy all applicable instance-global, instance-model,
user-global, and user-model limits. Manage ceilings with
eggpool users global-budget set (same windows, modes, and --model-id
scoping; a blank model covers all models) or the admin dashboard's instance
section. Absent ceilings mean no additional cap, and single-admin behavior is
otherwise unchanged.
The shared ceiling is enforced atomically: concurrent requests from different users cannot both take the final global slot. Retries, alternate models, identity switches, and legacy-key requests cannot bypass it. Unattributed requests count against the ceiling too. Each logical request counts once across failover and retries; hard/estimated semantics and the operator-tariff trust boundary match the per-user contract above. Global counters never depend on a user row, so disabling or deleting a user preserves history and remaining headroom. Members see their own rules, their own settled windows, and the remaining allowance on those rules. They also see the configured instance ceilings. Instance usage totals and other users' rows stay on the administrator response.
Default member profiles do not create or copy budget rules. Configure user and instance ceilings separately; changing the default model profile leaves all existing budget rules and usage unchanged.
The /v1/embeddings route has independent hard caps: 64 KiB request body,
32 text inputs, 16 KiB per text, 64 KiB aggregate UTF-8 text, 8192 requested
or returned dimensions, and 8 MiB provider response body. Inputs must be
non-empty strings. Only finite float vectors are accepted; the vector count
must equal the input count, indices must be unique and contiguous, and all
vectors must have equal length. If dimensions was requested, each vector
must have exactly that length. Base64 vectors and token-ID inputs are not
supported.
These limits do not alter the normal chat context/input/output limits. Hard token budgets use UTF-8 input bytes as a conservative pre-dispatch upper bound; embeddings generate no completion tokens. Hard USD budgets still require an operator tariff, following the ordinary hard-budget rule. Missing provider usage remains unreported rather than being estimated from returned vectors.