fix(provider): overhaul openai-subscription retry policy - #32
Merged
Conversation
Statusless errors like service_unavailable_error/server_is_overloaded failed fatally because retriability was an exact-match allowlist. Classification now defaults to retrying and excludes permanent errors by heuristic instead. - retry budget 10m -> 24h, time-driven only (MAX_RETRIES removed) - backoff cap 30s -> 60s; per-attempt 10m guard on send/credentials - honor x-ratelimit-reset-* on 429 over Retry-After: pace waits at 10m and retry until reset + 10m grace, clamped to the 24h budget - cap all server delay hints at 10m - reopen_stream keeps request_context in place so a cancelled reopen stays retryable from the first-event timeout path
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A turn died on the first
response.failedcarryingservice_unavailable_error/server_is_overloaded: retriability for statusless SSE errors was an exact-match allowlist, and neither the_error-suffixed type nor theserver_is_overloadedspelling was on it.Changes
Classification flipped to default-retry. Statusless errors now retry unless excluded: auth errors (401/403 or auth-shaped codes) and permanent errors (status list plus substring markers:
billing,content_policy,deactivated,insufficient,invalid,not_found,not_supported,permission,quota,unsupported). Unknown transient spellings can no longer kill a turn.Retry budget 10m → 24h, time-driven only (
MAX_RETRIESremoved). Backoff cap 30s → 60s, jitter schedule unchanged. A 10m per-attempt guard (ATTEMPT_TIMEOUT) covers send/credential/first-event operations the stream idle timeout can't; attempt timeouts retry within the budget rather than failing fatally.Rate-limit resets respected. On 429,
x-ratelimit-reset-*headers (max across headers; plain seconds, unit durations like6m30s, epoch s/ms) win overRetry-After. The deadline becomes reset + 10m grace, clamped to the 24h ceiling, with waits paced at most 10m apart. Reset headers on non-429 statuses are ignored (they're quota metadata on every response). All server delay hints are capped at 10m so a bogusRetry-Aftercan't stall a turn.reopen_streamkeepsrequest_contextin place across its awaits instead of taking it, so a reopen cancelled by the first-event timeout stays retryable instead of failing withstream retry context is unavailable.