From dde453d1e88c9f1c9abddc2c65a552ab213414bd Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Thu, 3 Sep 2026 02:18:23 +0900 Subject: [PATCH 1/2] docs(gap-baseline): record SDK-parity (33-37) scoping, reclassify item 32 Read contextual-orchestrator's server.py (8744 lines), batch_routing.py, and file_registry.py directly from protected main to build an evidence- based gap list against the OpenAI SDK, rather than guessing scope. Already implemented (broader than expected): chat/completions, responses, completions (legacy), embeddings, models, files, batch/embeddings, videos, images/generations, audio/{speech, transcriptions,generations}, plus a non-standard rerank extension. Genuinely missing: moderations, fine_tuning/jobs, audio/translations, images/{edits,variations}, uploads, realtime (WebSocket -- flagged as needing its own ADR before implementation, given the architectural shift to a persistent connection). Assistants API family (assistants/ threads/runs/vector_stores) noted as missing but likely low priority, since OpenAI itself is sunsetting it in favor of the already-implemented Responses API. Item 32 (batch scope) reclassified: batch_routing.py already has a per-request RoutingPolicy dispatching to pg-llm-batch, so this needs re-verification against the item's actual concern, not new implementation from a blank slate. Item 36 (S3-backed Files) confirmed as a real gap: current file_registry.py is a provider-affinity proxy, not a self-hosted object-storage-backed implementation. Implementation deliberately deferred per this session's throttle agreement: the GitHub Actions capacity crunch this document already records (60-slot ceiling) was re-checked before this entry and is unimproved (2154 queued / 2-4 in-progress) -- opening new feature PRs now would add to it rather than help clear it. Co-Authored-By: Claude Sonnet 5 --- docs/product-technical-gap-baseline.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 7888a5e04a..7ae484ebba 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2627,3 +2627,27 @@ Higgins, S. S., Crepalde, N., & Fernandes, L. (2021). Segmented multiplexity: A **900-second clarification.** The historical `NoemaRepairDeadlineExceeded` from the html4tree incident came from the retired caller repair path. The three literal `timeout --kill-after=20 900` invocations still present in `opencode-review-dispatch.yml` are separate containment limits for untrusted test-measurement commands; they are not model or Noema inference timeouts. Telemetry and runbooks must report the command class and phase separately. **Evidence / acceptance.** Permanent tests forbid retry/deadline/sampling symbols in the caller and prove one gateway request, one attempt annotation, control-character-safe telemetry, missing-value rejection, valid trailing-comma normalization, and exact changed-line guidance. Fresh exact-head repository checks and reviews remain the admission authority; predecessor-head evidence is not transferable. The remaining runtime work is to preserve distinct `request_too_large`, discovery, rate-limit, provider transport, malformed-output, stale-head, and sandbox-command-timeout categories in hosted logs. + +## Backlog items 33-37 (OpenAI SDK parity) scoping, and item 32 (batch scope) reclassification — 2026-09-02 + +**Status:** Scoping only, recorded per this session's throttle agreement. Implementation is deliberately deferred — opening new feature PRs right now would add to an already-saturated review queue (2154 queued / 2-4 in-progress GitHub Actions runs at the time of this entry, unchanged or worse than the 60-concurrent-runner-slot ceiling already documented elsewhere in this file) rather than clear it. Resume only once that queue meaningfully eases, or the repository owner explicitly directs otherwise — not on this session's own initiative. + +**Method.** Read `contextual-orchestrator/contextual_orchestrator/server.py` (8744 lines, a raw `http.server`-based OpenAI-compatible gateway, not a framework router) directly from protected `main`, plus `batch_routing.py` and `file_registry.py`, and grepped for every major OpenAI SDK endpoint family's path string. + +**Already implemented** (broader than expected): `/v1/chat/completions`, `/v1/responses`, `/v1/completions` (legacy, redirects to chat/completions), `/v1/embeddings`, `/v1/models`, `/v1/files`, `/v1/batch/embeddings`, `/v1/videos`, `/v1/images/generations`, `/v1/audio/{speech,transcriptions,generations}`, and `/v1/rerank` (a non-standard extension, not in the OpenAI SDK). + +**Genuinely missing** (item 34 candidates — in the SDK, absent here): +- `/v1/moderations` +- `/v1/fine_tuning/jobs` +- `/v1/audio/translations` +- `/v1/images/edits`, `/v1/images/variations` +- `/v1/uploads` +- `/v1/realtime` (WebSocket, persistent-connection architecture — genuinely different shape from every other endpoint here; deserves its own ADR before any implementation attempt, not a same-pattern addition) + +**Missing but likely low priority:** `/v1/assistants`, `/v1/threads`, `/v1/runs`, `/v1/vector_stores` (the Assistants API). OpenAI itself is sunsetting this family in favor of the Responses API, which this gateway already implements (`/v1/responses`). Building parity for a family the SDK's own vendor is deprecating is in tension with item 34's "if the SDK has it, implement it" principle — recorded as a judgment call to deprioritize, not a decision to never build it. + +**Item 32 (batch endpoint scope) reclassified: not a blank-slate gap, needs re-verification instead.** `batch_routing.py` already implements a `RoutingPolicy` that decides sync-vs-batch **per request**, driven by request hints and KV-configured thresholds, dispatching batch-eligible requests to `pg-llm-batch` through an injected OpenAI-compatible `BatchAPIClient`. This may already satisfy item 32's concern ("should the batch endpoint only serve batch models") as a policy decision rather than a hard endpoint restriction — confirming that needs a closer read of `RoutingPolicy`'s actual thresholds and call sites, not new implementation. Left open for whoever picks this up next. + +**Item 36 (S3-backed Files API) confirmed as a real gap.** `file_registry.py`'s current `FileOwner`/file-registry logic is a *provider-affinity* registry — it tracks which upstream provider a given gateway file id was proxied to, not a self-hosted store. No S3 (or other object-storage) integration exists. Item 36's "Files API can be self-implemented via S3" is not yet built. + +**Suggested implementation order, if/when resumed** (smallest and most independent first, one PR each, not bundled): `/v1/moderations` → `/v1/audio/translations` (extends the existing audio family) → `/v1/images/{edits,variations}` → item 36's S3-backed Files store (infrastructure-shaped, kept separate) → `/v1/uploads` → `/v1/fine_tuning/jobs` (needs persistent job state, more complex) → `/v1/realtime` last, gated behind its own ADR given the architectural shift to a persistent connection model. Item 22 (Keyverse) was scoped in parallel and found mostly already covered by `keyverse#103`'s ADR-accepted draft (ABAC/RBAC PDP + KV + credential store); see that item's own gap-baseline entry for detail. From 208d8b7e260d0c02f22afdd6a7585c179255a786 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 22:45:56 +0000 Subject: [PATCH 2/2] docs(gap-baseline): correct item-32 batch-scope over-claim, pin SHA, fix xref Addresses three Devin review findings: - "Batch scope gap is misclassified": the entry claimed RoutingPolicy already gates the batch endpoint to batch-capable models and that eligible requests dispatch to pg-llm-batch through an injected BatchAPIClient. Re-read server.py/cost_router.py directly: /api/v1/batch_routing_jobs calls coordinator.submit_batch() directly, which never consults RoutingPolicy; and CostRoutingCoordinator defaults batch_backend to LocalBatchBackend unless a caller injects one, which server.py's one constructor call does not do (PgLlmBatchBackend is never instantiated outside tests/). Corrected the entry to keep item 32 open pending real verification, with exact file:line evidence for the corrected claim. - "External evidence lacks a revision pin": pinned the server.py citation to the exact commit inspected. - "Cross-repository reference is not linkable": `keyverse#103` -> the binding-convention form `ContextualWisdomLab/keyverse#103`. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4 --- docs/product-technical-gap-baseline.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 7ae484ebba..c4260876ee 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2632,7 +2632,7 @@ Higgins, S. S., Crepalde, N., & Fernandes, L. (2021). Segmented multiplexity: A **Status:** Scoping only, recorded per this session's throttle agreement. Implementation is deliberately deferred — opening new feature PRs right now would add to an already-saturated review queue (2154 queued / 2-4 in-progress GitHub Actions runs at the time of this entry, unchanged or worse than the 60-concurrent-runner-slot ceiling already documented elsewhere in this file) rather than clear it. Resume only once that queue meaningfully eases, or the repository owner explicitly directs otherwise — not on this session's own initiative. -**Method.** Read `contextual-orchestrator/contextual_orchestrator/server.py` (8744 lines, a raw `http.server`-based OpenAI-compatible gateway, not a framework router) directly from protected `main`, plus `batch_routing.py` and `file_registry.py`, and grepped for every major OpenAI SDK endpoint family's path string. +**Method.** Read `contextual-orchestrator/contextual_orchestrator/server.py` (8744 lines, a raw `http.server`-based OpenAI-compatible gateway, not a framework router) directly from protected `main` (commit `212ff437dc297613289dba2e6064ade9942e07d8`), plus `batch_routing.py` and `file_registry.py`, and grepped for every major OpenAI SDK endpoint family's path string. **Already implemented** (broader than expected): `/v1/chat/completions`, `/v1/responses`, `/v1/completions` (legacy, redirects to chat/completions), `/v1/embeddings`, `/v1/models`, `/v1/files`, `/v1/batch/embeddings`, `/v1/videos`, `/v1/images/generations`, `/v1/audio/{speech,transcriptions,generations}`, and `/v1/rerank` (a non-standard extension, not in the OpenAI SDK). @@ -2646,8 +2646,8 @@ Higgins, S. S., Crepalde, N., & Fernandes, L. (2021). Segmented multiplexity: A **Missing but likely low priority:** `/v1/assistants`, `/v1/threads`, `/v1/runs`, `/v1/vector_stores` (the Assistants API). OpenAI itself is sunsetting this family in favor of the Responses API, which this gateway already implements (`/v1/responses`). Building parity for a family the SDK's own vendor is deprecating is in tension with item 34's "if the SDK has it, implement it" principle — recorded as a judgment call to deprioritize, not a decision to never build it. -**Item 32 (batch endpoint scope) reclassified: not a blank-slate gap, needs re-verification instead.** `batch_routing.py` already implements a `RoutingPolicy` that decides sync-vs-batch **per request**, driven by request hints and KV-configured thresholds, dispatching batch-eligible requests to `pg-llm-batch` through an injected OpenAI-compatible `BatchAPIClient`. This may already satisfy item 32's concern ("should the batch endpoint only serve batch models") as a policy decision rather than a hard endpoint restriction — confirming that needs a closer read of `RoutingPolicy`'s actual thresholds and call sites, not new implementation. Left open for whoever picks this up next. +**Item 32 (batch endpoint scope) reclassified — corrected per Devin review, item stays open, not "policy already covers it."** `batch_routing.py` does implement a `RoutingPolicy` (`cost_router.py:106`, `self.policy = routing_policy or RoutingPolicy(...)`) that `CostRoutingCoordinator` uses to decide sync-vs-batch for *ordinary* `/v1/chat/completions`-shaped calls. But re-reading `server.py` and `cost_router.py` directly (at `main` commit `212ff437dc297613289dba2e6064ade9942e07d8`) shows this does **not** substantiate the original claim: the explicit `/api/v1/batch_routing_jobs` endpoint (`server.py:7443`) calls `coordinator.submit_batch(...)` (`:7453`) directly — `submit_batch` (`cost_router.py:970`) resolves the request's target model via `_resolve_batch_request` (only for ZDR-tagged requests) and hands off to `self.batch_backend.submit(...)`; neither path consults `self.policy`/`RoutingPolicy` at all, so nothing restricts this endpoint to batch-capable models. Separately, `CostRoutingCoordinator.__init__` (`cost_router.py:117-127`) defaults `self.batch_backend` to `LocalBatchBackend` unless a caller explicitly injects one, and `server.py`'s only constructor call (`server.py:5487`, `CostRoutingCoordinator(orchestrator)`) does not inject one — `PgLlmBatchBackend` (the class that wraps `pg_llm_batch.BatchAPIClient`) is never instantiated anywhere outside `tests/`. So "dispatching batch-eligible requests to pg-llm-batch through an injected BatchAPIClient" does not describe the server's actual default wiring. Item 32 remains open pending real verification of `/api/v1/batch_routing_jobs`'s actual model-resolution and backend wiring — not resolved by `RoutingPolicy`'s existence for a different code path. **Item 36 (S3-backed Files API) confirmed as a real gap.** `file_registry.py`'s current `FileOwner`/file-registry logic is a *provider-affinity* registry — it tracks which upstream provider a given gateway file id was proxied to, not a self-hosted store. No S3 (or other object-storage) integration exists. Item 36's "Files API can be self-implemented via S3" is not yet built. -**Suggested implementation order, if/when resumed** (smallest and most independent first, one PR each, not bundled): `/v1/moderations` → `/v1/audio/translations` (extends the existing audio family) → `/v1/images/{edits,variations}` → item 36's S3-backed Files store (infrastructure-shaped, kept separate) → `/v1/uploads` → `/v1/fine_tuning/jobs` (needs persistent job state, more complex) → `/v1/realtime` last, gated behind its own ADR given the architectural shift to a persistent connection model. Item 22 (Keyverse) was scoped in parallel and found mostly already covered by `keyverse#103`'s ADR-accepted draft (ABAC/RBAC PDP + KV + credential store); see that item's own gap-baseline entry for detail. +**Suggested implementation order, if/when resumed** (smallest and most independent first, one PR each, not bundled): `/v1/moderations` → `/v1/audio/translations` (extends the existing audio family) → `/v1/images/{edits,variations}` → item 36's S3-backed Files store (infrastructure-shaped, kept separate) → `/v1/uploads` → `/v1/fine_tuning/jobs` (needs persistent job state, more complex) → `/v1/realtime` last, gated behind its own ADR given the architectural shift to a persistent connection model. Item 22 (Keyverse) was scoped in parallel and found mostly already covered by `ContextualWisdomLab/keyverse#103`'s ADR-accepted draft (ABAC/RBAC PDP + KV + credential store); see that item's own gap-baseline entry for detail.