Repository navigation
Route hourly LLM-calling workflow through contextual-orchestrator (orchestrator/free) in .github/workflows/hourly-product-development.yml #38
Description
What
.github/workflows/hourly-product-development.yml in this repository makes its LLM call directly against a provider
(NVIDIA NIM), bypassing the org's shared contextual-orchestrator gateway
and its orchestrator/free fail-closed zero-cost pool entirely.
OPENCODE_MODEL_CANDIDATES lists nvidia-nim/nvidia/llama-3.3-nemotron-super-49b-v1.5, nvidia-nim/nvidia/nemotron-3-super-120b-a12b, and nvidia-nim/deepseek-ai/deepseek-v4-pro, served through a local NIM broker on 127.0.0.1:8765 — a self-hosted direct-provider proxy, not contextual-orchestrator.
Why this is a violation, not a judgment call
ADR-0003inContextualWisdomLab/.githubscopes itself explicitly to
that repo's three central review pipelines (OpenCode autofix/dispatch,
Noema, Strix) — it does not authorize this repository's own workflow to
call a provider directly, and does not mention this repository at all.- The org owner has directed that all GitHub Actions workflows that make
an LLM call route throughorchestrator/free. ContextualWisdomLab/contextual-orchestrator's own hourly loop
(opencode-hourly-loop.yml) already demonstrates the pattern this
repository's loop should follow: vendor the pinned
contextual-orchestratorsidecar, register the five provider credentials
(BYTEZ_API_KEY,NVIDIA_NIM_API_KEY,NVIDIA_NIM_API_KEY_SUB,
OPENROUTER_API_KEY,OPENAI_API_KEY) into its KV, serve the loopback
gateway, and point OpenCode'smodelat
contextual_orchestrator_gateway/orchestrator/freeinstead of a direct
provider id/base URL. See
ContextualWisdomLab/contextual-orchestrator/pull/1013for a worked
example of the pool-string half of that pattern (that PR does not add the
gateway to a repo that lacks it; this repo's workflow doesn't have the
gateway wired in at all yet, which is the larger piece of work here).
What "fixed" looks like
- Vendor/provision the
contextual-orchestratorsidecar in this workflow
(mirroring.github'sscripts/ci/contextual_orchestrator_review_sidecar.sh
orcontextual-orchestrator's ownopencode-hourly-loop.ymljob). - Register the five provider secrets into the sidecar's KV instead of
passingNVIDIA_API_KEY/NVIDIA_NIM_API_KEYstraight to the direct
provider call. - Point the OpenCode config's
model(and anyOPENCODE_MODEL/
OPENCODE_MODEL_CANDIDATES/enabled_providersoverride) at the gateway's
orchestrator/freevirtual id instead of a directnvidia/nvidia-nim
provider id orintegrate.api.nvidia.combase URL. - Retire the repo-local NIM broker/proxy/bridge process this workflow
currently starts, if one exists, once the gateway sidecar replaces it. - Add or update a workflow-contract test (shell-syntax or string-pinning,
matching this org's usual pattern) covering the changed file, and run it.
This is intentionally left as a tracking issue rather than a same-pass fix:
each of these hourly loops has its own broker/proxy wiring that a blind
string substitution would not safely replace, and getting the sidecar
provisioning right needs repo-specific verification this issue's audit pass
did not have time to do safely.
Filed by the org-wide orchestrator/free routing audit (2026-09-02). See
ContextualWisdomLab/.github's docs/adr/0003-contextual-orchestrator-vendored-free-zdr.md
for the governed-gateway pattern (vendored sidecar, 5-credential KV, free-only
route admission) and docs/product-technical-gap-baseline.md's 2026-09-02
entry for the full audit disposition table.
This is a real, unauthorized violation (case b in the audit), not a half-fix:
routing a hardcoded direct-provider call through the shared gateway is a
bigger change than a model-string swap (own broker/proxy code, secret wiring,
and gateway startup all need to move), so it is being tracked here rather
than attempted blind in the audit pass.
🤖 Filed with Claude Code
Activity
seonghobae commentedon Sep 2, 2026
Verification performed (this pass did the repo-specific check the filing audit explicitly deferred)
I cloned
ThreadWeavefresh and read.github/workflows/hourly-product-development.ymlend to end,plus
scripts/ci/nim_proxy.py(380 lines) and every test/doc that references it. I also clonedContextualWisdomLab/.githubandContextualWisdomLab/contextual-orchestratorfresh and traced theactual request-handling code path (
server.py→orchestrator.py'sModelClient.request_settings),not just the workflow YAML.
Correcting the lead:
contextual-orchestrator#1013is not merged (state: open,merged: falseat verification time), and even once merged it only fixes the pool string(
orchestrator/auto→orchestrator/free) forcontextual-orchestrator's own hourly loop — whichruns its own in-repo gateway code directly (
python -m scripts.ci.serve_seeded_gateway), vendoringnothing. So there is currently no working cross-repo vendoring example for a downstream product
repo, only
.github's own local sidecar script (which never leaves.github).One real concern from the lead, resolved
I initially suspected the review sidecar's
ModelClient(max_output_tokens=REVIEW_MAX_OUTPUT_TOKENS=4096)(
contextual_orchestrator_review_launcher.py) would silently truncate a code-writing agent's turns(ThreadWeave's current NVIDIA config declares
"output": 8192). Traced throughorchestrator.py:request_settings(): the 4096 value is only a fallback used when the caller's requestomits
max_tokens; the caller's own value always wins when present..github's ownopencode.jsoncalready declares"orchestrator/free": {"limit": {"output": 32768}}for itsreview agents — higher than ThreadWeave's current ceiling — so this is not a blocker, just
something that needed tracing rather than assuming.
The actual blocker
ThreadWeaveenforces 100% statement+branch coverage (pyproject.toml,source = scripts/ci,fail_under = 100) and 100% docstring coverage (interrogate,fail-under = 100) on every scriptunder
scripts/ci/, gated by an explicit--include=allowlist inci.ymlthat must name everycovered file.
nim_proxy.py(380 lines) currently earns that bar via two dedicated test filestotaling 446 lines (
tests/test_nim_proxy.py,tests/test_nim_tls_context.py).The only proven vendoring precedent —
.github'scontextual_orchestrator_review_sidecar.sh(691lines of bash) plus
contextual_orchestrator_review_launcher.py(964 lines),contextual_orchestrator_review_policy.py(528 lines),zdr_policy.py(231 lines), and the logsanitizer (100 lines) — is ~1,823 lines of new Python, purpose-built and named for review-only
(
"edit": "deny") agents. None of it has any test coverage inThreadWeave, and.github's owndedicated contract-test file for just the sidecar shell script alone has 29 test functions (there are
more files covering the launcher and policy separately). Landing this safely in
ThreadWeavemeanseither:
repo doesn't semantically own (duplicating test investment
.githubalready made once, for thereview use case specifically — not the code-writing/bash-executing use case
ThreadWeaveactuallyneeds), or
(this repo's
CLAUDE.md: "100% coverage and 100% docstrings onscripts/ci/are hard gates, notaspirations") forbid doing casually.
Separately,
ThreadWeavewould be the first repo to route a code-writing, bash-executingautonomous agent (sandboxed under an unprivileged UID with
env -i) through this gateway — everyexisting consumer (
noema-review,opencode-review-dispatch,pr-review-autofix,strix) is areview-only agent that never edits or executes. The credential-isolation property (sandboxed process
gets a random per-run bearer, never a real provider key) does carry over cleanly — I verified
ORCHESTRATOR_TOKENis always freshly generated, never one of the five real secrets — but there is noprecedent run to cross-check the gateway's actual behavior under this repo's stricter sandbox before
trusting it in production.
What would move together in any real fix
tests/test_workflows.py(multiple exact-string assertions on the NIM broker wiring),tests/test_nim_proxy.py,tests/test_nim_tls_context.py,tests/test_hourly_product_egress_contract.py,tests/test_hourly_product_secret_fingerprint.py,tests/test_autonomous_documentation.py,tests/test_architecture_documentation.py,docs/adr/0006-work-conserving-autonomous-maintenance.md(Status: Accepted — explicitly names
NVIDIA_NIM_API_KEYas "the model execution boundary"),docs/operations/hourly-autonomous-maintenance.md, andARCHITECTURE.md.Recommendation
Keep this tracked; a blind swap risks either a broken/truncated autonomous dev loop or a gamed
coverage suite written under time pressure. What would unblock a real fix:
.githubinto a small, independently-tested, installablecomponent (e.g. published from
contextual-orchestratoras a hash-pinned package) thatThreadWeavecan depend on the way it depends on any other pinned dependency — so the coveragegate treats it as third-party code, not first-party code needing local 100% coverage.
once centrally rather than invented ad hoc per repo.
after) the workflow-wiring change.
No code was changed in this pass.
🤖 Generated with Claude Code
seonghobae commentedon Sep 30, 2026
2026-09-30 boundary correction: the issue's original "vendor/provision contextual-orchestrator source in this leaf" target is no longer an acceptable GREEN. Draft PR #43 correctly preserves the
orchestrator/freeintent but remains incomplete because ThreadWeave still owns provider-secret inventory and mutable contextual-orchestrator source/bootstrap. Completion now requires an immutable contextual-orchestrator owner release plus a versioned central reusable caller contract; ThreadWeave must consume only that released contract as a thin fail-closed caller. Keep the issue and PR open/Draft; do not copy owner source, read owner DB, use a temporary branch, or fall back to a paid/direct provider.