Skip to content

feat(advisor): default-off tiered decision advisor with v1-safe evidence - #54

Merged
shreyanshjain7174 merged 6 commits into
mainfrom
feat/decision-advisor
Oct 18, 2026
Merged

shreyanshjain7174 merged 6 commits into
mainfrom
feat/decision-advisor

Conversation

@shreyanshjain7174

Copy link
Copy Markdown
Contributor

Plan

Phase 3 of the AgentGate scale-out prompt: a tiered local decision advisor. Off by default. It stays off until real model numbers exist.

What changed

  • internal/advisor.
    • Advisor interface, no-op, and HTTP client. The client has a strict timeout, a 64 KiB response cap, no redirects, no proxy, and loopback or .svc.cluster.local URLs only.
    • Responses are parsed strictly: unknown fields rejected, probabilities in [0,1].
    • The cascade policy runs in Go with thresholds from config.
  • Registry. risky: true per action, default false. Every non-GET action in the shipped catalogs is tagged.
  • Gateway.
    • The advisor runs only for risky actions, only when AGENTGATE_ADVISOR_URL is set, and only after every deterministic check passes, before vault or upstream access.
    • Escalate only: an allow can become deny with advisor_escalated or advisor_unavailable. A deny never becomes an allow (property test).
    • Failures:
      • 150 ms end-to-end timeout.
      • Timeouts, connection errors and 5xx follow advisor_unavailable (default allow_with_flag).
      • Oversized or truncated input and advisor 4xx are classed as advisor_input_rejected and always escalate, so an agent cannot force a fail-open.
    • The state goes out as {service, action, params}, marshalled without HTML escaping. A task claim inside params cannot raise confidence.
  • Evidence without touching v1. params_sha256 stays DigestParams(params) on every gateway receipt.
    • Advisor evidence goes into a companion v1 receipt in the same transaction, right after the action receipt: Service=advisor, Action=decision:advise, and a reserved AgentKeyID=agentgate:advisor that no API key can have. It is bound to its record through HashRecord per DECISION.md.
    • The record hash-links back to the action receipt's seq, entry hash and params digest.
    • Agent requests for service=advisor or decision:* actions get 400 with no receipt. The registry rejects a service named advisor.
  • Export and verify.
    • The default JSONL export is byte-identical to main before this PR, so released verifiers keep working.
    • ?records=1 adds record lines.
    • agentgate-verify checks companion bindings when records are present, including a companion at the start of a range via the signed manifest anchor. On a default export it reports companions as unchecked. --require-advisor-records makes missing evidence a failure.
    • v1 cannot prove every risky receipt has a companion. That is documented.
  • services/decision-advisor/. A stdlib Python service with /v1/decide, /healthz, /metrics and input limits.
    • Backends: heuristic (deterministic, for tests and CI), plus onnx and vllm-sr adapters. The adapters raise NotImplementedError with doc links, since the runtime APIs are not verified yet; no invented calls.
    • Non-root Dockerfile on a digest-pinned base, weights mounted read-only, no runtime egress.
    • Helm snippets: Tier A on CPU, Tier B on a GPU pool.
  • bench/advisor/. 60 synthetic cases, labeled as synthetic since there are no real receipt logs yet.
  • Docs: README, docs/spec/proposals/advisor-evidence.md (with the v2 field sketch), and a DECISION.md note on opt-in records. The RECEIPT-v1 §7 note is informative only.

Numbers (heuristic backend only)

  • Escalation precision 55%, recall 100% on 60 synthetic cases. The heuristic escalates broadly (confidence 0.7, below the 0.8 threshold).
  • HTTP round trip p50 0.186 ms, p99 0.391 ms.
  • ONNX and GPU numbers are pending, which is why the advisor stays off.

Validation

  • go build ./...: passed
  • go vet ./...: passed
  • go test -race ./...: passed
  • semgrep --config auto --error --quiet ./internal ./cmd ./pkg ./services: passed
  • python3 -m unittest discover in services/decision-advisor: 12 tests, passed
  • RECEIPT-v1 version lock: unchanged

Review

Four rounds by a second model (Claude Opus 5.5). Found and fixed:

  • the original binding changed the meaning of params_sha256;
  • agents could force fail-open with oversized input;
  • task spoofing;
  • agents could poison exports by imitating companion names;
  • leading-companion ranges were rejected;
  • default exports were unreadable by released verifiers and failed this branch's own verifier.

shreyanshjain7174 and others added 6 commits October 18, 2026 08:26
Bind advisor evidence without changing receipt v1.
Store companion records in the receipt transaction.
Block model adapters until the upstream API is verified.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep gateway params digests and sign separate advisor companions.
Reject input failures instead of treating them as unavailable.
Ignore payload task claims and verify exported record bindings.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the gateway state shape without spoofable task claims.
Record conservative approval: 55% precision and 100% recall.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reserve the internal producer ID so requests cannot poison history.
Keep default exports readable by released v1 verifiers.
Verify leading companions against signed manifest anchors.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the reserved identity and signed-anchor verification.
Keep the record extension outside frozen v1 defaults.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Default v1 exports contain no advisor evidence lines.
Keep chain verification usable without hiding unchecked evidence.
Require advisor records when supplied or explicitly requested.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@shreyanshjain7174
shreyanshjain7174 merged commit 4ad6cc8 into main Oct 18, 2026
1 check passed
@shreyanshjain7174
shreyanshjain7174 deleted the feat/decision-advisor branch October 24, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant