Skip to content

feat(agent): support Standard Schema validators - #107

Open
LukasParke wants to merge 16 commits into
mainfrom
agent/agent-standard-schema
Open

LukasParke wants to merge 16 commits into
mainfrom
agent/agent-standard-schema

Conversation

@LukasParke

@LukasParke LukasParke commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds Standard Schema v1 support to @openrouter/agent, allowing tool input, output, event, context, shared-context, check, and custom-hook schemas to use validators such as Valibot, ArkType, and Effect Schema without requiring Zod schemas.

What changed

  • added a small Standard Schema adapter around schema['~standard'].validate(value) with sync and async validation support
  • preserved Zod v4 as the detected fast path using the existing z4.parse, safeParse, and z4.toJSONSchema behavior
  • widened tool, agent-tool, shared-context, check-schema, and custom-hook schema types while preserving Zod inference
  • normalized Standard Schema issues into the existing tool validation error shape
  • added wireInputSchema as the explicit wire-schema escape hatch for non-Zod input validators
  • kept recursive ~-key sanitization for generated and caller-supplied JSON Schema
  • documented Standard Schema usage and added a minor changeset

Design decisions

JSON Schema strategy

Input JSON Schema conversion now uses a three-tier chain:

  1. Zod keeps the existing z4.toJSONSchema(..., { target: 'draft-7' }) fast path, including Zod versions older than 4.2.
  2. Other validators may expose the StandardJSONSchemaV1 companion trait; the agent calls schema['~standard'].jsonSchema.input({ target: 'draft-07' }) and falls through if conversion throws.
  3. wireInputSchema remains the deterministic explicit override and fallback. Explicit caller intent wins over the trait.

This adds no runtime dependency: StandardJSONSchemaV1 ships in the existing @standard-schema/spec dependency. Zod 4.2+, ArkType 2.1.28+, Zod Mini, VineJS, and Sury implement the trait natively; Valibot supports it via toStandardJsonSchema() from @valibot/to-json-schema.

Only input schemas need conversion because output, event, context, shared-context, check, and custom-hook schemas remain local validators. Generated and supplied schemas retain recursive ~-key sanitization.

SDK boundary

The agent converts every client tool to the SDK's existing function-tool wire shape before making a request. The boundary remains the already-untyped parameters: Record<string, unknown> JSON Schema handoff, so this PR does not depend on the parallel @openrouter/sdk Standard Schema work. No SDK schema types are exposed or cast into the agent's public validator surface.

Context mutation

Initial context validation supports any synchronous Standard Schema validator. ctx.setContext() and ctx.setSharedContext() remain synchronous APIs, so they reject validators whose ~standard.validate returns a Promise. Zod keeps its existing per-field partial-update behavior; generic Standard Schema context updates validate the merged context object.

Test coverage

  • Zod regression: validation and JSON Schema generation
  • Valibot input transforms and output validation
  • Valibot event and context validation
  • Standard Schema issue-path mapping into existing errors
  • StandardJSONSchemaV1 conversion, throw fallback, explicit override, and ~ metadata sanitization
  • missing wireInputSchema failure for non-Zod tools
  • async Standard Schema validation through tool execution
  • compile-time inference for Standard Schema input/output/event/context and unchanged Zod inference
  • full monorepo build, typecheck, lint, and unit tests

Verification:

  • pnpm build
  • pnpm typecheck
  • pnpm lint
  • pnpm test — 918 agent tests and 168 MCP tests passed

API example

import { tool } from '@openrouter/agent';
import { toStandardJsonSchema } from '@valibot/to-json-schema';
import * as v from 'valibot';

// Trait path: toStandardJsonSchema exposes StandardJSONSchemaV1, so no
// wireInputSchema is needed.
const search = tool({
  name: 'search',
  inputSchema: toStandardJsonSchema(v.object({ query: v.string() })),
  outputSchema: v.object({ results: v.array(v.string()) }),
  execute: async ({ query }) => ({ results: await searchWeb(query) }),
});

// Escape hatch: validation-only Standard Schema inputs supply the
// provider-facing JSON Schema explicitly (always wins when present,
// including for Zod schemas that z.toJSONSchema cannot convert).
const lookup = tool({
  name: 'lookup',
  inputSchema: v.object({ id: v.pipe(v.string(), v.transform(Number)) }),
  wireInputSchema: {
    type: 'object',
    properties: { id: { type: 'string' } },
    required: ['id'],
  },
  execute: async ({ id }) => db.get(id), // id is the transformed number
});

Naming: wireInputSchema (unified with #118)

This PR used to add inputJsonSchema for all tools. #118 then added wireInputSchema for manual tools only. Both did the same job, so this PR now keeps one name and one code path:

  • Name: wireInputSchema. Neither name has shipped (npm @openrouter/agent@0.11.0 has neither). wireInputSchema is already on main and in the pending 1.0.0 release (chore: version packages #119), and the port bots may already be tracking it. inputJsonSchema was only ever on this branch.
  • Scope: it now works on every tool kind, not only execute: false tools. Validation-only Standard Schema inputs need it to execute. The manual-only type restriction from feat(agent): allow manual tools to supply a wire input schema #118 is removed.
  • Code path: convertToolsToAPIFormat calls convertSchemaToJsonSchema(inputSchema, wireInputSchema) for every client tool. An explicit wireInputSchema is sanitized and wins over Zod and the StandardJSONSchemaV1 trait. The isManualTool branch is gone.
  • The type is still Readonly<Record<string, unknown>>, as in feat(agent): allow manual tools to supply a wire input schema #118, so existing callers compile unchanged.

🤖 Generated with Claude Code

@LukasParke
LukasParke marked this pull request as ready for review August 11, 2026 23:02
devin-ai-integration[bot]

This comment was marked as resolved.

- reject thenable-returning validators (not just instanceof Promise) in
  validateSchemaSync so async validators can't silently pass setContext
- restore Zod parity for Standard Schema context updates: filter unknown
  keys and store the validator's (possibly transformed) output values
- reword async-validator error; fix stale isVoidSchema doc; use the
  ObjectSchema alias for sharedContextSchema
Importing ObjectSchema from ./schema.js added a 16th internal edge to
model-result.ts, tripping sentrux's no_god_files gate. Revert to the
external type imports (unresolved, uncounted) to stay at fan-out 15.
devin-ai-integration[bot]

This comment was marked as resolved.

…and error propagation

- convertSchemaToJsonSchema: explicit inputJsonSchema now wins for Zod
  too (z4.toJSONSchema throws on unrepresentable constructs, where the
  escape hatch is the only way through)
- validatePartialAgainstSchema: Object.hasOwn instead of `in` so
  prototype-named keys (constructor, toString) are filtered again
- unifiedExecutionResult: normalize caught values eagerly so
  `throw undefined` / Promise.reject() still surface as tool errors
- isVoidSchema: probe Standard Schema validators with undefined so
  v.void() custom hooks skip result validation like z.void()
devin-ai-integration[bot]

This comment was marked as resolved.

…void probe, changeset example

- validatePartialAgainstSchema: persist raw caller-supplied values (Zod
  parity); validator output is only used to filter unknown keys. Storing
  transformed output poisoned the store for type-changing validators.
- isVoidSchema: a Standard Schema is void only if it accepts undefined
  AND rejects null/string/number/object probes, so v.any()/v.optional()
  keep result validation like their Zod equivalents.
- changeset: add the fenced consumer example required by
  .agents/skills/public-api-examples.
devin-ai-integration[bot]

This comment was marked as resolved.

…alidators

The unified log sink used validateSchemaSync, so a tool declaring an
async Standard Schema event validator threw out of its own body on the
first ctx.log while the same event passed via yield. Zod keeps the sync
throw; non-Zod entries are validated out-of-band and forwarded on
success (invalid ones dropped with a warning).
devin-ai-integration[bot]

This comment was marked as resolved.

Value probing cannot distinguish 'accepts only undefined' from
'undefined | T' (v.optional(v.object(...)) rejects every finite sentinel
set), so the sentinel probe could silently disable result validation for
ordinary optional schemas. Non-Zod result schemas are now always
validated: side-effect-only handlers returning undefined still pass a
v.void() result schema, and handlers returning real values on a void
hook are warned — stricter than Zod, but sound.
devin-ai-integration[bot]

This comment was marked as resolved.

…Name threading

- Union schema-aware safe-parse helper (safeParseSchemaSync) into approval
  gates so non-Zod validators work on the #94 paths
- Port correlated-event conditional types to ObjectSchema/InferSchemaOutput
- Keep main's curried TShared overload, typed via InputSchemaConfig
- Update stale valibot test: valibot >= 1.4 carries the JSON Schema trait
  natively; use a trait-less validator for the missing-json-schema case

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 2 additional findings in Devin Review.

Open in Devin Review

Comment thread packages/agent/tsconfig.typecheck.json Resolved
Resolve conflicts with #118 (manual-tool wireInputSchema) and #110 (dropped
preliminaryResults). Neither inputJsonSchema nor wireInputSchema has shipped
(npm 0.11.0 has neither), and wireInputSchema is already on main and in the
pending 1.0.0 release, so keep that name. Drop inputJsonSchema and widen
wireInputSchema from manual tools to every tool kind. One code path now
serializes it: convertSchemaToJsonSchema. Also type-check
manual-tool-wire-schema.test-d.ts, which was not in tsconfig.typecheck.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@w0nche0l w0nche0l left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENT-WRITTEN: Review of f61331f. The design looks sound to me, and I'd like two behavior points settled before this lands. Four inline comments follow.

What I checked:

  • The branch is 12 commits behind main and CI last ran on 2026-10-21. I merged main locally: the merge is clean, and typecheck, lint and all 1,268 unit tests pass on the result. I did not run e2e.
  • A validator with no Zod and an explicit wireInputSchema works for both sync and async validate. The wire schema reaches parameters as written (2020-12 $schema and oneOf kept).
  • The inline findings come from small probes run on that merged tree, except where a comment says it was read from the code.

Context: we want this for MCP tools. Today wrapMcpTool converts the server's JSON Schema to Zod, and z.fromJSONSchema rejects valid arguments for some real schemas (Shopify's catalog like field: untyped oneOf branches). With this PR an MCP tool can carry a JSON Schema validator plus the server's schema as wireInputSchema, with no conversion. #146 overlaps with this PR on wireInputSchema and conflicts in tool-executor.ts and tool-types.ts; I'd rather land this first and reduce #146 to the MCP wrapper change.

Comment on lines +716 to +727
if (isZodSchema(fn.eventSchema)) {
onYield(validateSchemaSync(fn.eventSchema, entry));
return;
}
void safeValidateSchema(fn.eventSchema, entry).then((validation) => {
if (validation.success) {
onYield(validation.data);
} else {
console.warn(
`[tool] ${fn.name}: dropping invalid log entry: ${validation.error.message}`,
);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENT-WRITTEN: ctx.log changes behavior for every non-Zod event schema, including synchronous ones.

This branch is taken for any non-Zod eventSchema, and safeValidateSchema is always async. I ran the same tool with a Zod and a synchronous Valibot eventSchema, logging a valid entry, an invalid one, then a valid one:

Zod Valibot (sync)
Invalid entry throws ZodError in the tool body does not throw; dropped with console.warn
First event delivered before the next statement after the next statement

So a synchronous validator loses the throw and the ordering although nothing about it needs the out-of-band path.

Suggestion: call validate once, and if the result is not a thenable, handle it synchronously exactly like the Zod branch (throw on issues, onYield in order). Keep the out-of-band path only for a real promise.

// exactly the values `execute` will receive.
if (typeof requireApproval === 'function') {
const parsed = z4.safeParse(tool.function.inputSchema, toolCall.arguments);
const parsed = safeParseSchemaSync(tool.function.inputSchema, toolCall.arguments);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENT-WRITTEN: An async input validator makes the tool-level requireApproval predicate unreachable.

safeParseSchemaSync returns success: false for a validator whose validate returns a promise, so this falls into the fail-closed branch. With a predicate that returns false:

Input validator toolRequiresApproval Predicate called
sync false yes
async true no

Failing closed is the safe direction, but it is silent: the tool always asks for approval and the predicate never runs. The README says contextSchema needs a synchronous validator and that validators "may validate synchronously or asynchronously"; it does not mention this gate.

Either await the validator here (this function is already async), or document the limit and reject async input validators on tools that set requireApproval.

}

const parsed = z4.safeParse(tool.function.inputSchema, toolCall.arguments);
const parsed = safeParseSchemaSync(tool.function.inputSchema, toolCall.arguments);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENT-WRITTEN: The call-level check receives unparsed arguments when the input validator is async.

Same cause as the comment below. With a sync validator the check gets parsed.data (defaults and transforms applied); with an async one it gets the raw wire arguments, because the sync parse reports failure. The comment further down explains why the predicate must see the values execute will receive, and that no longer holds here for async validators.

Awaiting the validator fixes both sites.

return 'ready';
}
const parsed = z4.safeParse(tool.function.inputSchema, effective.arguments);
const parsed = safeParseSchemaSync(tool.function.inputSchema, effective.arguments);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENT-WRITTEN: Read from the code, not run: with an async input validator, a PreToolUse hook that mutates the input looks like it is always blocked here as "PreToolUse produced invalid input", since safeParseSchemaSync reports failure for a promise-returning validator. validatePreparedMutation is already async, so it could await the validator.

Related test gap: standard-schema.test.ts has no case for requireApproval, PreToolUse or sharedContextSchema with a Standard Schema validator. One case each for a sync and an async validator would pin the behavior chosen for the two comments in conversation-state.ts.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants