Skip to content

test(sdk): read evaluation state as json parts - #178

Merged
HugoRCD merged 1 commit into
mainfrom
test/evaluation-state-parts
Oct 8, 2026
Merged

HugoRCD merged 1 commit into
mainfrom
test/evaluation-state-parts

Conversation

@HugoRCD

@HugoRCD HugoRCD commented Oct 8, 2026

Copy link
Copy Markdown
Member

🔗 Linked issue

None. The scheduled eve canary failed: https://github.com/vercel-labs/github-tools/actions/runs/37739369560

📚 Description

ai 7.0.133 changed how experimental_evaluate passes state to evaluation models. Every state form is converted to an array of parts, so an object state now reaches the provider as [{ type: 'json', value }]. See the ai@7.0.133 release notes.

Two tests in src/evaluation.test.ts compared the raw state received by the mock evaluation model with a plain object, so they failed when the canary bumped ai to latest:

  • requireApproval: 'auto' > skips approval for a low-risk call the user asked for
  • createGithubAgent({ preset: 'auto' }) > asks one question per preset except maintainer

The new evaluationState test helper unwraps a single json part and returns any other state unchanged. The assertions themselves are unchanged. The SDK still passes the same object state, so runtime code is not affected.

Local checks: SDK typecheck, lint and test pass (176 tests) on the pinned ai 7.0.130. They also pass after applying the canary's dependency bump (ai 7.0.133 with latest eve and @ai-sdk/*), which was not committed.

This change only touches tests, so it has no changeset.

📝 Checklist

  • I have linked an issue or discussion.
  • I have updated the documentation accordingly.

@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
github-tools-docs Ready Ready Preview, v0 Oct 8, 2026 7:34am UTC
github-tools-test-agent Ready Ready Preview, v0 Oct 8, 2026 7:34am UTC

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Thank you for following the naming conventions! 🙏

@HugoRCD
HugoRCD merged commit 7b1d3d7 into main Oct 8, 2026
16 checks passed
@HugoRCD
HugoRCD deleted the test/evaluation-state-parts branch October 8, 2026 07:36

This branch was successfully deployed

2 active deployments
Preview – github-tools-test-agent — e2a5dfe9 Deployed Oct 8, 2026 by vercel[bot]
Preview – github-tools-docs — e2a5dfe9 Deployed Oct 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant