Repository navigation
🤖 tests: replay and fallback coverage for the toolInstructions record - #7330
Conversation
- replayVerify: a Tool: section change mid-segment, recorded the way a live turn builds it (frozen descriptions plus a <tool-instructions> row), replays through readToolInstructionsRecord and buildReplayRequest to the same prompt. The fixture gets a liveToolInstructions mode that records the request with assemblePromptPayload and writes its system-section records on the assistant row. - turnContextAssembler: a description rewritten after getToolsForModel sends as before with no record and no row; a corrupt frozen record keeps the live description and sends the live text in a row. --- _Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_ <!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8dd31622d6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
🟢 e2e stories (Haiku): 14 passed27 agent steps · 7 replayed from cache · 32 model calls · 230.5k tokens (75% cached) All 14 tests in 9 files
e2e 0.20.0 · 2m 58s · web · run artifacts 🟢 e2e Electron stories (Haiku): 3 passed3 agent steps · 3 model calls · 8.0k tokens (32% cached) All 3 tests in 3 files
e2e 0.20.0 · 19.7s · electron · run artifacts 🟢 e2e agent repros (Haiku): no tests selectedSpend: $0.00 of the $1.00 cap (proxy ledger) e2e 0.20.0 · 5.3s · 2 targets (web, phone) · run artifacts |
… turns The liveToolInstructions fixture path built its request without the volatile tail split and other request-time inputs the envelope logs, and with recordDevtools: false it skipped the assembly, so the assistant row lost the records production writes. The fixture now builds the request on every turn with the same inputs and history preparation as replay, logs the sent system content in the envelope as production does, and gates only the DevTools entries on recordDevtools. Addresses Codex PRRT_kwDOPxxmWM6vksx- and PRRT_kwDOPxxmWM6vksyK. --- _Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_ <!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
|
@codex review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Summary
Tests for three gaps in the
Tool:section freeze from #7323, as its Merge Advisor asked. No production code changes. The tests found no bug.Implementation
replayVerify.test.ts): three turns with theTool: bashtext "Use rg.", then "Use fd.", then "Use fd." again.replayVerifySessionmust rebuild each recorded request throughreadToolInstructionsRecordandbuildReplayRequest. Turn 2 must carry the<tool-instructions tool="bash">row while its tool descriptions keep "Use rg.", so the test cannot pass vacuously.replayFixture.ts): before this PR the fixture built every recorded request through the replay path itself (replay compared with itself), on Sonnet 4.5, which never sends rows. The newliveToolInstructionsoption records the request with the liveassemblePromptPayloadon Opus 5.5 over the direct route, and writes the records on the assistant row.turnContextAssembler.test.ts): a tool description rewritten aftergetToolsForModelsends as before, with no record and no row. A corrupt frozen record keeps the live description and does not crash.Validation
Each test fails when the path it covers is broken:
toolInstructionsread inreplayVerify.tsremoved, and with the pass-through inreplayRequestBuilder.tsremoved.endsWithguard inlocateToolInstructionsdisabled.frozen == nullguard removed.Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:high• Cost:$593.65