Skip to content

🤖 fix: freeze AGENTS.md Tool: sections per segment, re-freeze instructions on a model or agent switch - #7323

Merged
ThomasK33 merged 4 commits into
mainfrom
tk/f10-ledger-followups
Oct 23, 2026
Merged

ThomasK33 merged 4 commits into
mainfrom
tk/f10-ledger-followups

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

Summary

An edit to an AGENTS.md Tool: <name> section no longer rewrites the cached tool block. These sections now follow the same per-segment ledger as the custom instructions (#7304) and the skill index (#7308). A new toolInstructions record freezes them at the segment start, and the request puts the frozen text back at the end of each tool description. A change goes in a system row as <tool-instructions tool="..."> blocks until the next compaction folds it in. Separately, a model or agent switch inside a segment now re-freezes the <model-...>/<mode-...> instruction blocks instead of stacking a row on the old ones.

Background

Implementation

  • volatileSystemTail.ts: a third ledger channel, toolInstructions. Its record text is the sorted JSON map of each tool in the request to its instructions ("" for none). A new optional render on the channel turns that JSON into tagged blocks for the row.
  • turnContextAssembler.ts (planToolInstructionsTools): swaps the live instructions at the end of each description for the frozen ones. If any description no longer ends with "\n\n" + instructions, the request is sent as before (the same fallback as the skill index).
  • switchesModelOrAgent compares the segment's last assistant row (canonical model, agentId) with the request. Only the instruction channel re-freezes. Replay re-freezes when the turn recorded a system record, which after a freeze can only come from a re-freeze.
  • Plumbing: MuxMetadata.toolInstructions, its schema, SYSTEM_SECTION_RECORD_FIELDS, workspaceViewTail, replayVerify and replayRequestBuilder.

Validation

  • aiService.test.ts runs the real getToolsForModel path: with a Tool: section edited between turns, the tool block is byte-identical. turnContextAssembler.test.ts covers a changed section, a first section on another tool, replay, a repeated turn and removal. Each fails with the freeze, the description swap or the turnRequestBuilder plumbing reverted.
  • The switch test covers a model switch, an agent switch, replay, and a same-model change (still a row). It fails with the switch check off and with the replay rule off.
  • The other advisor item (removing the blocks after an assistant-last resume records the marker with placement system) costs no cache rewrite: the request sends "" for the marker, and the next turn reads it as applied. A scratch test showed a byte-identical system message and no rows, so this PR leaves that path unchanged.

Risks

Medium-low. It changes Anthropic tool descriptions only while a Tool: section changed inside a segment.

  • If a later wrapper edits a tool description, the freeze turns off for that request (old behavior).
  • Two agents with identical <agent-instructions>: switching between them now re-freezes, which costs one rewrite that a row would have avoided. I expect this to be rare and did not measure it.
  • The skill-index, tool-instruction and volatile-tail channels do not re-freeze on a switch. That is possible at no cost but out of scope.
  • Since the first draft: a switch also re-freezes after an earlier row (Codex round 1), and a route switch for the same model counts as a switch (round 2).
  • Follow-ups (Merge Advisor, tracked as F31): no replay test through replayVerify/replayRequestBuilder for a Tool: row; untested fallbacks for a rewritten description and a corrupt record.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high • Cost: $589.85

A model or agent switch inside a segment kept the first model's
<model-...>/<mode-...> sections in the frozen system prompt and sent the
new ones in a row under them. Each model has its own prompt cache, and an
agent switch changes <agent-instructions> before the blocks, so the
switch already rewrites the cache: the blocks now re-freeze at the new
sections. Replay re-freezes where the turn recorded a "system" record.

---
_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
…nges

getToolsForModel appends each `Tool: <name>` section to its tool's
description, inside the cached tool block, so an edit rewrote the whole
cached prefix. The sections now follow the skill-index ledger: a new
toolInstructions record (the JSON map of tool to instructions) freezes
them at the segment start, the request puts the frozen text back at the
end of each description, and a change goes in a system row of tagged
blocks until the next compaction. A description that no longer ends with
its instructions sends as before.

Addresses Codex PRRT_kwDOPxxmWM6vgBAW.

---
_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T10:01:56.356920Z 2c79774 Manual request
🔒 Security Review ✅ Completed 2026-10-23T10:02:29.420509Z 2c79774 Manual request

Security findings

Advisory findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c027079b3e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/utils/messages/volatileSystemTail.ts Outdated
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e stories (Haiku): 14 passed

27 agent steps · 7 replayed from cache · 31 model calls · 218.8k tokens (70% cached)
Spend: $0.10 of the $1.00 cap (proxy ledger)

All 14 tests in 9 files
Test Agent Time
🟢 chatRestart.e2e.ts · 1 passed 2 steps · 4 calls 24.2s
🟢 a chat survives a restart of the server 2 steps · 4 calls 24.2s
🟢 compaction.e2e.ts · 1 passed 3 steps · 7 calls 44.8s
🟢 /compact replaces the old chat with a summary 3 steps · 7 calls 44.8s
🟢 pluginTrustRestart.e2e.ts · 1 passed 4 steps · 2 calls 9.9s
🟢 a trusted plugin hook stays trusted after the server restarts 4 steps · 2 calls 9.9s
🟢 settingsRoundTrip.e2e.ts · 1 passed 3 steps · 5 calls 29.3s
🟢 a chat survives trips through Settings 3 steps · 5 calls 29.3s
🟢 sidebarPerWorkspace.e2e.ts · 1 passed 4 steps · 1 call 4.8s
🟢 a new workspace starts with its right sidebar collapsed while another keeps it open 4 steps · 1 call 4.8s
🟢 smoke.e2e.ts · 1 passed 2 steps · 1 call 2.7s
🟢 the agent opens the playground workspace 2 steps · 1 call 2.7s
🟢 streamErrors.e2e.ts · 1 passed 1 step · 1 call 7.5s
🟢 stream errors show in the chat, and the chat recovers 1 step · 1 call 7.5s
🟢 todoNudge.e2e.ts · 1 passed 2 steps · 4 calls 23.5s
🟢 the agent finishes open TODO items after the turn-end nudge 2 steps · 4 calls 23.5s
🟢 touchAlignment.e2e.ts · 6 passed 6 steps · 6 calls 29.8s
🟢 the Settings close button is level with the title on a touch screen 390px wide 1 step · 1 call 4.0s
🟢 the Settings close button is level with the title on a touch screen 768px wide 1 step · 1 call 4.3s
🟢 Backup settings checkboxes are small and level with their labels on a touch phone 1 step · 1 call 4.7s
🟢 the heartbeat checkbox is small and level with its label on a touch phone 1 step · 1 call 10.5s
🟢 a reply's model and time stay on one line on a touch phone 1 step · 1 call 3.2s
🟢 the workspace actions menu stays on the title row on a touch phone 1 step · 1 call 3.1s

e2e 0.20.0 · 3m 0s · web · run artifacts

🟢 e2e Electron stories (Haiku): 3 passed

3 agent steps · 3 model calls · 8.0k tokens (21% cached)
Spend: $0.02 of the $1.00 cap (proxy ledger)

All 3 tests in 3 files
Test Agent Time
🟢 electron/deepLink.e2e.ts · 1 passed 1 step · 1 call 5.3s
🟢 a deep link of a second launch starts a chat in the running app 1 step · 1 call 5.3s
🟢 electron/menuBar.e2e.ts · 1 passed 1 step · 1 call 6.3s
🟢 the workspace menu bar is the desktop title bar 1 step · 1 call 6.3s
🟢 electron/smoke.e2e.ts · 1 passed 1 step · 1 call 4.7s
🟢 Xum Electron opens the playground workspace 1 step · 1 call 4.7s

e2e 0.20.0 · 18.4s · electron · run artifacts

🟢 e2e agent repros (Haiku): no tests selected

Spend: $0.00 of the $1.00 cap (proxy ledger)

e2e 0.20.0 · 5.3s · 2 targets (web, phone) · run artifacts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: c027079b3e

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread src/node/services/turnContextAssembler.ts
planSystemSection returned on "no change" before the re-freeze check. So
after an AGENTS.md change sent as a row, a model or agent switch with no
new edit kept the stale frozen blocks plus the old row. The re-freeze now
runs first and folds the live blocks and the earlier rows into the system
prompt. A switch with nothing to fold still changes and records nothing.
Replay re-freezes from the turn's "system" record, so it matches.

Addresses Codex PRRT_kwDOPxxmWM6vjONA.

---
_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: da6b5146b0

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da6b5146b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/turnContextAssembler.ts Outdated
…locks

switchesModelOrAgent compared canonical models, so anthropic: -> coder:
for the same Claude model was no switch. The <model-...> section is
tagged with the raw model string, so its blocks changed and went in a
row. A changed routeProvider (both rows set) now counts as a switch too;
old rows without the field force no re-freeze.

Addresses Codex PRRT_kwDOPxxmWM6vjdlK.

---
_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 2c797742b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 2c797742b3

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 42c551e Oct 23, 2026
37 of 38 checks passed
@ThomasK33
ThomasK33 deleted the tk/f10-ledger-followups branch October 23, 2026 10:20
azaj01 pushed a commit to azaj01/mux that referenced this pull request Oct 25, 2026
…coder#7330)

## Summary

Tests for three gaps in the `Tool:` section freeze from coder#7323, as its
Merge Advisor asked. No production code changes. The tests found no bug.

## Implementation

- **Replay test** (`replayVerify.test.ts`): three turns with the `Tool:
bash` text "Use rg.", then "Use fd.", then "Use fd." again.
`replayVerifySession` must rebuild each recorded request through
`readToolInstructionsRecord` and `buildReplayRequest`. Turn 2 must carry
the `<tool-instructions tool="bash">` row while its tool descriptions
keep "Use rg.", so the test cannot pass vacuously.
- **Fixture mode** (`replayFixture.ts`): before this PR the fixture
built every recorded request through the replay path itself (replay
compared with itself), on Sonnet 4.5, which never sends rows. The new
`liveToolInstructions` option records the request with the live
`assemblePromptPayload` on Opus 5.5 over the direct route, and writes
the records on the assistant row.
- **Fallbacks** (`turnContextAssembler.test.ts`): a tool description
rewritten after `getToolsForModel` sends as before, with no record and
no row. A corrupt frozen record keeps the live description and does not
crash.

## Validation

Each test fails when the path it covers is broken:
- The replay test fails with the `toolInstructions` read in
`replayVerify.ts` removed, and with the pass-through in
`replayRequestBuilder.ts` removed.
- The rewritten-description case fails with the `endsWith` guard in
`locateToolInstructions` disabled.
- The corrupt-record case crashes with the `frozen == null` guard
removed.

---

_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking:
`high` • Cost: `$593.65`_

<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high
costs=593.65 -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant