Skip to content

🤖 fix: keep the cached tool block when the skill list changes mid-session - #7308

Merged
ThomasK33 merged 4 commits into
tk/f10-custom-instructions-ledgerfrom
tk/f10-skill-index-ledger
Oct 23, 2026
Merged

ThomasK33 merged 4 commits into
tk/f10-custom-instructions-ledgerfrom
tk/f10-skill-index-ledger

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

Summary

A skill added to or removed from a workspace mid-session no longer rewrites the cached Anthropic tool block. Inside a context segment (between compactions), the "Available skills" list in the agent_skill_read tool description stays at the list the segment first sent, and Xum reads it from the ledger (chat.jsonl). When the live list differs, Xum sends the new list as one system row after the newest user message, so the model knows about the new skill on that turn. Every later rebuild puts the row back in place. The next compaction folds the current list into the description. Stacked on #7304, which does the same for AGENTS.md.

Background

Measured on one host (10-22 08:12 to 10-23 01:28 UTC, 1,548 Anthropic turns): 10 cache.prefix_break events on agent_skill_read in the tools section, 0.64 rewritten histories per 100 turns. The description grew from 11,129 to 11,392 chars after skill commits merged on main, when agents merged main into their worktree. The tools section is the first part of the cached prefix, so each change rewrote the whole history.

Implementation

  • New MuxMetadata.skillIndex = { text, placement: "system" | "row" }, the same shape, schema and rule as customInstructions and volatileSystemTail. A corrupt value loads as no record. Xum writes it only at a segment start and on a change.
  • skillIndexSectionFor (agent_skill_read.ts) reports the description prefix and the skill list, and the assembler freezes the list only while the description still starts with that prefix. It does not search the description for markers.
  • A segment that starts with no skills listed records an empty baseline, so a first skill added later is a row.
  • If no skill is listed any more, the description keeps the frozen list and a row says the list is now empty.
  • Freezing the description is safe for new skills: the row names them, and agent_skill_read loads any skill by name.
  • Same gating as the other system-section rows (Anthropic and Coder routes, models that bind thinking to the prefix).

Validation

  • Real-request prefix test: a skill added between turns keeps the tools section byte-identical on turns 1 to 3, turns 2 and 3 carry the same row, and the compaction turn has the new list in the description. A removed last skill and a first skill added mid-segment behave the same way. Each test fails with its fix reverted.

Risks

Medium-low. It changes only how the agent_skill_read description is built for Anthropic turns.

  • Until the next compaction, the model sees the old list in the tool description and the new list in a row.
  • Old histories (no record) behave as before. An older build ignores the field.
  • Tool: <name> sections from AGENTS.md also go into tool descriptions and are still not frozen (follow-up).

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high • Cost: $585.33

…sion

Inside a context segment, the "Available skills" index of the agent_skill_read
tool description now stays at the list the segment first sent, read from the
ledger (MuxMetadata.skillIndex). A changed list goes in one system row after
the newest user message, and every rebuild puts the row back in place, so the
cached tool block stays byte-identical. A new skill still reaches the model: the
row names it, and agent_skill_read loads any skill by name. The next
compaction folds the current list into the description. Same mechanism and
gating as the custom-instructions and volatile-tail records.

---
_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
@ThomasK33
ThomasK33 added this pull request to stack #7309 October 23, 2026 06:35
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T07:26:52.086597Z cd90375 New commits
🔒 Security Review ✅ Completed 2026-10-23T07:27:22.731156Z cd90375 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e stories (Haiku): 14 passed

27 agent steps · 7 replayed from cache · 32 model calls · 236.9k tokens (76% cached)
Spend: $0.09 of the $1.00 cap (proxy ledger)

All 14 tests in 9 files
Test Agent Time
🟢 chatRestart.e2e.ts · 1 passed 2 steps · 4 calls 23.8s
🟢 a chat survives a restart of the server 2 steps · 4 calls 23.8s
🟢 compaction.e2e.ts · 1 passed 3 steps · 7 calls 43.2s
🟢 /compact replaces the old chat with a summary 3 steps · 7 calls 43.2s
🟢 pluginTrustRestart.e2e.ts · 1 passed 4 steps · 2 calls 12.5s
🟢 a trusted plugin hook stays trusted after the server restarts 4 steps · 2 calls 12.5s
🟢 settingsRoundTrip.e2e.ts · 1 passed 3 steps · 4 calls 29.7s
🟢 a chat survives trips through Settings 3 steps · 4 calls 29.7s
🟢 sidebarPerWorkspace.e2e.ts · 1 passed 4 steps · 1 call 3.9s
🟢 a new workspace starts with its right sidebar collapsed while another keeps it open 4 steps · 1 call 3.9s
🟢 smoke.e2e.ts · 1 passed 2 steps · 1 call 4.0s
🟢 the agent opens the playground workspace 2 steps · 1 call 4.0s
🟢 streamErrors.e2e.ts · 1 passed 1 step · 1 call 9.9s
🟢 stream errors show in the chat, and the chat recovers 1 step · 1 call 9.9s
🟢 todoNudge.e2e.ts · 1 passed 2 steps · 6 calls 29.6s
🟢 the agent finishes open TODO items after the turn-end nudge 2 steps · 6 calls 29.6s
🟢 touchAlignment.e2e.ts · 6 passed 6 steps · 6 calls 29.1s
🟢 the Settings close button is level with the title on a touch screen 390px wide 1 step · 1 call 7.6s
🟢 the Settings close button is level with the title on a touch screen 768px wide 1 step · 1 call 5.0s
🟢 Backup settings checkboxes are small and level with their labels on a touch phone 1 step · 1 call 4.8s
🟢 the heartbeat checkbox is small and level with its label on a touch phone 1 step · 1 call 3.2s
🟢 a reply's model and time stay on one line on a touch phone 1 step · 1 call 5.3s
🟢 the workspace actions menu stays on the title row on a touch phone 1 step · 1 call 3.3s

e2e 0.20.0 · 3m 9s · web · run artifacts

🟢 e2e Electron stories (Haiku): 3 passed

3 agent steps · 3 model calls · 8.1k tokens (32% cached)
Spend: $0.01 of the $1.00 cap (proxy ledger)

All 3 tests in 3 files
Test Agent Time
🟢 electron/deepLink.e2e.ts · 1 passed 1 step · 1 call 6.5s
🟢 a deep link of a second launch starts a chat in the running app 1 step · 1 call 6.5s
🟢 electron/menuBar.e2e.ts · 1 passed 1 step · 1 call 6.3s
🟢 the workspace menu bar is the desktop title bar 1 step · 1 call 6.3s
🟢 electron/smoke.e2e.ts · 1 passed 1 step · 1 call 4.7s
🟢 Xum Electron opens the playground workspace 1 step · 1 call 4.7s

e2e 0.20.0 · 19.6s · electron · run artifacts

🟢 e2e agent repros (Haiku): no tests selected

Spend: $0.00 of the $1.00 cap (proxy ledger)

e2e 0.20.0 · 5.3s · 2 targets (web, phone) · run artifacts

# Conflicts:
#	src/node/services/turnContextAssembler.test.ts
#	src/node/utils/messages/volatileSystemTail.ts
…ewrite

The merge of d9b2461 moved the empty-baseline fix into
planRemovableSection, so the skill index gets it too: a segment that
starts with no skill listed records "" and a first skill becomes a
replacement row. This test covers that path and fails with the
baseline record removed.

---
_Generated with `xum` • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
<!-- mux-attribution: model=anthropic:claude-opus-5-5 thinking=high -->
@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 5afa16c Oct 23, 2026
75 of 99 checks passed
@ThomasK33
ThomasK33 deleted the tk/f10-skill-index-ledger branch October 23, 2026 09:16
@ThomasK33

Copy link
Copy Markdown
Member Author

Post-merge perf note (perf owner), covering #7304 (91bb729f88) and #7308 (5afa16c745). Neither was sent to me before merge. No action is needed.

What changed on the turn path: each turn now plans three system sections (volatile tail, custom instructions, skill index) instead of one. Each plan slices the segment after the latest context boundary and, once a segment has a frozen record, rebuilds the message array with one flatMap over the history. So the per-turn planning cost is now about 3x, and still linear in the history length. No new I/O.

Measured on main 5afa16c745 (bun, synthetic segment with a frozen record on all three channels, last row a user row; host load ~33, so absolute values are high, but the ratios hold):

Rows in history One section (before) Three sections (now)
1,000 0.07-0.10 ms 0.11-0.14 ms
10,000 ~0.19 ms ~0.56 ms
100,000 2.2-5.9 ms 7.4-8.3 ms

This is far below the send-path trigger I track on #6912 (>= 50 ms per turn). If a fourth section joins this pattern, or segments of 100k+ rows become common, the three plans can share one segment read and one flatMap. That is a follow-up, not a fix needed now.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant