-
Notifications
You must be signed in to change notification settings - Fork 134
feat(workspace): tell the model what the bound workspace serves #1182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
158bf95
feat(workspace): tell the model what the bound workspace serves
suryaiyer95 8aec1f3
refactor(workspace): simplify the awareness section after review
suryaiyer95 5028607
refactor(workspace): route the human-facing surfaces through the same…
suryaiyer95 531fffc
chore(workspace): restack onto the current precedence head — two more…
86994ed
fix(workspace): make the awareness section true for partial coverage,…
a21f61e
refactor(workspace): project the served inventory once for the wareho…
4fb41bf
fix(workspace): make the workspace name inert where it enters the sna…
4f6ae42
fix(workspace): strip C1 controls and line separators from the worksp…
b4b95fa
fix(workspace): make the awareness section's silence claim true, and …
suryaiyer95 9c74f4d
test(workspace): assert which branch settles the empty-catalog case
suryaiyer95 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,209 @@ | ||
| // altimate_change start — workspace tool awareness. | ||
| // | ||
| // The model-facing half of workspace precedence. `precedence.ts` decides which calls | ||
| // are routed to the bound workspace's engine and REFUSES the ones that are; this | ||
| // module tells the model that up front, so it calls the engine tool first instead of | ||
| // learning the rule by being refused. | ||
| // | ||
| // Why a system-prompt section and not a richer tool description: `session/system.ts` | ||
| // records, from this repo's own benchmark trace analysis, that a lazily-described | ||
| // capability fired in "<1% of tool calls", and that guidance placed at the END of a | ||
| // section was "treated as background reference rather than binding directive" while | ||
| // the same content placed FIRST was applied. The precedence suffix appended by | ||
| // `describeNativeTool` is exactly that shape — trailing, non-imperative, and it never | ||
| // names the engine key — which is why it did not change behaviour. | ||
| // | ||
| // PURELY ADDITIVE BY CONSTRUCTION. This module renders a string and nothing else. It | ||
| // has no effect on which calls are shadowed, on what a shadowed call returns, or on | ||
| // any tool body. | ||
| // | ||
| // Its safety property is scoped, and worth stating exactly rather than generously: a | ||
| // project this session knows is NOT linked to a workspace assembles a byte-identical | ||
| // system prompt to before this shipped. `pilot-off`, `unbound` and | ||
| // `nothing-materialised` all render "", and `derive` settles the link read before it | ||
| // reads the escape hatch, so once that read says `unbound` no reason that speaks is | ||
| // still reachable. (`binding-unreadable` is the read failing, not saying no — the | ||
| // project may or may not be linked, and the copy for it claims neither.) The | ||
| // module is NOT silent for every disabled state: the hatch and the three uncertain | ||
| // states (`binding-unreadable`, `unattributed`, `derive-failed`) each render a short | ||
| // paragraph steering to the local tools, because in all four the engine's tools can | ||
| // still be in the catalog while routing refuses them, and silence would leave the | ||
| // model free to call what it can see. `DISABLED_COPY` below is the decision table. | ||
| // | ||
| // SERVER-SIDE ONLY, for the same reason `precedence.ts` is: the TUI plugin runtime | ||
| // loads plugins in a separate module realm, so an import from there would read a | ||
| // different, always-empty `Precedence` map. Import this only from the session layer. | ||
| import { type Capability, type Precedence, inertWorkspaceName, servedInventory } from "./precedence" | ||
|
|
||
| /** Hard ceiling on the rendered section. Deliberately independent of | ||
| * `UNIFIED_INJECTION_BUDGET`: this is a routing directive, not knowledge, and must | ||
| * never compete with memory for space. Four integrations x three capabilities lands | ||
| * far under this; the cap exists so a future engine advertising many integrations | ||
| * degrades predictably instead of crowding the prompt. */ | ||
| export const MAX_SECTION_CHARS = 2_000 | ||
|
|
||
| const HEADING = "## Workspace integrations" | ||
|
|
||
| /** How each capability is named to the model. Keyed on the `Capability` union, so a | ||
| * new capability is a compile error here rather than an unlabelled row. */ | ||
| const CAPABILITY_LABEL: Record<Capability, string> = { | ||
| sql_execute: "execute", | ||
| sql_explain: "explain plan", | ||
| schema_inspect: "table stats / schema inspection", | ||
| } | ||
|
|
||
| /** The `Capability` union IS the native tool id — `describeNativeTool` relies on the | ||
| * same identity (`precedence.ts`, `(CAPABILITIES as string[]).includes(toolID)`), so | ||
| * there is no separate mapping to keep in step. */ | ||
| const localToolOf = (c: Capability) => `\`${c}\`` | ||
|
|
||
| /** Derived, never hand-written: `CAPABILITY_LABEL` is exhaustive over `Capability`, | ||
| * so a new capability updates this list by construction. A literal here would go | ||
| * stale silently and tell the model an incomplete set of local tools — the exact | ||
| * over-steering the converse paragraph exists to prevent. */ | ||
| const ALL_LOCAL_TOOLS = (Object.keys(CAPABILITY_LABEL) as Capability[]).map(localToolOf).join(", ") | ||
|
|
||
| /** Said when the escape hatch is on. Engine tools can still materialise in that | ||
| * session — `derive` refuses before it looks at them, but the MCP client connects the | ||
| * configured entry regardless — so silence here would leave the model free to reach | ||
| * for tools it can see and should not use. Never reached on a project known to have no | ||
| * link: the flag is process-wide, so `derive` reads it after the link rather than | ||
| * before, and a project that reads as unbound settles as `unbound` and says nothing. */ | ||
| const ESCAPE_HATCH_SECTION = [ | ||
| HEADING, | ||
| "", | ||
| "Workspace routing is disabled for this session (`--integrations=local`). Use the local " + | ||
| `warehouse tools (${ALL_LOCAL_TOOLS}) for every connection, even if \`datamate_*\` tools ` + | ||
| "are present in this catalog.", | ||
| ].join("\n") | ||
|
|
||
| /** Said when routing is off because it could not be established — the link unreadable, | ||
| * the engine not attributable to the bound workspace, or the derivation failed. | ||
| * `check()` fails open in those states and the engine's tools may still be in the | ||
| * catalog (under `unattributed` they may belong to a DIFFERENT workspace, which is why | ||
| * routing refused them), so the model is steered to the local tools the same way the | ||
| * hatch does. The workspace is not named: nothing here has verified it. Nor is one | ||
| * asserted to exist — `binding-unreadable` is reached whenever the link read throws, | ||
| * which a project with no link can do, so this copy claims only what is true in all | ||
| * three states. */ | ||
| const UNVERIFIED_SECTION = [ | ||
| HEADING, | ||
| "", | ||
| "Workspace routing could not be established for this session. Use the local warehouse tools " + | ||
| `(${ALL_LOCAL_TOOLS}) for every connection, even if \`datamate_*\` tools are present in this ` + | ||
| "catalog.", | ||
| ].join("\n") | ||
|
|
||
| /** What a non-routing session is told, keyed on the union so a new `disabledReason` | ||
| * is a compile error here rather than silently rendering nothing. Silence is reserved | ||
| * for the states where there is nothing the model could misuse: the pilot off, no | ||
| * binding, or no engine tools materialised. Those keep the system prompt byte-identical | ||
| * to before this module existed. No reason that speaks survives a link read that | ||
| * settled as `unbound` — that is the property the silence claim above rests on, and | ||
| * the reason the hatch is read after the link rather than before it. */ | ||
| const DISABLED_COPY: Record<NonNullable<Precedence["disabledReason"]>, string> = { | ||
| "pilot-off": "", | ||
| "escape-hatch": ESCAPE_HATCH_SECTION, | ||
| unbound: "", | ||
| "binding-unreadable": UNVERIFIED_SECTION, | ||
| unattributed: UNVERIFIED_SECTION, | ||
| "derive-failed": UNVERIFIED_SECTION, | ||
| "nothing-materialised": "", | ||
| } | ||
|
|
||
| /** | ||
| * Render the section, or "" when there is nothing to steer. | ||
| * | ||
| * Pure projection of the snapshot `Precedence.refresh` stored for this turn — the same | ||
| * object the tool descriptions were built from and that `check()` will read mid-turn. | ||
| * One snapshot, one truth: the section cannot advertise a routing the guard would not | ||
| * perform. (The exposed tool list is pinned to the turn's first catalog while this | ||
| * snapshot is refreshed per step, so on a later step the two can name different | ||
| * engine keys if another session replaced the engine mid-turn — the lease work that | ||
| * pins the raw tool map closes that, not this module.) | ||
| * | ||
| * Called once per STEP, not per turn: the prompt loop reassembles the system array on | ||
| * every generation, so a 40-tool-call turn renders this 40 times. Kept cheap and | ||
| * allocation-light for that reason, and deliberately not memoised — the snapshot is | ||
| * refreshed per step, ahead of this render, and a cached section outliving its | ||
| * snapshot would advertise routing that no longer holds. | ||
| */ | ||
| export function systemSection(precedence: Precedence | undefined): string { | ||
| if (!precedence) return "" | ||
| if (!precedence.enabled) return precedence.disabledReason ? DISABLED_COPY[precedence.disabledReason] : "" | ||
|
|
||
| const served = servedInventory(precedence) | ||
| if (served.length === 0) return "" | ||
|
|
||
| // `type` is the canonical local driver type (`postgres`), not the user-facing | ||
| // connection name nor the engine's integration id (`postgresql`) — it is what the | ||
| // local connection registry carries, so it is what the model must match against. | ||
| const typeLines = served.map(({ type, served: rows, local }) => { | ||
| const servedPart = rows.map((r) => `${CAPABILITY_LABEL[r.capability]}: \`${r.modelKey}\``).join("; ") | ||
| const localPart = local.length | ||
| ? ` (${local.map((c) => CAPABILITY_LABEL[c]).join(" and ")} for ${type} stay on the local ` + | ||
| `${local.map(localToolOf).join(" / ")})` | ||
| : "" | ||
| return `- ${type} — ${servedPart}${localPart}` | ||
| }) | ||
|
|
||
| return assemble(precedence.workspaceName, precedence.workspaceId, typeLines) | ||
| } | ||
|
|
||
| /** The workspace name is customer-authored and lands in the system prompt — the | ||
| * highest-trust surface there is. The snapshot already carries it inert (one line, | ||
| * no control characters, bounded — `inertWorkspaceName`); here it is JSON-quoted as | ||
| * well, so quotes cannot break out of the sentence, and the numeric id, when known, | ||
| * is named alongside as the stable identifier. Re-applying the sanitiser costs | ||
| * nothing and keeps this surface safe even for a snapshot built elsewhere. */ | ||
| function workspaceLabel(name: string, id: string | undefined): string { | ||
| const bounded = inertWorkspaceName(name) | ||
| return id ? `${JSON.stringify(bounded)} (id ${id})` : JSON.stringify(bounded) | ||
| } | ||
|
|
||
| /** Build the section from its type lines, enforcing the char cap by dropping trailing | ||
| * types rather than truncating mid-sentence — down to none if a single line is | ||
| * oversized, so the ceiling is a real one. The converse paragraph is never dropped: | ||
| * without it the section reads as "prefer the workspace for everything", which is the | ||
| * over-steering failure this design most needs to avoid. It changes shape when types | ||
| * were omitted, though: the omitted types ARE served, so forbidding `datamate_*` for | ||
| * "types not listed" would contradict the omission line — the partial list is said to | ||
| * be partial instead, and the prohibition is kept only for types the workspace does | ||
| * not serve. The count is stated once, on the list where it belongs; the converse | ||
| * carries only what the model should DO about the omission. */ | ||
| function assemble(workspaceName: string, workspaceId: string | undefined, typeLines: string[]): string { | ||
| const label = workspaceLabel(workspaceName, workspaceId) | ||
| const render = (lines: string[]) => { | ||
| const omitted = typeLines.length - lines.length | ||
| const converse = | ||
| omitted > 0 | ||
| ? "For the served types omitted above, prefer the `datamate_*` tool for that type when one is in the " + | ||
| `catalog. Connection types this workspace does not serve use the local tools (${ALL_LOCAL_TOOLS}).` | ||
| : `Every other connection type uses the local tools (${ALL_LOCAL_TOOLS}). Do not use ` + | ||
| "`datamate_*` warehouse tools for connection types that are not listed above." | ||
| return [ | ||
| HEADING, | ||
| "", | ||
| `This project is bound to Altimate workspace ${label}. For each connection type below, the ` + | ||
| "local tool for a capability that names a workspace tool will NOT execute — it returns a " + | ||
| "redirect. Call the named workspace tool directly; capabilities not named for a type stay on " + | ||
| "the local tools:", | ||
| "", | ||
| ...lines, | ||
| ...(omitted > 0 | ||
| ? [`- …and ${omitted} further connection type${omitted === 1 ? "" : "s"} served by this workspace.`] | ||
|
ralphstodomingo marked this conversation as resolved.
|
||
| : []), | ||
| "", | ||
| converse, | ||
| ].join("\n") | ||
| } | ||
|
|
||
| let lines = typeLines | ||
| let out = render(lines) | ||
| while (out.length > MAX_SECTION_CHARS && lines.length > 0) { | ||
| lines = lines.slice(0, -1) | ||
| out = render(lines) | ||
| } | ||
| return out | ||
| } | ||
| // altimate_change end | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.