Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,13 +137,15 @@ For CI, set `PIFY_TRUST_PROJECT=1`. It is an environment variable rather than a

## A child can ask instead of guessing

A scoped child that hits a decision it should not be making — an unstated product, API or scope choice, or missing access — calls `ask_supervisor`, and the question reaches **you** through the usual dialog. Only when there is a UI to ask through: a headless run has nobody to answer, so the tool is deliberately not offered there — measured in `test/live/ask-wire.mjs` (the tool absent headless, `agent_run` still present). The offered-when-interactive half needs a TUI, which a scripted `-p` probe cannot see into; it is exercised by using it, not by a test that fakes it.
A scoped child that hits a decision it should not be making — an unstated product, API or scope choice, or a choice of an authorised file or resource — calls `ask_supervisor`, and the question reaches **you** through the usual dialog. The question and context appear above the answer field, with a reminder that replying does not add tools or change access permissions. Only when there is a UI to ask through: a headless run has nobody to answer, so the tool is deliberately not offered there — measured in `test/live/ask-wire.mjs` (the tool absent headless, `agent_run` still present). The offered-when-interactive half needs a TUI, which a scripted `-p` probe cannot see into; it is exercised by using it, not by a test that fakes it.

It reaches you rather than the parent agent for a structural reason: the parent is blocked inside the tool call that spawned the child, so it could not answer anyway. And the decision is yours regardless.
Missing tools, unavailable credentials and denied operations are execution blockers, not decisions for this dialog. The child is instructed to finish what the brief authorises and report the resource, required tool, exact error (redacting secrets) and remaining work to the parent, without retrying or bypassing the denial. This is model guidance, not a new permission mechanism or a guarantee that a model will never ask unnecessarily.

A genuine decision reaches you rather than the parent agent for a structural reason: the parent is blocked inside the tool call that spawned the child, so it could not answer anyway. And the decision is yours regardless.

A declined question is not permission to guess. The child is told to finish what the brief authorises and report what it could not decide. Three questions per run, and there is deliberately no progress channel — a child's progress belongs in its report, not in an interruption.

Measured across qwen3-235b, gpt-5.5 and claude-sonnet-4.5: six runs out of six asked rather than inventing an answer.
The original decision-question framing was measured across qwen3-235b, gpt-5.5 and claude-sonnet-4.5: six runs out of six asked rather than inventing an answer. That measurement does not cover the revised access-blocker guidance.

## `@agent` at the prompt

Expand Down
5 changes: 2 additions & 3 deletions extensions/subagent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ import {
ASK_REASONS,
ASK_TOOL_DESCRIPTION,
ASK_TOOL_NAME,
askBody,
askTitle,
formatAnswer,
type AskReason,
Expand Down Expand Up @@ -291,8 +290,8 @@ export default function subagent(pi: ExtensionAPI) {
questionsLeft--;
const answer = await withUiLock(() =>
ctx.ui.input(
askTitle(def.name, params.reason),
askBody(params.question, params.context).slice(0, 500),
askTitle(def.name, params.reason, params.question, params.context),
"Your answer",
),
);
return {
Expand Down
15 changes: 12 additions & 3 deletions src/ask.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,34 @@ export type AskReason = (typeof ASK_REASONS)[number];

export const ASK_TOOL_NAME = "ask_supervisor";

export const ACCESS_BOUNDARY = [
"If a required tool or credential is unavailable, or an operation is denied, do not ask the user to grant access.",
"Complete what the brief authorises and report the resource, required tool, exact error (redacting secrets), and remaining work to the caller.",
"Do not retry or bypass an access denial.",
"A human reply does not add tools or change access permissions.",
].join(" ");

export const ASK_TOOL_DESCRIPTION = [
"Ask the person supervising this delegated task, and wait for their answer.",
"Use it when continuing would mean inventing a decision that is theirs:",
"need_decision (a product, API, or scope choice with no defensible default),",
"clarify_scope (the brief admits two readings that lead to different work),",
"missing_access (a file, credential, or permission the task needs is not available).",
"missing_access (a human choice of an authorised file or resource).",
ACCESS_BOUNDARY,
"Do not use it to report progress, to confirm something you can verify yourself,",
"or to ask permission for work the brief already authorised. Each question costs",
"the user an interruption, so ask the smallest question that unblocks you.",
].join(" ");

/** How the question is shown, and what the child gets back. */
export function askTitle(agent: string, reason: AskReason): string {
export function askTitle(agent: string, reason: AskReason, question: string, context?: string): string {
const label: Record<AskReason, string> = {
need_decision: "needs a decision",
clarify_scope: "needs the scope clarified",
missing_access: "is missing access",
};
return `Subagent ${agent} ${label[reason]}`;
// ui.input's placeholder is not a message; keep the question in its visible title.
return `Subagent ${agent} ${label[reason]}\n\n${askBody(question, context)}\n\nReplying does not add tools or change access permissions.`;
}

export function askBody(question: string, context: string | undefined): string {
Expand Down
7 changes: 5 additions & 2 deletions src/prompts.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { AgentDef, RunState } from "./types.ts";
import { ACCESS_BOUNDARY } from "./ask.ts";
import { outcomeLine } from "./outcome.ts";
import { UNTRUSTED_REPORT_NOTE, neutralizeReport } from "./pending.ts";

Expand Down Expand Up @@ -50,6 +51,7 @@ const CHILD_BASE = [
"end the report with a line reading exactly `OUTCOME: blocked` (a decision, access or",
"information you do not have) or `OUTCOME: failed` (you tried and it does not work),",
"so the caller does not have to infer it from your prose. Say nothing if it went fine.",
ACCESS_BOUNDARY,
].join(" ");

/**
Expand All @@ -58,7 +60,8 @@ const CHILD_BASE = [
* ask_supervisor tool is unreachable in practice: the base framing forbids
* asking, and a model follows the prompt over a tool description.
*
* Measured across qwen3-235b, gpt-5.5 and claude-sonnet-4.5 on two framings
* The original decision exception was measured across qwen3-235b, gpt-5.5
* and claude-sonnet-4.5 on two framings
* of an underspecified task: with the tool registered and this line present,
* six runs out of six asked instead of inventing the answer.
*/
Expand All @@ -67,7 +70,7 @@ export function childFraming(canAsk: boolean): string {
return [
CHILD_BASE,
"The one exception: if continuing would mean inventing a decision that belongs to the",
"supervisor — an unstated product/API/scope choice, or missing access — call ask_supervisor",
"supervisor — an unstated product/API/scope choice, or a choice of an authorised file or resource — call ask_supervisor",
"and wait for the answer instead of guessing. Ask the smallest question that unblocks you.",
].join(" ");
}
Expand Down
31 changes: 28 additions & 3 deletions test/subagent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -412,9 +412,12 @@ test("v0.6 a finished run with no answer is never reported as still running", ()
});

test("v0.6 ask_supervisor asks the smallest question, and handles a refusal", () => {
assert.equal(askTitle("reviewer", "need_decision"), "Subagent reviewer needs a decision");
assert.equal(askTitle("scout", "clarify_scope"), "Subagent scout needs the scope clarified");
assert.equal(askTitle("worker", "missing_access"), "Subagent worker is missing access");
assert.equal(askTitle("reviewer", "need_decision", "Which database?"),
"Subagent reviewer needs a decision\n\nWhich database?\n\nReplying does not add tools or change access permissions.");
assert.equal(askTitle("scout", "clarify_scope", " Which files? ", " Sources only. "),
"Subagent scout needs the scope clarified\n\nWhich files?\n\nSources only.\n\nReplying does not add tools or change access permissions.");
assert.equal(askTitle("worker", "missing_access", "Which input file?", " "),
"Subagent worker is missing access\n\nWhich input file?\n\nReplying does not add tools or change access permissions.");

assert.equal(askBody("Which database?", " "), "Which database?");
assert.equal(askBody(" Which database? ", "Postgres is already a dependency."),
Expand All @@ -435,6 +438,28 @@ test("v0.6 ask_supervisor asks the smallest question, and handles a refusal", ()
assert.ok(ASK_EXHAUSTED.includes(String(ASK_BUDGET)));
});

test("ask_supervisor keeps long, multiline questions and context in the visible title", () => {
const question = `Which resource?\n${"Details about the missing access. ".repeat(30)}END OF QUESTION`;
const context = "Already checked:\n- local files\n- agent tools\nEND OF CONTEXT";
assert.ok(question.length > 500);
assert.equal(askTitle("scout", "missing_access", question, context),
`Subagent scout is missing access\n\n${question}\n\n${context}\n\nReplying does not add tools or change access permissions.`);
});

test("access blockers are reported to the caller, not escalated as permission requests", () => {
for (const instructions of [ASK_TOOL_DESCRIPTION, childFraming(true), childFraming(false)]) {
assert.match(instructions, /tool or credential is unavailable/);
assert.match(instructions, /operation is denied/);
assert.match(instructions, /do not ask the user to grant access/);
assert.match(instructions, /report the resource, required tool, exact error \(redacting secrets\), and remaining work to the caller/);
assert.match(instructions, /Do not retry or bypass an access denial/);
assert.match(instructions, /reply does not add tools or change access permissions/);
}
assert.ok(ASK_TOOL_DESCRIPTION.includes("missing_access (a human choice of an authorised file or resource)"));
assert.ok(childFraming(true).includes("a choice of an authorised file or resource"));
assert.ok(!childFraming(true).includes("or missing access — call"));
});

test("v0.6 the ask tool must be on the allowlist, or it is silently dropped", () => {
// pi's `tools` option is an allowlist and it filters customTools too: a
// custom tool missing from it is registered and then removed, and the child
Expand Down
Loading