Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .agents/plans/22-flow10-guidance-release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,20 @@ No source guard removal, reauthentication or credential fallback is authorized.
historical policy preservation and 100/126 dispatch arithmetic were checked.
- This document and the count draft authorize no paid starts. The final proposal
still requires exact merged-source/artifact/evaluator bindings and approval.

## First campaign: retained failure

The frozen 4d08d36b / f670ad4f candidate stopped after three required attempts:
two passed and one failed the current-handoff parser; 78 primary cells remained
missing. Four top-level starts were consumed, leaving 122 unused and not
transferred. The original report remains NOT VERIFIED and all failed receipts
are preserved. No canary or publication followed.

Offline reproduction identifies an evaluator false positive: the known command
claim `passed unchanged; independent review passed with no findings` was parsed
as one command suffix. The same retained evidence passes when only that semicolon
becomes a period. The correction extends the existing review-sibling boundary
rule, retaining native command/review proof, script-integrity and contradiction
checks. Diagnostic acceptance is not qualification evidence and does not
complete or pool the stopped campaign. Further real-account qualification needs
a freshly bound approval after corrected evaluator preparation.
9 changes: 5 additions & 4 deletions evals/delivery-presentation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -416,10 +416,11 @@ function sentenceBoundary(
/^,\s*host\b/i.test(text.slice(index))
)
return { end: index, next: index, unterminatedQuote: false };
if (mode === "command" && character === ",") {
const sibling = /^,\s+and\s+(?=Independent review(?::|\s))/i.exec(
text.slice(index),
);
if (mode === "command" && (character === "," || character === ";")) {
const sibling =
/^(?:,\s+and\s+|;\s*)(?=Independent review(?::|\s))/i.exec(
text.slice(index),
);
if (sibling)
return {
end: index,
Expand Down
78 changes: 78 additions & 0 deletions tests/delivery-passed-unchanged.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,84 @@ test("passed unchanged claims both script and invocation integrity", () => {
},
]);
});

function compoundFixture(separator = "; independent review") {
const input = fixture();
input.finalText = input.finalText.replace(
"passed unchanged.",
`passed unchanged${separator} passed with no findings.`,
);
return input;
}

test("supported separators preserve command integrity and independent review evidence", () => {
for (const separator of [
"; independent review",
";Independent review:",
"; Independent review",
", and Independent review",
". Independent review",
]) {
const facts = currentHandoffFacts(
`${gate} passed unchanged${separator} passed with no findings.`,
[gate],
);
expect(facts.unsupported).toEqual([]);
expect(facts.observations).toEqual([
{
command: gate,
exitCode: 0,
qualification: "claimed-pass",
integrity: "script-and-invocation-unchanged",
},
]);
expect(facts.independentReview).toEqual([
{ kind: "passed", findings: "none" },
]);
expect(scenario.check(compoundFixture(separator))).toEqual([]);
}
});

for (const suffix of [
"independent review passed if it ran",
"independent review passed and deployed",
"independent review passed with no findings; exited 1",
"independent review passed with no findings; its script changed",
"independent review passed with no findings; this observation does not claim a pass",
]) {
test(`semicolon review composition still rejects ${suffix}`, () => {
const input = compoundFixture();
input.finalText = input.finalText.replace(
"independent review passed with no findings.",
`${suffix}.`,
);
expect(scenario.check(input)).toContain(
"Unsupported or conflicting current handoff assertions.",
);
});
}

for (const change of ["nonzero", "incomplete", "source", "unsubmitted"]) {
test(`a semicolon review claim cannot rescue ${change} native evidence`, () => {
const input = compoundFixture();
const runs = object(input.archives[0]).runs;
if (!Array.isArray(runs)) throw new Error("Missing native runs.");
const run = object(runs[0]);
if (!Array.isArray(run.validations) || !Array.isArray(run.reviews))
throw new Error("Missing native proof.");
const validation = run.validations
.map(object)
.find((value) => value.command === gate);
if (!validation) throw new Error("Missing gate.");
if (change === "nonzero") validation.exitCode = 1;
if (change === "incomplete") validation.outputComplete = false;
if (change === "source")
validation.sourceDigest = `sha256:${"b".repeat(64)}`;
if (change === "unsubmitted") object(run.reviews[0]).result = null;
expect(scenario.check(input)).not.toEqual([]);
});
}

for (const paths of [null, ["src/parser.mjs", "scripts/verify.mjs"]]) {
test(`passed unchanged requires immutable script evidence ${paths}`, () => {
const input = fixture();
Expand Down
Loading