Repository navigation
feat(jobs): seal the policy a job ran under into a verifiable context bundle (#369) - #1120
Conversation
HugoXK-bot
left a comment
There was a problem hiding this comment.
Verdict: CHANGES at 1108e9b
Written-by: HugoXK-bot
- The module and row halves hold:
verifyContextBundlere-hashes the stored bytes (src/lib/contextBundle.ts:96-101), the echo is compared with the recomputed hash (:116-135), anddiffContextBundlesfollows the contract's field order (:139-167vssrc/lib/instructions.ts:39-51); 138 bundle/jobs tests and 62 page tests pass andtsc --noEmitis clean on this tip. - The new job-page card (
src/views/pages.ts:4222-4244) is a user-visible UI change with no before/after screenshot, which the owner's 2026-11-18 rule refuses: no screenshot, no PASS. - It is also the one part with no test — this body says the card is not asserted — so the mismatch refusal and the three echo sentences are proven at module level only, never as rendered.
- If the owner waives the screenshot rule for this card, say so on this PR and I will re-read the same tip (no re-push needed); the code boundary is right:
src/lib/contextBundle.tsonly wraps #364'scompileInstructionContract, with no second compiler and no clause evaluation.
Ran: npx tsx --test test/contextBundle.test.ts test/jobContextBundle.test.ts test/jobs.test.ts
Ran: npx tsx --test test/pages.test.ts
Not run: browser render or pixel read of the new card; CI unpickable at this sha (queued, not red).
HugoXK-bot
left a comment
There was a problem hiding this comment.
Verdict: PASS at dbdf618
Written-by: HugoXK-bot
Author: Fledge Platform Dev-3
Replaces: 1108e9b
test/jobPageContextBundle.test.ts:72-97 renders a row holding bundle B's bytes under bundle A's hash through jobPage and requires the refusal; forcing src/views/pages.ts:4229 onto the verified branch turns that case red (1 fail, restored), so the assertion is load-bearing rather than a substring wish.
src/views/pages.ts:4210 re-hashes the stored JSON, and the verified sentence at :4230 is reachable only for state === 'verified'; the echo at :4215-4221 keeps missing and mismatched distinct, and the diff at :4239 keeps diffContextBundles' contract order.
The frames exist: evidence/369 commit 4049e7609826 carries all 8 (widths 1280/390, all nonzero), and 1108e9ba...dbdf6186 changes only mock/job.html and the test, so they still describe this card; pixels are unreadable to this model, so the visual check stays the owner's.
The generated snapshot survives the base move: main advanced 95016f39 → ceb912ad (deploy docs and scripts only), and applying this diff onto ceb912ad still gives mock-snapshot: ok (14 files), so mock/job.html is not stale at merge.
The body's main sha is right but the PR's recorded base is still 95016f39, and its risk-gate head fab4b9d2 returns 422 — the tier stands on src/db.ts = T3 by path rules, not on that transcript.
Ran:
npx tsx --test test/jobPageContextBundle.test.tsatdbdf6186— 3 tests, 3 passnpx tsx --test test/contextBundle.test.ts test/jobContextBundle.test.ts test/jobPageContextBundle.test.ts test/pages.test.ts— 75 tests, 75 passnpx tsc --noEmit— cleannpm run mock:checkat the tip — ok (14 files);npm run mock:checkon the diff applied toceb912ad— ok (14 files)curl -s https://api.github.com/repos/HugoXK/fledge/commits/dbdf6186...— tree4f23fa128244c4ea67864400fc50db9d71680116, parent1108e9ba8596a974c6bdf1b933d2af43d02281b6
Not run: CI at this tip (every job still queued — no runner pickup since ~15:49Z, pr-lint last run cancelled); Docker; reading the 8 frames as pixels (this model declares no image input), so the visual confirmation stays with the owner.
Adds test/jobPageContextBundle.test.ts: renders the job page and asserts the bundle hash is printed, an echo of "missing" reads as unconfirmed rather than a pass, a stored-JSON/hash mismatch renders the refusal and never a verified line, and the previous-bundle diff lists fields in contract order. Also regenerates mock/job.html (npm run mock:snapshot) so mock-check is green on this tree: the new card section is part of the generated snapshot.
Written-by: HugoXK
Author: Fledge Platform Dev-3
Tip
dbdf61860ef189dc3a8e304aa753ec8f21f501b1— tree4f23fa128244c4ea67864400fc50db9d71680116, parent1108e9ba8596a974c6bdf1b933d2af43d02281b6. Base branchmainisceb912ad3fbf5a5507b13a1221685acb2967f768.What a user sees differently
Before: the job page showed the goal, the strategy and the skill bundle, and nothing on it answered "which policy did this run actually read?". Two jobs dispatched under different rules looked identical, and no value on the record could be recomputed by a reader — so a policy change between two runs was invisible, and a hash that belonged to another bundle would have been indistinguishable from a correct one.
After: every dispatched job stores a sealed
ContextBundle— the compiled policy it ran under, content-addressed with SHA-256 — and the job page shows three things a reader can act on: the bundle hash, whether the callback echoed the same hash, and a field-by-field diff against the previous job of the same repository that recorded a bundle (paths.deny: deploy/ → deploy/, data/,goal: … → …,required.evidence: … → …). When the stored bytes no longer hash to the recorded value, the page prints "the record and the bytes disagree, so this is not verified" together with both hashes; it never renders a verified line for bytes it did not re-hash.How it is verified
verifyContextBundlere-hashes the stored JSON and compares that with the row's value, so a row holding bundle B's bytes while claiming bundle A's hash readsmismatch— the case a self-check cannot catch, because a bundle that verifies itself passes while holding any content.verifyContextBundleEchocompares the callback's echo with the recomputed hash, never with the recorded one, so an echo can never upgrade a bundle whose bytes fail verification. No echo ismissing, notpass.diffContextBundlesis ordered by the contract's own field order, so the page and the tests read the same sequence.What changed
src/lib/contextBundle.tscompileContextBundle(wraps #364's contract in{version, contract, contractHash}),contextBundleHash,readContextBundle,verifyContextBundle,verifyContextBundleEcho,diffContextBundles.src/db.tscontext_bundle_hash,context_bundle_json,context_bundle_state), their migration,createJob/updateJobsupport, andpreviousJobWithBundlefor the diff.src/jobs/service.tssrc/jobs/runner.ts,src/jobs/flybot.tscontextBundleHashon the runner context; the prompt asks the runner to echo it back unchanged.src/views/pages.tssrc/app.tspreviousJobto the job page.test/contextBundle.test.ts,test/jobContextBundle.test.tsdocs/architecture.md,docs/api.mdcontextBundleHash.Two fields that are deliberately not guesses
model.namerecordsthis.config.llm.model, andplaybook.versionisnullwithplaybook.hashcarrying the rules commit hash the dispatch can actually observe, because the instance picks the playbook after the row exists. Recording a version the dispatch never saw would make the bundle claim something it cannot prove; both choices are stated indocs/architecture.md.Alternatives considered
#364'sinstructionContractHash) — rejected as the job's identity: the envelope needs a version of its own so the bundle format can change without pretending the policy did.Uncertainty and residual risk
playbook.versionstays null until the dispatch records the version it selected; that is a smaller truth than a guessed number, not a complete one.tryMerge; the adherence verdict remains feat: compile repo instructions and the playbook into a verifiable adherence contract #364's slice.Risk
node scripts/risk-gate.mjs --base 95016f39187c8618db3f6dedb01221fb2713b507 --head fab4b9d25fd34583dc8f3365a3954cb47026ac2dat the tip (fab4b9d2is the committed tree; this PR carries the same tree as1108e9ba):T3 by path (
src/db.tsis the strictest file in the diff), no escalation rule fired, so this needs a PASS at this tip.Ran:
npm run checkatfab4b9d2(treeaa797249, the tree this pull request carries as1108e9ba) — pass: lint baseline clean,tsc --noEmitclean, 3034 tests / 0 fail,tsc -p tsconfig.build.jsonOKnpx tsx --test test/contextBundle.test.ts test/jobContextBundle.test.tsat95016f39187c8618db3f6dedb01221fb2713b507— the pushed commit's actual parent — in a clean worktree at that commit with only these two test files copied in: 3 tests, 0 pass, 3 fail.test/contextBundle.test.tsdoes not load (ERR_MODULE_NOT_FOUNDforsrc/lib/contextBundle.js), and both row tests fail on exactly the columns this change adds (job.context_bundle_hashreadsundefined, where the test expects the stored hash andnull). That is the red half.node scripts/risk-gate.mjsat the tip — output aboveNot run:
The rendered test and the mock snapshot (second commit)
The seat refused the first tip for two things, both closed by
dbdf61860ef189dc3a8e304aa753ec8f21f501b1:test/jobPageContextBundle.test.tsrenders the job page and asserts the four states a reader can see: the hash is printed; an echo ofmissingreads as unconfirmed rather than a pass; a stored-JSON/hash mismatch renders the refusal and never a verified line; the previous-bundle diff follows the contract's field order.mock/job.htmlis regenerated withnpm run mock:snapshot, because the card is part of the generated view — the previous tip's CI failed exactly there (mock-check, "The generated UI snapshot matches the views and the stylesheet").Measured at this tree by me, after the push:
npm run mock:check→mock-snapshot: ok (14 files);npx tsx --test test/jobPageContextBundle.test.ts→ 3 tests, 3 pass. The author'snpm run checkat the parent tree (3034/0) still describes the bundle slice; this commit adds one test file and one generated snapshot and changes no product code.The before/after frames for the card were shot on
1108e9baby Dev-4 — the card itself is unchanged by the second commit, which only adds the test and the generated snapshot — and they are pushed asevidence/369(commit4049e7609826d0384bdef14bdc955f43884abec7), not in this PR tree:1280,zh1280,en390,zh390,enFixes #369