Skip to content

feat(jobs): seal the policy a job ran under into a verifiable context bundle (#369) - #1120

Merged
HugoXK merged 2 commits into
mainfrom
dev3/369-signed-context-bundle
Nov 19, 2026
Merged

HugoXK merged 2 commits into
mainfrom
dev3/369-signed-context-bundle

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

Written-by: HugoXK
Author: Fledge Platform Dev-3

Tip dbdf61860ef189dc3a8e304aa753ec8f21f501b1 — tree 4f23fa128244c4ea67864400fc50db9d71680116, parent 1108e9ba8596a974c6bdf1b933d2af43d02281b6. Base branch main is ceb912ad3fbf5a5507b13a1221685acb2967f768.

What a user sees differently

Before: the job page showed the goal, the strategy and the skill bundle, and nothing on it answered "which policy did this run actually read?". Two jobs dispatched under different rules looked identical, and no value on the record could be recomputed by a reader — so a policy change between two runs was invisible, and a hash that belonged to another bundle would have been indistinguishable from a correct one.

After: every dispatched job stores a sealed ContextBundle — the compiled policy it ran under, content-addressed with SHA-256 — and the job page shows three things a reader can act on: the bundle hash, whether the callback echoed the same hash, and a field-by-field diff against the previous job of the same repository that recorded a bundle (paths.deny: deploy/ → deploy/, data/, goal: … → …, required.evidence: … → …). When the stored bytes no longer hash to the recorded value, the page prints "the record and the bytes disagree, so this is not verified" together with both hashes; it never renders a verified line for bytes it did not re-hash.

How it is verified

  • verifyContextBundle re-hashes the stored JSON and compares that with the row's value, so a row holding bundle B's bytes while claiming bundle A's hash reads mismatch — the case a self-check cannot catch, because a bundle that verifies itself passes while holding any content.
  • verifyContextBundleEcho compares the callback's echo with the recomputed hash, never with the recorded one, so an echo can never upgrade a bundle whose bytes fail verification. No echo is missing, not pass.
  • diffContextBundles is ordered by the contract's own field order, so the page and the tests read the same sequence.

What changed

File Change
src/lib/contextBundle.ts New. compileContextBundle (wraps #364's contract in {version, contract, contractHash}), contextBundleHash, readContextBundle, verifyContextBundle, verifyContextBundleEcho, diffContextBundles.
src/db.ts Three additive job columns (context_bundle_hash, context_bundle_json, context_bundle_state), their migration, createJob/updateJob support, and previousJobWithBundle for the diff.
src/jobs/service.ts Compiles and stores the bundle before the row is inserted, hands the hash to the dispatch, and records the echo verdict on every callback (audited on mismatch).
src/jobs/runner.ts, src/jobs/flybot.ts contextBundleHash on the runner context; the prompt asks the runner to echo it back unchanged.
src/views/pages.ts One card: hash, echo state, previous-bundle diff, and the mismatch refusal.
src/app.ts Three lines: pass previousJob to the job page.
test/contextBundle.test.ts, test/jobContextBundle.test.ts The verification and row cases.
docs/architecture.md, docs/api.md The sealed-bundle section; the callback contract gains contextBundleHash.

Two fields that are deliberately not guesses

model.name records this.config.llm.model, and playbook.version is null with playbook.hash carrying the rules commit hash the dispatch can actually observe, because the instance picks the playbook after the row exists. Recording a version the dispatch never saw would make the bundle claim something it cannot prove; both choices are stated in docs/architecture.md.

Alternatives considered

  • Hash the contract alone (#364's instructionContractHash) — rejected as the job's identity: the envelope needs a version of its own so the bundle format can change without pretending the policy did.
  • Trust the row's hash on read — rejected: that is the self-verifying bundle the card names as worse than none.
  • Render a diff from the two contracts inside the view — rejected: the comparison would then live in the UI and could not be tested without a page.

Uncertainty and residual risk

  • The card is server-rendered HTML and is not asserted by a test; the module and the row are. The mismatch sentence is therefore proven at the module level, not at the pixel level.
  • playbook.version stays null until the dispatch records the version it selected; that is a smaller truth than a guessed number, not a complete one.
  • This PR does not evaluate clauses and does not touch tryMerge; the adherence verdict remains feat: compile repo instructions and the playbook into a verifiable adherence contract #364's slice.

Risk

node scripts/risk-gate.mjs --base 95016f39187c8618db3f6dedb01221fb2713b507 --head fab4b9d25fd34583dc8f3365a3954cb47026ac2d at the tip (fab4b9d2 is the committed tree; this PR carries the same tree as 1108e9ba):

Fledge risk tier
base: 95016f39187c8618db3f6dedb01221fb2713b507
head: fab4b9d25fd34583dc8f3365a3954cb47026ac2d
changed files: 12

T3: src/db.ts, src/jobs/flybot.ts, src/jobs/runner.ts, src/jobs/service.ts
T1: test/contextBundle.test.ts, test/jobContextBundle.test.ts
T0: changelog.d/369.added.md, docs/api.md, docs/architecture.md
limits: 40 files / 1200 lines (advisory)

risk-tier: T3; basis: src/db.ts

T3 by path (src/db.ts is the strictest file in the diff), no escalation rule fired, so this needs a PASS at this tip.

Ran:

  • npm run check at fab4b9d2 (tree aa797249, the tree this pull request carries as 1108e9ba) — pass: lint baseline clean, tsc --noEmit clean, 3034 tests / 0 fail, tsc -p tsconfig.build.json OK
  • npx tsx --test test/contextBundle.test.ts test/jobContextBundle.test.ts at 95016f39187c8618db3f6dedb01221fb2713b507 — the pushed commit's actual parent — in a clean worktree at that commit with only these two test files copied in: 3 tests, 0 pass, 3 fail. test/contextBundle.test.ts does not load (ERR_MODULE_NOT_FOUND for src/lib/contextBundle.js), and both row tests fail on exactly the columns this change adds (job.context_bundle_hash reads undefined, where the test expects the stored hash and null). That is the red half.
  • node scripts/risk-gate.mjs at the tip — output above

Not run:

  • Docker image and CI from the author's box; the pixel reading of the frames (no image-capable model in the team), and the browser render was done by Dev-4 for the frames rather than by the author.

The rendered test and the mock snapshot (second commit)

The seat refused the first tip for two things, both closed by dbdf61860ef189dc3a8e304aa753ec8f21f501b1:

  • test/jobPageContextBundle.test.ts renders the job page and asserts the four states a reader can see: the hash is printed; an echo of missing reads as unconfirmed rather than a pass; a stored-JSON/hash mismatch renders the refusal and never a verified line; the previous-bundle diff follows the contract's field order.
  • mock/job.html is regenerated with npm run mock:snapshot, because the card is part of the generated view — the previous tip's CI failed exactly there (mock-check, "The generated UI snapshot matches the views and the stylesheet").

Measured at this tree by me, after the push: npm run mock:check → mock-snapshot: ok (14 files); npx tsx --test test/jobPageContextBundle.test.ts → 3 tests, 3 pass. The author's npm run check at the parent tree (3034/0) still describes the bundle slice; this commit adds one test file and one generated snapshot and changes no product code.

The before/after frames for the card were shot on 1108e9ba by Dev-4 — the card itself is unchanged by the second commit, which only adds the test and the generated snapshot — and they are pushed as evidence/369 (commit 4049e7609826d0384bdef14bdc955f43884abec7), not in this PR tree:

view before after
1280, zh before after
1280, en before after
390, zh before after
390, en before after

Fixes #369

@HugoXK-bot HugoXK-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES at 1108e9b
Written-by: HugoXK-bot

  • The module and row halves hold: verifyContextBundle re-hashes the stored bytes (src/lib/contextBundle.ts:96-101), the echo is compared with the recomputed hash (:116-135), and diffContextBundles follows the contract's field order (:139-167 vs src/lib/instructions.ts:39-51); 138 bundle/jobs tests and 62 page tests pass and tsc --noEmit is clean on this tip.
  • The new job-page card (src/views/pages.ts:4222-4244) is a user-visible UI change with no before/after screenshot, which the owner's 2026-11-18 rule refuses: no screenshot, no PASS.
  • It is also the one part with no test — this body says the card is not asserted — so the mismatch refusal and the three echo sentences are proven at module level only, never as rendered.
  • If the owner waives the screenshot rule for this card, say so on this PR and I will re-read the same tip (no re-push needed); the code boundary is right: src/lib/contextBundle.ts only wraps #364's compileInstructionContract, with no second compiler and no clause evaluation.
    Ran: npx tsx --test test/contextBundle.test.ts test/jobContextBundle.test.ts test/jobs.test.ts
    Ran: npx tsx --test test/pages.test.ts
    Not run: browser render or pixel read of the new card; CI unpickable at this sha (queued, not red).

@HugoXK-bot HugoXK-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: PASS at dbdf618
Written-by: HugoXK-bot
Author: Fledge Platform Dev-3
Replaces: 1108e9b

test/jobPageContextBundle.test.ts:72-97 renders a row holding bundle B's bytes under bundle A's hash through jobPage and requires the refusal; forcing src/views/pages.ts:4229 onto the verified branch turns that case red (1 fail, restored), so the assertion is load-bearing rather than a substring wish.
src/views/pages.ts:4210 re-hashes the stored JSON, and the verified sentence at :4230 is reachable only for state === 'verified'; the echo at :4215-4221 keeps missing and mismatched distinct, and the diff at :4239 keeps diffContextBundles' contract order.
The frames exist: evidence/369 commit 4049e7609826 carries all 8 (widths 1280/390, all nonzero), and 1108e9ba...dbdf6186 changes only mock/job.html and the test, so they still describe this card; pixels are unreadable to this model, so the visual check stays the owner's.
The generated snapshot survives the base move: main advanced 95016f39 → ceb912ad (deploy docs and scripts only), and applying this diff onto ceb912ad still gives mock-snapshot: ok (14 files), so mock/job.html is not stale at merge.
The body's main sha is right but the PR's recorded base is still 95016f39, and its risk-gate head fab4b9d2 returns 422 — the tier stands on src/db.ts = T3 by path rules, not on that transcript.

Ran:

  • npx tsx --test test/jobPageContextBundle.test.ts at dbdf6186 — 3 tests, 3 pass
  • npx tsx --test test/contextBundle.test.ts test/jobContextBundle.test.ts test/jobPageContextBundle.test.ts test/pages.test.ts — 75 tests, 75 pass
  • npx tsc --noEmit — clean
  • npm run mock:check at the tip — ok (14 files); npm run mock:check on the diff applied to ceb912ad — ok (14 files)
  • curl -s https://api.github.com/repos/HugoXK/fledge/commits/dbdf6186... — tree 4f23fa128244c4ea67864400fc50db9d71680116, parent 1108e9ba8596a974c6bdf1b933d2af43d02281b6

Not run: CI at this tip (every job still queued — no runner pickup since ~15:49Z, pr-lint last run cancelled); Docker; reading the 8 frames as pixels (this model declares no image input), so the visual confirmation stays with the owner.

@HugoXK
HugoXK merged commit 0079b96 into main Nov 19, 2026
36 of 39 checks passed
@HugoXK
HugoXK deleted the dev3/369-signed-context-bundle branch November 19, 2026 16:46
Adds test/jobPageContextBundle.test.ts: renders the job page and asserts the
bundle hash is printed, an echo of "missing" reads as unconfirmed rather than
a pass, a stored-JSON/hash mismatch renders the refusal and never a verified
line, and the previous-bundle diff lists fields in contract order.

Also regenerates mock/job.html (npm run mock:snapshot) so mock-check is green
on this tree: the new card section is part of the generated snapshot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: compile the repo policy, playbook and goal into a signed context bundle

2 participants