Skip to content

feat(delivery): keep what happened after the merge, not only that it merged (#371) - #1062

Merged
HugoXK merged 15 commits into
mainfrom
dev4/371-delivery-outcome
Nov 19, 2026
Merged

HugoXK merged 15 commits into
mainfrom
dev4/371-delivery-outcome

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

Written-by: HugoXK
Author: Fledge Platform Dev-4

Fixes #371

What changed

Fledge recorded that a pull request merged and then forgot it. This is the post-merge ledger and its window, with no canary pause and no merge-gate change (both non-goals of the PASSed body).

Ledger. delivery_outcomes holds one row per merged pull request: merge commit, instant, title, and the producing job's cost_usd, provider_model and work_type. The unique key is (repo_id, pr_url), so a replayed webhook event inserts nothing. Facts attach to the row and are never rewritten to say something else.

Facts. A revert is git's own This reverts commit <sha>. trailer, found by scanning the base branch's recent commits after a merge (GithubClient.listRecentCommits, one new outbound read, best-effort: a token or network failure leaves the row un-reverted rather than failing the webhook). The first sighting keeps its instant; a second scan changes nothing. A follow-up issue whose body names the delivered pull request is linked once — a second comment on the same issue names the same link. A failing default-branch run whose head is a merge commit adds one post-merge failure, so a second failing run is a second fact.

Window. The repository's Delivery tab reads 90 days: merged, reverted, revert/follow-up/post-merge-failure rates, running cost and cost per kept change. A fact nobody recorded prints —, never 0; under two deliveries the card says the number is a hint, not a rate; a repository with no delivery says so instead of showing a zero card.

Retention. RETENTION_CLASSES.deliveryOutcomes (365 days, merged_at) is pruned by the same pruneOldData run as the jobs class, so a delivery and the job that produced it are deleted together; docs/deploy.md names the variable. test/retention.test.ts's pinned default map gains the one entry.

Rework at 92d3b6f

The coverage leg failed at df9b114 (112/185 changed lines covered (60.5%, floor 80%), uncovered additions at src/views/pages.ts:1477-1494) and pr-lint failed on the body's ## Ran heading and on src/repos/ owning docs/api.md. Three commits fix that, and the branch then merged main at 3968056 (two conflicts, both resolved additively).

  1. test/deliveryOutcome.test.ts grew from 6 to 13 cases. The Delivery card is rendered, not just summarised: every fact recorded, nothing recorded, the — a missing number prints, the rate labels, "too few deliveries is a hint", and the empty ledger in words.
  2. The write path is covered too: a merge copies the producing job's cost and model, a merge with no job carries neither, a failing default-branch run at a merge commit counts one post-merge failure per run (two after two runs), and listRecentCommits returns one page and drops an entry it cannot name. The webhook cases post through the real route against a fake API that refuses every call, so the best-effort paths are what is under test.
  3. docs/api.md documents listRecentCommits under "Post-merge ledger reads", which is the src/repos/ doc.

The merge of 3968056 resolves src/db.ts (both imports kept: the delivery-outcome read and actionIdFor from #1063) and docs/architecture.md (the #1040 producer paragraph is kept and this PR's ledger paragraph follows it). The diff against 3968056 is the same 13 files +726/−3; the merge adds no product change of its own.

Merge at 61fe7dd

main moved to 050af326 (#380 skill bundles, #984 dispatch ref) and the branch no longer merged cleanly: src/views/pages.ts conflicted in exactly two places — the repoPage input type (this card's deliveryOutcomes, main's skillSources/canMaintain) and the page body (this card's deliveryOutcomesCard, main's skillsCard). Both are kept additively: the input carries all three fields, the overview section keeps skillsCard, the delivery section keeps deliveryOutcomesCard. src/app.ts, src/db.ts and the docs merged without conflict. The product change of this PR is unchanged by the merge, and the diff against 050af326 is still 13 files +726/−3.

Merge at a9d0648b, then five more main tips

main moved past this branch from 050af326 to e725796e (260d8d1b #1094, 0ad5cd8b #1042, a9d0648b #1096, bb5e2012 #1095, 0c699ae6 #1092, 0d10abad #966, e725796e #1061), merged once at each of a9d0648b, bb5e2012, 0c699ae6, 0d10abad and e725796e. None conflicted in a product file: src/views/pages.ts, src/app.ts, src/db.ts, docs/api.md and docs/architecture.md merged cleanly each time — main's side of them was the release-slot work, the memory key field, the DeepWiki panel and the job-proposed memory notes, none of it in repoPage's Delivery tab.

The a9d0648b merge carries the regenerated mock/ snapshot, per the rule that an HTML-surface change ships its snapshot: npm run mock:check was red at that merge tip with exactly the two pages this card touches — repository.html and repository-demo.html — and npm run mock:snapshot rewrote those two and nothing else. The regenerated delta is the card's empty state: <section class="card" id="delivery-outcomes" data-delivery-outcomes="0"> with its one sentence, inserted between the review-trend card and the pending-PR card.

The later merges needed no second regeneration of this card: bb5e2012 touched only src/lib/visualMatrix.ts, 0c699ae6 the memory page's Key field, and 0d10abad the DeepWiki panel, and e725796e the memory job-note surface — main regenerated its own snapshot for the DeepWiki one, and this branch merged it as-is. npm run mock:check is green at the final tip over 14 files (13 plus mock/assets/fledge-deepwiki.css, which 0d10abad added).

docs/deploy.md keeps its retention row for RETENTION_CLASSES.deliveryOutcomes; the diff against e725796e is 15 files +728/−5, and against the merged base cc60fdf9 the card is 15 files +772/−5.

Merge at cc60fdf9 — the visual leg, and why it was skipped

The ci-ok red was not this branch's code: the run at 54253f8e read the classifier from the branch's own tree, which predates #1106, so the merged required-set demanded a visual job the old classifier never scheduled (required job did not succeed: visual (skipped)).

cc60fdf9 is merged in one commit, no conflicts and no product change: src/views/pages.ts, src/app.ts, src/db.ts, docs/api.md, docs/architecture.md and the mock/ baseline all merged cleanly, and npm run mock:check stayed green over 14 files without regenerating the snapshot (main's regeneration already covers this card's two pages).

With the merged classifier the leg is scheduled for this diff — node scripts/ci-changes.mjs --git-base cc60fdf9 --git-head HEAD prints "visual":true — and at this tip the visual job ran and passed: job 135643035305 in run 44547105990, conclusion success.

The matrix prints exactly these two lines when it is clean, and the same tree printed them under npm run visual:matrix here:

visual matrix: 6 pages x 9 widths x 2 locales + 1 narrow case(s) clean
visual gate: 6 pages x 3 widths x 2 schemes x 2 locales, touch/focus/name/leftover clean

Screenshots

evidence/371 carries the files; the PR tree does not. Same seed, 1280 px, zh-CN, the Delivery tab.

  • Before (cb3a0e2b): delivery-before-1280.png — 300894 B, 1280×2745. No delivery card; the merge left only a job row.
  • After (df9b114): delivery-after-1280.png — 344109 B, 1280×3223. One delivery: merged 1 / reverted 1, revert rate 100%, follow-up 1, post-merge failure 1, cost per kept change — because the only change was reverted.
  • DOM: before {card: false}; after {card: true, count: 1, merged: 1, reverted: 1, revertRate: "100%", costPerKept: "—"}.

The frames were captured at df9b114, before both later merges (3968056, 61fe7dd). Neither touches deliveryOutcomesCard's markup: 3968056 moves dashboardPage, the job page and the repo page's budget block; 050af326 adds the skill-bundle card to the repo page's overview section (#380); public/app.css moved only the select/summary minimum heights plus new dashboard rules; a9d0648b regenerated mock/, which this branch regenerated itself, and bb5e2012 touched only src/lib/visualMatrix.ts.

Review at 7789c786 — the failure rate counted runs, not deliveries

The finding is right: src/views/pages.ts:1704 printed deliveryRate(deliverySummary.postMergeFailures, deliverySummary.merged), and summariseDeliveries sums post_merge_failures — failing runs, of which one delivery can carry several (src/lib/deliveryOutcome.ts:74). So merged=1 with two failing runs rendered 0% · 0% · 200%, a rate that cannot exist.

Fixed the way the finding proposed, at the delivery level. DeliveryOutcomeSummary gains withPostMergeFailure — the deliveries whose post_merge_failures > 0 — and the rate prints that over merged, the same shape withFollowUp already had. The run total stays postMergeFailures and is unchanged on the row, where "2 failing runs" is a count and is true. The card line also carries the rate as data-delivery-post-merge-failure-rate so the bound is readable, not only the sentence.

Pinned in test/deliveryOutcome.test.ts: one delivery with post_merge_failures: 2 gives withPostMergeFailure 1, postMergeFailures 2, renders data-delivery-post-merge-failure-rate="100%", and the test asserts the page never prints 200%. At 7789c786 the same case failed on withPostMergeFailure being undefined.

Ran:

  • npm run check at this tip (44c04de) — pass, # tests 3001 # pass 3001 # fail 0 (lint, typecheck, build included).
  • npm run mock:check at this tip — mock-snapshot: ok (14 files). It was red at the a9d0648b merge tip with exactly repository.html and repository-demo.html; npm run mock:snapshot rewrote those two only and they are in that merge commit. No later main merge, cc60fdf9 included, needed a regeneration.
  • npm run test:coverage at this tip, then node scripts/coverage-diff.mjs --base cc60fdf932dacce252a77552ccf93fc63ef1368f --head HEAD --report coverage.txt --floor 80 — coverage-diff: ok — 160/189 changed lines covered (84.7%, floor 80%); over the review commit alone against 7789c786 it is 5/6 changed lines covered (83.3%, floor 80%).
  • npx tsx --test test/deliveryOutcome.test.ts test/retention.test.ts — pass at this tip, # tests 26 # pass 26 # fail 0.
  • node scripts/risk-gate.mjs --base cc60fdf932dacce252a77552ccf93fc63ef1368f — risk-tier: T3; basis: scripts/lib/retention.mjs; over the review commit alone against 7789c786 — risk-tier: T2; basis: src/lib/deliveryOutcome.ts.
  • node scripts/version.mjs check — version: 0.4.2 ok; node scripts/changelog.mjs check — changelog: 59 fragment(s) ok.
    Not run: a real GitHub webhook carrying a revert merge; the Docker image; a real FlyBot run.

Risk

The ledger is additive: no existing table or job field changes, and no gate reads it. The only moved surface is the repository's Delivery tab, which gains a card. The revert scan runs after a merge and can fail without affecting the merge path; the row then simply stays un-reverted until the next merge in that repository scans again. Cost and model are copied at merge time, so a later job-cost correction does not rewrite a delivery — that is the append-only choice, and it can under-report a cost that was still settling.

Fledge Platform Dev-4 and others added 6 commits November 19, 2026 06:01
…ion (#371)

Red at main cb3a0e2: the ledger, the matcher, the summary and the retention
class do not exist, so every case fails at import.

Written-by: Fledge Platform Dev-4
The read side of the post-merge ledger: a revert is git's own trailer, not a
title, and a summary leaves a cost nobody recorded as null so the view prints —
instead of a confident zero.

Written-by: Fledge Platform Dev-4
…merged (#371)

A merged pull request becomes one delivery_outcomes row — merge commit, instant,
title, and the producing job's cost, model and work type — and the facts after
it attach to that row: a revert found from git's own trailer in the base
branch's recent commits (recorded once, at its first sighting), a follow-up
issue whose body names the pull request, and every failing default-branch run
whose head is the merge commit. The unique key is (repo_id, pr_url), so a
replayed webhook inserts nothing.

The repository's Delivery tab reads one 90-day window: merged, reverted, the
revert/follow-up/post-merge-failure rates and the cost per kept change, with —
where a number was never recorded and a sample label under two deliveries. No
merge gate and no scheduler pause: the ledger records.

The row is pruned with the jobs by RETENTION_CLASSES.deliveryOutcomes (365 days)
and docs/deploy.md names the window.

Ran: npm run check — pass (2784/2784)
Not run: a real GitHub webhook with a revert, the Docker image

Written-by: Fledge Platform Dev-4
The coverage floor counted the post-merge card as uncovered: no test rendered it. Two cases now render the repo page with the ledger - one with every fact, one with none - and the empty ledger in words. docs/api.md documents listRecentCommits, which src/repos/ now owns.
The changed-line floor also counted the write side: a merge copying its job cost and model, a merge with no job, and the one-page commit read that finds a revert trailer. Each gets the smallest case that fails without the line.
A merged pull request writes the row the page reads, and a failing default-branch run at a merge commit counts one post-merge failure per run. Both run through the real webhook route against a fake GitHub API that refuses every call, so the best-effort paths are the ones under test.
@HugoXK

HugoXK commented Nov 19, 2026

Copy link
Copy Markdown
Owner Author

Written-by: Fledge Platform Dev-4

Owner-authored PR; Fledge Platform Dev-4 takes over the remaining fixes on the owner's behalf: coverage tests for the Delivery card and the ledger write path, the docs/api.md commit read, and the PR body's Ran: and docs items.

Fledge Platform Dev-4 and others added 7 commits November 19, 2026 09:28
Written-by: Fledge Platform Dev-4

Conflicts: src/db.ts keeps both imports (delivery outcome read and the #1063 action id); docs/architecture.md keeps the #1040 producer paragraph and appends the #371 ledger paragraph.
main gained the skill-bundle card in repoPage (#380) while this branch adds the
delivery-outcomes card, so the page's input and its overview/delivery sections
keep both: skillsCard in overview, deliveryOutcomesCard in delivery.

@HugoXK-bot HugoXK-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES at 7789c78

  • src/views/pages.ts:1704 prints the third figure as deliveryRate(deliverySummary.postMergeFailures, deliverySummary.merged), but the numerator sums failing runs (src/lib/deliveryOutcome.ts:74) while the denominator is deliveries: I rendered one delivery with two failing runs through repoPage and the card shows 0% · 0% · 200%; two deliveries with three runs shows 150%.
  • Fix the third figure so it cannot exceed 100% as a rate: either count deliveries with post_merge_failures > 0 over merged (the same shape as the follow-up rate), or drop the % and print runs per delivery. Pin merged=1, post_merge_failures=2 in test/deliveryOutcome.test.ts so the number is bounded by a test.
  • The rest checked out: the merged-state snapshot is green (I applied #1105's diff on top of this tip and npm run mock:check is ok over 14 files), the ledger dedupes on (repo_id, pr_url) (src/db.ts:1628-1660), retention shares the 365-day jobs window (scripts/lib/retention.mjs:124-135), and the frames on evidence/371 are 300894/344109 B at 1280×2745/1280×3223.

Ran: npm run check at 7789c786 — pass, # tests 2975 # pass 2975 # fail 0; npm run mock:check — mock-snapshot: ok (14 files); npx tsx render of one delivery with two failing runs — 0% · 0% · 200%
Not run: a real GitHub webhook carrying a revert merge; the Docker image; npm run test:coverage.

@HugoXK-bot HugoXK-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES at 54253f8
Written-by: HugoXK-bot
Author: Fledge Platform Verdict

  • The 200% fix is right: src/lib/deliveryOutcome.ts:96-97 adds withPostMergeFailure (deliveries whose post_merge_failures > 0), src/views/pages.ts:1704 prints that over merged, the run total stays postMergeFailures, and test/deliveryOutcome.test.ts:741-770 pins merged=1/failures=2 → data-delivery-post-merge-failure-rate="100%" with assert.doesNotMatch(/200%/); npx tsx --test test/deliveryOutcome.test.ts test/retention.test.ts → 26/26 pass.
  • Merged-state snapshot is green: the diff applied onto current main cc60fdf9 then npx tsx scripts/mock-snapshot.ts --check → mock-snapshot: ok (14 files).
  • But ci-ok is failure on this head (run 44540329139): its changes step read the base tree at e725796e (the run's base sha), whose ciChanges.mjs has no visual output, while the default-branch gate merged at 14:01Z (2a956803) binds visual to that output (scripts/ci-ok.mjs:84), so visual skipped and the gate reported "required job did not succeed: visual (skipped)".
  • Merge current main into the tip so the classifier the run reads and the required set the gate enforces agree, then ping me at the new sha; the product change needs nothing further.
  • The card frames in the body (evidence/371) still match this tip: the rate switch only changes a value when one delivery carries two or more failures, which the shown frame does not contain.

Fledge Platform Dev-4 and others added 2 commits November 19, 2026 14:32
…failure rate (#371)

The card printed `deliveryRate(postMergeFailures, merged)`: the numerator summed failing runs (one delivery can carry several, src/lib/deliveryOutcome.ts) while the denominator counted deliveries, so merged=1 with post_merge_failures=2 rendered 200%. The rate now counts deliveries that carried at least one failing run, the same shape the revert and follow-up rates already use; the run total stays the per-row count. test/deliveryOutcome.test.ts pins merged=1 / post_merge_failures=2 at 100% and forbids a 200% render.

@HugoXK-bot HugoXK-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: PASS at 44c04de
Written-by: HugoXK-bot
Replaces: 54253f8
Ran: npm run mock:check && npx tsx --test test/pages.test.ts test/deliveryOutcome.test.ts test/retention.test.ts
Not run: the full suite locally and the browser matrix by hand; CI test is green at this head and the visual job 135643035305 in run 44547105990 succeeded at this head.
src/views/pages.ts:1704 divides withPostMergeFailure by merged, so one delivery with post_merge_failures: 2 prints 100%, not 200% (test/deliveryOutcome.test.ts:192-193).
The red ci-ok at 54253f8e was the skipped visual leg; with the merged classifier visual ran at this head and passed (14:53:49Z), and its log prints both lines the body quotes.
The tip merge brings only main in; merged onto the current main 186e71d4 it takes cleanly, mock:check is ok over 14 files, typecheck is clean, and 88/88 tests in pages, deliveryOutcome and retention pass on that merged tree.
Before/after frames are on evidence/371: docs/images/371/delivery-before-1280.png, delivery-after-1280.png.

@HugoXK
HugoXK merged commit 95016f3 into main Nov 19, 2026
36 of 37 checks passed
@HugoXK
HugoXK deleted the dev4/371-delivery-outcome branch November 19, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: track what happened after merge, not only that the pull request merged

2 participants