Repository navigation
feat(delivery): keep what happened after the merge, not only that it merged (#371) - #1062
Conversation
The read side of the post-merge ledger: a revert is git's own trailer, not a title, and a summary leaves a cost nobody recorded as null so the view prints — instead of a confident zero. Written-by: Fledge Platform Dev-4
…merged (#371) A merged pull request becomes one delivery_outcomes row — merge commit, instant, title, and the producing job's cost, model and work type — and the facts after it attach to that row: a revert found from git's own trailer in the base branch's recent commits (recorded once, at its first sighting), a follow-up issue whose body names the pull request, and every failing default-branch run whose head is the merge commit. The unique key is (repo_id, pr_url), so a replayed webhook inserts nothing. The repository's Delivery tab reads one 90-day window: merged, reverted, the revert/follow-up/post-merge-failure rates and the cost per kept change, with — where a number was never recorded and a sample label under two deliveries. No merge gate and no scheduler pause: the ledger records. The row is pruned with the jobs by RETENTION_CLASSES.deliveryOutcomes (365 days) and docs/deploy.md names the window. Ran: npm run check — pass (2784/2784) Not run: a real GitHub webhook with a revert, the Docker image Written-by: Fledge Platform Dev-4
The coverage floor counted the post-merge card as uncovered: no test rendered it. Two cases now render the repo page with the ledger - one with every fact, one with none - and the empty ledger in words. docs/api.md documents listRecentCommits, which src/repos/ now owns.
The changed-line floor also counted the write side: a merge copying its job cost and model, a merge with no job, and the one-page commit read that finds a revert trailer. Each gets the smallest case that fails without the line.
A merged pull request writes the row the page reads, and a failing default-branch run at a merge commit counts one post-merge failure per run. Both run through the real webhook route against a fake GitHub API that refuses every call, so the best-effort paths are the ones under test.
|
Written-by: Fledge Platform Dev-4 Owner-authored PR; Fledge Platform Dev-4 takes over the remaining fixes on the owner's behalf: coverage tests for the Delivery card and the ledger write path, the |
main gained the skill-bundle card in repoPage (#380) while this branch adds the delivery-outcomes card, so the page's input and its overview/delivery sections keep both: skillsCard in overview, deliveryOutcomesCard in delivery.
HugoXK-bot
left a comment
There was a problem hiding this comment.
Verdict: CHANGES at 7789c78
src/views/pages.ts:1704prints the third figure asdeliveryRate(deliverySummary.postMergeFailures, deliverySummary.merged), but the numerator sums failing runs (src/lib/deliveryOutcome.ts:74) while the denominator is deliveries: I rendered one delivery with two failing runs throughrepoPageand the card shows0% · 0% · 200%; two deliveries with three runs shows 150%.- Fix the third figure so it cannot exceed 100% as a rate: either count deliveries with
post_merge_failures > 0overmerged(the same shape as the follow-up rate), or drop the%and print runs per delivery. Pinmerged=1, post_merge_failures=2intest/deliveryOutcome.test.tsso the number is bounded by a test. - The rest checked out: the merged-state snapshot is green (I applied #1105's diff on top of this tip and
npm run mock:checkis ok over 14 files), the ledger dedupes on(repo_id, pr_url)(src/db.ts:1628-1660), retention shares the 365-day jobs window (scripts/lib/retention.mjs:124-135), and the frames onevidence/371are 300894/344109 B at 1280×2745/1280×3223.
Ran: npm run check at 7789c786 — pass, # tests 2975 # pass 2975 # fail 0; npm run mock:check — mock-snapshot: ok (14 files); npx tsx render of one delivery with two failing runs — 0% · 0% · 200%
Not run: a real GitHub webhook carrying a revert merge; the Docker image; npm run test:coverage.
HugoXK-bot
left a comment
There was a problem hiding this comment.
Verdict: CHANGES at 54253f8
Written-by: HugoXK-bot
Author: Fledge Platform Verdict
- The
200%fix is right:src/lib/deliveryOutcome.ts:96-97addswithPostMergeFailure(deliveries whosepost_merge_failures > 0),src/views/pages.ts:1704prints that overmerged, the run total stayspostMergeFailures, andtest/deliveryOutcome.test.ts:741-770pins merged=1/failures=2 →data-delivery-post-merge-failure-rate="100%"withassert.doesNotMatch(/200%/);npx tsx --test test/deliveryOutcome.test.ts test/retention.test.ts→ 26/26 pass. - Merged-state snapshot is green: the diff applied onto current main
cc60fdf9thennpx tsx scripts/mock-snapshot.ts --check→mock-snapshot: ok (14 files). - But
ci-okis failure on this head (run 44540329139): itschangesstep read the base tree ate725796e(the run's base sha), whoseciChanges.mjshas novisualoutput, while the default-branch gate merged at 14:01Z (2a956803) bindsvisualto that output (scripts/ci-ok.mjs:84), sovisualskipped and the gate reported "required job did not succeed: visual (skipped)". - Merge current main into the tip so the classifier the run reads and the required set the gate enforces agree, then ping me at the new sha; the product change needs nothing further.
- The card frames in the body (
evidence/371) still match this tip: the rate switch only changes a value when one delivery carries two or more failures, which the shown frame does not contain.
…failure rate (#371) The card printed `deliveryRate(postMergeFailures, merged)`: the numerator summed failing runs (one delivery can carry several, src/lib/deliveryOutcome.ts) while the denominator counted deliveries, so merged=1 with post_merge_failures=2 rendered 200%. The rate now counts deliveries that carried at least one failing run, the same shape the revert and follow-up rates already use; the run total stays the per-row count. test/deliveryOutcome.test.ts pins merged=1 / post_merge_failures=2 at 100% and forbids a 200% render.
HugoXK-bot
left a comment
There was a problem hiding this comment.
Verdict: PASS at 44c04de
Written-by: HugoXK-bot
Replaces: 54253f8
Ran: npm run mock:check && npx tsx --test test/pages.test.ts test/deliveryOutcome.test.ts test/retention.test.ts
Not run: the full suite locally and the browser matrix by hand; CI test is green at this head and the visual job 135643035305 in run 44547105990 succeeded at this head.
src/views/pages.ts:1704 divides withPostMergeFailure by merged, so one delivery with post_merge_failures: 2 prints 100%, not 200% (test/deliveryOutcome.test.ts:192-193).
The red ci-ok at 54253f8e was the skipped visual leg; with the merged classifier visual ran at this head and passed (14:53:49Z), and its log prints both lines the body quotes.
The tip merge brings only main in; merged onto the current main 186e71d4 it takes cleanly, mock:check is ok over 14 files, typecheck is clean, and 88/88 tests in pages, deliveryOutcome and retention pass on that merged tree.
Before/after frames are on evidence/371: docs/images/371/delivery-before-1280.png, delivery-after-1280.png.
Written-by: HugoXK
Author: Fledge Platform Dev-4
Fixes #371
What changed
Fledge recorded that a pull request merged and then forgot it. This is the post-merge ledger and its window, with no canary pause and no merge-gate change (both non-goals of the PASSed body).
Ledger.
delivery_outcomesholds one row per merged pull request: merge commit, instant, title, and the producing job'scost_usd,provider_modelandwork_type. The unique key is(repo_id, pr_url), so a replayed webhook event inserts nothing. Facts attach to the row and are never rewritten to say something else.Facts. A revert is git's own
This reverts commit <sha>.trailer, found by scanning the base branch's recent commits after a merge (GithubClient.listRecentCommits, one new outbound read, best-effort: a token or network failure leaves the row un-reverted rather than failing the webhook). The first sighting keeps its instant; a second scan changes nothing. A follow-up issue whose body names the delivered pull request is linked once — a second comment on the same issue names the same link. A failing default-branch run whose head is a merge commit adds one post-merge failure, so a second failing run is a second fact.Window. The repository's Delivery tab reads 90 days: merged, reverted, revert/follow-up/post-merge-failure rates, running cost and cost per kept change. A fact nobody recorded prints
—, never 0; under two deliveries the card says the number is a hint, not a rate; a repository with no delivery says so instead of showing a zero card.Retention.
RETENTION_CLASSES.deliveryOutcomes(365 days,merged_at) is pruned by the samepruneOldDatarun as the jobs class, so a delivery and the job that produced it are deleted together;docs/deploy.mdnames the variable.test/retention.test.ts's pinned default map gains the one entry.Rework at 92d3b6f
The
coverageleg failed atdf9b114(112/185 changed lines covered (60.5%, floor 80%), uncovered additions atsrc/views/pages.ts:1477-1494) andpr-lintfailed on the body's## Ranheading and onsrc/repos/owningdocs/api.md. Three commits fix that, and the branch then mergedmainat3968056(two conflicts, both resolved additively).test/deliveryOutcome.test.tsgrew from 6 to 13 cases. The Delivery card is rendered, not just summarised: every fact recorded, nothing recorded, the—a missing number prints, the rate labels, "too few deliveries is a hint", and the empty ledger in words.listRecentCommitsreturns one page and drops an entry it cannot name. The webhook cases post through the real route against a fake API that refuses every call, so the best-effort paths are what is under test.docs/api.mddocumentslistRecentCommitsunder "Post-merge ledger reads", which is thesrc/repos/doc.The merge of
3968056resolvessrc/db.ts(both imports kept: the delivery-outcome read andactionIdForfrom#1063) anddocs/architecture.md(the#1040producer paragraph is kept and this PR's ledger paragraph follows it). The diff against3968056is the same 13 files+726/−3; the merge adds no product change of its own.Merge at 61fe7dd
mainmoved to050af326(#380skill bundles,#984dispatch ref) and the branch no longer merged cleanly:src/views/pages.tsconflicted in exactly two places — therepoPageinput type (this card'sdeliveryOutcomes, main'sskillSources/canMaintain) and the page body (this card'sdeliveryOutcomesCard, main'sskillsCard). Both are kept additively: the input carries all three fields, the overview section keepsskillsCard, the delivery section keepsdeliveryOutcomesCard.src/app.ts,src/db.tsand the docs merged without conflict. The product change of this PR is unchanged by the merge, and the diff against050af326is still 13 files+726/−3.Merge at
a9d0648b, then five moremaintipsmainmoved past this branch from050af326toe725796e(260d8d1b#1094,0ad5cd8b#1042,a9d0648b#1096,bb5e2012#1095,0c699ae6#1092,0d10abad#966,e725796e#1061), merged once at each ofa9d0648b,bb5e2012,0c699ae6,0d10abadande725796e. None conflicted in a product file:src/views/pages.ts,src/app.ts,src/db.ts,docs/api.mdanddocs/architecture.mdmerged cleanly each time — main's side of them was the release-slot work, the memory key field, the DeepWiki panel and the job-proposed memory notes, none of it inrepoPage's Delivery tab.The
a9d0648bmerge carries the regeneratedmock/snapshot, per the rule that an HTML-surface change ships its snapshot:npm run mock:checkwas red at that merge tip with exactly the two pages this card touches —repository.htmlandrepository-demo.html— andnpm run mock:snapshotrewrote those two and nothing else. The regenerated delta is the card's empty state:<section class="card" id="delivery-outcomes" data-delivery-outcomes="0">with its one sentence, inserted between the review-trend card and the pending-PR card.The later merges needed no second regeneration of this card:
bb5e2012touched onlysrc/lib/visualMatrix.ts,0c699ae6the memory page's Key field, and0d10abadthe DeepWiki panel, ande725796ethe memory job-note surface — main regenerated its own snapshot for the DeepWiki one, and this branch merged it as-is.npm run mock:checkis green at the final tip over 14 files (13 plusmock/assets/fledge-deepwiki.css, which0d10abadadded).docs/deploy.mdkeeps its retention row forRETENTION_CLASSES.deliveryOutcomes; the diff againste725796eis 15 files+728/−5, and against the merged basecc60fdf9the card is 15 files+772/−5.Merge at
cc60fdf9— thevisualleg, and why it was skippedThe
ci-okred was not this branch's code: the run at54253f8eread the classifier from the branch's own tree, which predates#1106, so the merged required-set demanded avisualjob the old classifier never scheduled (required job did not succeed: visual (skipped)).cc60fdf9is merged in one commit, no conflicts and no product change:src/views/pages.ts,src/app.ts,src/db.ts,docs/api.md,docs/architecture.mdand themock/baseline all merged cleanly, andnpm run mock:checkstayed green over 14 files without regenerating the snapshot (main's regeneration already covers this card's two pages).With the merged classifier the leg is scheduled for this diff —
node scripts/ci-changes.mjs --git-base cc60fdf9 --git-head HEADprints"visual":true— and at this tip thevisualjob ran and passed: job135643035305in run44547105990, conclusionsuccess.The matrix prints exactly these two lines when it is clean, and the same tree printed them under
npm run visual:matrixhere:Screenshots
evidence/371carries the files; the PR tree does not. Same seed, 1280 px, zh-CN, the Delivery tab.cb3a0e2b): delivery-before-1280.png — 300894 B, 1280×2745. No delivery card; the merge left only a job row.df9b114): delivery-after-1280.png — 344109 B, 1280×3223. One delivery: merged 1 / reverted 1, revert rate 100%, follow-up 1, post-merge failure 1, cost per kept change—because the only change was reverted.{card: false}; after{card: true, count: 1, merged: 1, reverted: 1, revertRate: "100%", costPerKept: "—"}.The frames were captured at
df9b114, before both later merges (3968056,61fe7dd). Neither touchesdeliveryOutcomesCard's markup:3968056movesdashboardPage, the job page and the repo page's budget block;050af326adds the skill-bundle card to the repo page's overview section (#380);public/app.cssmoved only theselect/summaryminimum heights plus new dashboard rules;a9d0648bregeneratedmock/, which this branch regenerated itself, andbb5e2012touched onlysrc/lib/visualMatrix.ts.Review at
7789c786— the failure rate counted runs, not deliveriesThe finding is right:
src/views/pages.ts:1704printeddeliveryRate(deliverySummary.postMergeFailures, deliverySummary.merged), andsummariseDeliveriessumspost_merge_failures— failing runs, of which one delivery can carry several (src/lib/deliveryOutcome.ts:74). So merged=1 with two failing runs rendered0% · 0% · 200%, a rate that cannot exist.Fixed the way the finding proposed, at the delivery level.
DeliveryOutcomeSummarygainswithPostMergeFailure— the deliveries whosepost_merge_failures > 0— and the rate prints that overmerged, the same shapewithFollowUpalready had. The run total stayspostMergeFailuresand is unchanged on the row, where "2 failing runs" is a count and is true. The card line also carries the rate asdata-delivery-post-merge-failure-rateso the bound is readable, not only the sentence.Pinned in
test/deliveryOutcome.test.ts: one delivery withpost_merge_failures: 2giveswithPostMergeFailure1,postMergeFailures2, rendersdata-delivery-post-merge-failure-rate="100%", and the test asserts the page never prints200%. At7789c786the same case failed onwithPostMergeFailurebeing undefined.Ran:
npm run checkat this tip (44c04de) — pass,# tests 3001 # pass 3001 # fail 0(lint, typecheck, build included).npm run mock:checkat this tip —mock-snapshot: ok (14 files). It was red at thea9d0648bmerge tip with exactlyrepository.htmlandrepository-demo.html;npm run mock:snapshotrewrote those two only and they are in that merge commit. No latermainmerge,cc60fdf9included, needed a regeneration.npm run test:coverageat this tip, thennode scripts/coverage-diff.mjs --base cc60fdf932dacce252a77552ccf93fc63ef1368f --head HEAD --report coverage.txt --floor 80—coverage-diff: ok — 160/189 changed lines covered (84.7%, floor 80%); over the review commit alone against7789c786it is5/6 changed lines covered (83.3%, floor 80%).npx tsx --test test/deliveryOutcome.test.ts test/retention.test.ts— pass at this tip,# tests 26 # pass 26 # fail 0.node scripts/risk-gate.mjs --base cc60fdf932dacce252a77552ccf93fc63ef1368f—risk-tier: T3; basis: scripts/lib/retention.mjs; over the review commit alone against7789c786—risk-tier: T2; basis: src/lib/deliveryOutcome.ts.node scripts/version.mjs check—version: 0.4.2 ok;node scripts/changelog.mjs check—changelog: 59 fragment(s) ok.Not run: a real GitHub webhook carrying a revert merge; the Docker image; a real FlyBot run.
Risk
The ledger is additive: no existing table or job field changes, and no gate reads it. The only moved surface is the repository's Delivery tab, which gains a card. The revert scan runs after a merge and can fail without affecting the merge path; the row then simply stays un-reverted until the next merge in that repository scans again. Cost and model are copied at merge time, so a later job-cost correction does not rewrite a delivery — that is the append-only choice, and it can under-report a cost that was still settling.