Repository navigation
Revert proof for #6641 (do not merge) - #6643
erikdarlingdata wants to merge 1 commit into
Conversation
d2af72d to
24fa6ed
Compare
24fa6ed to
1b6162b
Compare
| private static string NamedQueriesSentence(IReadOnlyDictionary<string, Fact> facts, string key, string label, Func<RankedObject, string> figure) | ||
| { | ||
| if (!facts.TryGetValue(key, out var f) || f.Ranked.Count == 0) | ||
| if (!facts.TryGetValue(key, out var f) || f.Ranked.Count >= 0) |
There was a problem hiding this comment.
The mutation here is a tautology, and that leaves the case the sentence exists for unproven.
f.Ranked.Count >= 0 is always true, so NamedQueriesSentence returns empty for every fact — both call sites (FactAdvice.cs:1640 QUERY_SPILLS, FactAdvice.cs:1726 PARAMETER_SENSITIVITY) lose their sentence. That is caught: Darling.Tests/FactsNameQueriesAdviceTests.QuerySpillsAdvice_NamesTheSpillingQueries (asserts "Most spilled: Db/0xSB (500 pages (3.91 MB)), Db/0xSA (300 pages (2.34 MB)).") and ParameterSensitivityAdvice_NamesThePlans both fail here. Good.
But because the mutation kills the sentence for every arity, it proves nothing about the boundary. Both of those tests hand the composer a two-entry Ranked list (FactsNameQueriesAdviceTests.cs:30 and :48), and they are the only tests in the tree that assert either sentence (grep for Most spilled / Worst plans finds exactly one assertion). So the weaker mutation
if (!facts.TryGetValue(key, out var f) || f.Ranked.Count > 1) // sentence only at two or more
inverted to f.Ranked.Count < 2 would survive #6641's whole suite — and the one-query case is this method's stated reason for existing. FactRankedPayload.cs:74-78: "a card of one spilling query has no ranked array (it rides at two or more), and get_analysis_facts carries no object_name for any fact"; this method's own doc at FactAdvice.cs:1658-1660: "Unlike NameTheRestList it includes entry [0], because these cards have no other sentence that names the subject." One spilling query is the shape with the least other naming, and nothing fails if its name silently drops out of the card.
A Ranked = [single] arm on QuerySpillsAdvice_NamesTheSpillingQueries closes it, and is cheap: the fixture is already there.
| ArgumentNullException.ThrowIfNull(payload); | ||
| ArgumentNullException.ThrowIfNull(fact); | ||
| if (!NamesItsSubjectInPayload(fact.Key) || string.IsNullOrEmpty(fact.ObjectName)) | ||
| if (!NamesItsSubjectInPayload(fact.Key) || fact.ObjectName is not null || string.IsNullOrEmpty(fact.ObjectName)) |
There was a problem hiding this comment.
Covered, and the boundary is covered too — unlike the advice sentence.
fact.ObjectName is not null || string.IsNullOrEmpty(fact.ObjectName) is unsatisfiable-to-attach: non-null takes the first arm, null takes the second, so AttachSubject always returns the caller's reference and lines 91-94 are dead. The compiler cannot see the tautology (no warning), so this one rides on tests alone, and two of #6641's find it:
FactsNameQueriesAdviceTests.FactPayload_CarriesObjectName_ForTheTwoQueryFactsOnly(:71) —(JsonObject)FactRanked.AttachSubject(card, Spills(), …)gets back the anonymouscard, so the cast throwsInvalidCastException.FactsNameQueriesAdviceTests.FactPayload_AttachSubjectThenAttach_KeepsRankedLast_AndTheSubjectOnTheWire(:61) —json["object_name"]is absent, soAssert.Equal("Db/0xSB", null)fails.
Worth recording for the proof: this site's arity boundary is pinned — :74-75 passes a QUERY_SPILLS fact with an ObjectName and no Ranked at all and asserts the name still lands, and :79 pins the empty-name case to reference identity. That is the one-subject arm the advice sentence at FactAdvice.cs:1663 is missing.
| ObjectName = null, | ||
| Ranked = new(), |
There was a problem hiding this comment.
This is a stamping revert, not a feature revert — say so, because it changes what the proof covers.
spillRanked is still built: the second read at :114-126 runs on every pass that found spills and its result is now discarded. Same at :297-342 for ranked in CollectParameterSensitivityFactsAsync, and in both Darling twins. Neither is a compile error (both locals are written through Add), so CI stays green on the build and the only signal is the assertion failures.
That is the right shape for a mutation proof of the stamping seam, and it is caught: Lite.Tests/FactsNameQueriesTests.QuerySpills_NamesTheSpillingQueries_MostSpillsFirst_WithinTheCap (:83 fact.ObjectName, :86 fact.Ranked), ParameterSensitivity_NamesTheOffenders_WorstFirst_WithinTheCap (:59, :62) and ParameterSensitivity_TiedRatios_AreNamedInKeyOrder (:106-107) all fail, and Darling.Tests/FactsNameQueriesLiveTests fails on the PostgreSQL side (pr.yml :328-331 starts the store and run-ci-leg.ps1:210 exports DARLING_TEST_PG, so those two are not skipped on this PR's CI — the Darling collector half is genuinely proven, not silently skipped).
What it does not prove, and no claim in the body says it does: that a change which drops the read — QuerySpillsRankedSql and the ranked.Count < FactRanked.MaxObjects accumulation — is caught. It is, by the same three assertions, so there is no gap here; just note that the diff leaves live work behind a dead result, which is why the proof must not be merged even by accident.
"Each new test in #6641 should fail here" — three of them passRead blind first, the diff is four mutations in four files: both collectors stop stamping The body's one testable claim is the last line, and taking the body's own framing (#6641 is the PR that added the
The third one is the interesting one, because it is the only The one real coverage gap the proof surfaces is at Two smaller notes on the body:
|
There was a problem hiding this comment.
Verdict: CHANGES REQUESTED
Claims
- VERIFIED — "The Darling and Lite collectors set no name on PARAMETER_SENSITIVITY and QUERY_SPILLS" —
Lite/Analysis/DuckDbFactCollector.QueryPerf.cs:143and:354,Darling/PerformanceMonitor.Darling.Analysis/PgFactCollector.QueryPerf.cs:144and the PARAMETER_SENSITIVITY twin:ObjectName = nullat all four sites. - VERIFIED — "The card text names no queries" —
FactAdvice.cs:1663guards onf.Ranked.Count >= 0, always true, soNamedQueriesSentencereturnsstring.Emptyfor both call sites (:1640,:1726). - VERIFIED — "The fact payload adds no object_name" —
FactRankedPayload.cs:88is a tautology (ObjectName is not null || IsNullOrEmpty(ObjectName)), soAttachSubjectalways returns the caller's reference and:91-94is dead. - VERIFIED — "Rebuilt on 10-24 at 1b6162b, on dev after #6641 merged (c114110)" — the checkout's merge commit is
Merge 1b6162b6331461f19a311bb20343d5119352b4ff into c1141105d58bdecb78b322a8b35d2603d177c033. - VERIFIED — "It undoes four parts of the change" — four mutations in four files; the collectors'
Ranked = new()is the fourth, and it is the one the three bullets below the sentence do not name (they describe a name-only revert, but therankedarray leaves both tools' payloads too). - UNVERIFIED — "The revert is the same four-file change as the first build" — the earlier build is not linked and this checkout holds a single commit, so there is no prior diff to compare against.
- REFUTED — "Each new test in #6641 should fail here" — three of the
#6640-tagged tests pass under the full revert:Lite.Tests/FactsNameQueriesTests.QueryName_ReadsANullPartAsAPlaceholder_NotAnEmptyOne(:110-118, pins the untouchedFactRanked.QueryNameatAnalysisModels.cs:54);Darling.Tests/FactRankedTests.BothToolFiles_AttachTheRankedBlockOncePerTool_AndNeverSpellItAsAProperty(:152-155, reads the unchanged call-site text inMcpAnalysisTools.cs:363/DarlingMcpTools.cs:398);Lite.Tests/StoryNamedHopsTests.MemoryGrantPendingRootConsumingQuerySpills_NamesTheSpillingQueryOnTheRootCard(:226-241, builds its fact withObjectName = "Sales/0xAB"and a two-entryRankedat:231-233, and asserts a sentence composed byFactIdentity.DescribeatFactIdentity.cs:163-164, not by the mutated helper).
Riskiest lines
PerformanceMonitor.Analysis/FactAdvice.cs:1663— the guard is now arity-blind, and the only two tests that assert either sentence both pass a two-entryRanked, so the single-ranked-query case this method exists for (FactRankedPayload.cs:74-78) is unguarded — covered for the all-arity mutation byDarling.Tests/FactsNameQueriesAdviceTests.QuerySpillsAdvice_NamesTheSpillingQueriesandParameterSensitivityAdvice_NamesThePlans; UNCOVERED for aCount < 2variant.PerformanceMonitor.Analysis/FactRankedPayload.cs:88— a tautology the compiler cannot flag, soobject_nameleavesget_analysis_factswith no build signal — covered byDarling.Tests/FactsNameQueriesAdviceTests.FactPayload_CarriesObjectName_ForTheTwoQueryFactsOnly(InvalidCastExceptionon the(JsonObject)cast at:71) andFactPayload_AttachSubjectThenAttach_KeepsRankedLast_AndTheSubjectOnTheWire(:61).Lite/Analysis/DuckDbFactCollector.QueryPerf.cs:143-144(and:354-355, plus both Darling twins) — the ranked read at:114-126still runs and its result is discarded, so the fact ships without subject or ranked list while the work is still paid for — covered byLite.Tests/FactsNameQueriesTests.QuerySpills_NamesTheSpillingQueries_MostSpillsFirst_WithinTheCap(:83,:86),ParameterSensitivity_NamesTheOffenders_WorstFirst_WithinTheCap(:59,:62) andParameterSensitivity_TiedRatios_AreNamedInKeyOrder(:106-107); the PostgreSQL half byDarling.Tests/FactsNameQueriesLiveTests, which runs on this PR (pr.yml:328-331starts the store,run-ci-leg.ps1:210exportsDARLING_TEST_PG).
Traps near this diff
- "Ruling (#5833): tests are load-bearing. … never answer a trap with a test that reads source text or asserts how code is spelled (a signature at a line, a SQL fragment's wording, a comment's text, a trait roster, a coverage percentage)" — applies —
Darling.Tests/FactRankedTests.cs:152-155's#6640assertion counts the literal"FactRanked.Attach(FactRanked.AttachSubject(new"in the tool sources, so it cannot fail on any of these four mutations; it is honestly labelled a source-read parity census at:130-137, so it is not a defect, but it must not be counted toward "each new test fails here". - "Wait facts are keyed on the raw
wait_type… A key in the graph is not a key in the data; grep the collector that emits it" — applies — after this diff no collector stampsObjectNameon either query fact, soFactIdentity.SqlServerKeys' two#6640entries (FactIdentity.cs:87-88) and theirDescribearms (:163-166) return null against a real store while the roster test atStoryNamedHopsTests.cs:36and the hop test at:226stay green on hand-built facts — the exact dormant-arm shape, here deliberately and temporarily. - "Lite and Darling ship in parity (CONTRIBUTING's two-store rule): a fix to one SKU's reader, tool, alert or description without the counterpart's is a finding" — does not apply — the two collector files are mutated symmetrically (
DuckDbFactCollector.QueryPerf.cs:143/:354againstPgFactCollector.QueryPerf.cs:144and its PARAMETER_SENSITIVITY twin), and the other two files are the sharedPerformanceMonitor.Analysisproject. - "
FactAdvice.PopulateStoryTextruns BEFOREClusterIntoIncidents/StampClusters/Reconcile… so a sentence added after folding must re-read the frozenStoryTextblob" — does not apply — no sentence is added;FactAdvice.cs:1663only suppresses one that already composed insideCompose, upstream ofPopulateStoryText. - "
.gitattributessaystext=auto eol=crlf… A diff that touches every line of a file is a line-ending conversion until proven otherwise" — does not apply — 10 additions and 10 deletions across four files, each a single-line replacement.
Divergence (blind read vs PR body)
Two places. First, the body claims every new #6641 test fails here; three pass, and the one that matters is StoryNamedHopsTests.MemoryGrantPendingRootConsumingQuerySpills_…, the only #6640 test that walks collector→card, which short-circuits that walk by hand-building the fact at :231-233. The collector stamping is still genuinely proven, by FactsNameQueriesTests and the live Darling pair — but a reader who takes the body at its word would credit the wrong test. Second, the body's three bullets describe a name-only revert while the diff also blanks Ranked at all four collector sites, which is what removes the ranked array from get_analysis_facts and analyze_server; the sentence above the bullets says "four parts", so the count is right and the roster is short by one.
Reviewer checklist
- Run this PR's CI and confirm the failure roster is exactly the seven expected tests (three
FactsNameQueriesTests, twoFactsNameQueriesAdviceTestspayload, twoFactsNameQueriesAdviceTestsadvice) plus the twoFactsNameQueriesLiveTests, withQueryName_ReadsANullPartAsAPlaceholder_NotAnEmptyOne,BothToolFiles_AttachTheRankedBlockOncePerTool_…andMemoryGrantPendingRootConsumingQuerySpills_…green — the read-only tree can predict that list but not produce it. - Confirm the
testjob'splangave this PR adarling_scopeother thannone, since that is what starts the PostgreSQL store the two live tests need; a plan that skipped it would turn the Darling collector half from "failed" into "never ran". - On dev after this PR is closed, add the single-ranked arm to
QuerySpillsAdvice_NamesTheSpillingQueriesso the one-query card's name is held by a test, then re-run this same mutation withf.Ranked.Count < 2to confirm it now fails.
|
Claude posting for Erik Darling. Proof done: 9 of the 11 new tests failed with the change undone, and the results are on #6641. Closing as planned. |
Claude posting for Erik Darling.
Revert proof for #6641. Do not merge. This PR is closed after its CI run.
Rebuilt on 10-24 at 1b6162b, on dev after #6641 merged (c114110), so it runs #6641's final tests. The revert is the same four-file change as the first build.
It undoes four parts of the change, and keeps the tests:
Each new test in #6641 should fail here.