diff --git a/.abcd/development/brief/04-surfaces/18-ideate.md b/.abcd/development/brief/04-surfaces/18-ideate.md index d9edc82e3b..762cf7ffee 100644 --- a/.abcd/development/brief/04-surfaces/18-ideate.md +++ b/.abcd/development/brief/04-surfaces/18-ideate.md @@ -42,15 +42,21 @@ never decides whether an idea is any good. | # | Leg | What it produces | |---|---|---| -| 1 | Primary-source research | A claims table: each load-bearing claim, the **primary** source it was checked against, and the finding (`verified` / `falsified` / `unverifiable`) | -| 2 | Record grill | Hits on the existing record, each cited by a record id the binary proves resolves, carrying a relation (`covered` / `contradicted` / `superseded`) | -| 3 | Adversarial review | Kill attempts, each with an outcome (`survived` / `partial` / `fatal`) | +| 1 | Primary-source research | A claims table: each load-bearing claim, the **primary** source it was checked against (an `https://` URL, a filed `source:`, or a record id), and the finding (`verified` / `quoted` / `falsified` / `unverifiable`) | +| 2 | Record grill | Hits on the existing record, each cited by a record id the binary proves resolves, carrying a relation (`covered` / `contradicted` / `superseded`), and the searches run (terms, places, and how many results came back) | +| 3 | Adversarial review | One section per reviewer, under its own label: its kill attempts, each with an outcome (`survived` / `partial` / `fatal`), its own verdict, and the sources it consulted; where the reviewers' verdicts differ, the ruling that settled the split (who ruled, the verdict, the reason) | The order is validated rather than assumed: the legs travel as an ordered array, and a payload whose legs are missing, reordered, or duplicated is refused. Each leg changes what the next one is looking at, so running them out of order produces a different and weaker result. +The verdict also says where the idea came from (`origin_sources`). A claim +checked only against one of those sources is `quoted`, not `verified`: an idea +agreeing with its own source is not evidence. A `reframed` verdict carries the +idea's surviving shape in `reframing`, because that, not the original wording, +is what may graduate. + Only records carry citable ids, and leg 2's citations are held to that: the brief, the principles, the research notes and the lines of a repository's frozen decisions log have no id to cite, so a hit on one of those rides in the @@ -63,7 +69,9 @@ an artefact whose provenance it does not know. That is the evaluator-outside-the-loop principle applied to ideas, and it is the one measured debiasing effect the protocol rests on. The binary cannot observe how an agent was run, so this obligation lives in the command page's orchestration script, which -instructs the host to strip authorship and prior-leg framing before the hand-off. +instructs the host to strip authorship and prior-leg framing before the hand-off, +and to give each reviewer exactly one idea in its context: a reviewer that has +already judged another idea compares the two instead of attacking this one. ## Behaviour @@ -129,21 +137,32 @@ The refusals a caller actually meets fall into six groups, each protecting something a later reader depends on: - **The payload cannot be tied to a definition**: no `schema_version`, one this - build does not support, or no semver `prompt_version`. A verdict is the output + build does not support, or no semver `prompt_version`. A version-1 payload is + refused with a message naming version 2, never upgraded: version 1 had nowhere + to put a search, a second reviewer or a reframing. A verdict is the output of a named prompt at a named version, and an unstamped payload cannot be traced back to what produced it. A key the contract does not declare refuses the document too; the refusal names it redacted through the canonical scanner, and describes it when the scanner cannot be trusted. -- **The evidence does not hold up**: a cited record id that does not resolve, a - cited value that is not a record id at all, a claim naming no primary source, an - out-of-enum verdict or outcome. Every closed set is closed, and an unregistered +- **The evidence does not hold up**: a record id that does not resolve, written + outside a web address in a claim or its primary source, a grill note, a kill + attempt, a reviewer's label or sources, the ruling, a rejected alternative, an + origin source or the reframing (the refusal names where it was written; the + grill's search terms and places are not checked, because a searched-for id + that names nothing is the finding), a cited value that is not a record id + at all, a claim naming no primary source or one of no known shape, a + `source:` key the sources corpus does not carry (where a corpus exists), a + grill with no hits that records no searches, a `verified` claim whose source is + one of the idea's own origin sources, an out-of-enum verdict or outcome. Every closed set is closed, and an unregistered value refuses the document rather than being coerced. The refusal names the field and describes the value by its length, never quoting it: closed-set members are not redacted on the way in, so a quote would carry whatever was pasted there into the terminal and the transcript. - **The record would mislead a later reader**: an empty `rejected_alternatives` list with no explicit marker saying nothing was weighed, because silence and - "nothing was weighed" read identically; or a verdict record that already exists + "nothing was weighed" read identically; a split between reviewers with no + ruling, or a `reframed` verdict with no reframing, because each leaves the + reader to guess what was decided; or a verdict record that already exists for this slug and date, because overwriting erases a recorded reason. - **The store is not answered**: neither answer, or both. The product thinker's answer decides who can ever read the verdict, so the verb never @@ -176,8 +195,9 @@ the three host legs have already been paid for. The create is exclusive, which i the no-overwrite guarantee. The pointer is a new file of its own, so there is no shared file to read, modify and write, and no lock beyond the decision writer's own mint lock. If the decision cannot be recorded after the verdict -landed, the verdict is removed again, so a run never leaves a verdict nothing -points at. +landed, the verdict is removed again, so a run does not leave a verdict nothing +points at; if that removal fails too, the refusal says so and names the file to +remove by hand. ## References diff --git a/.abcd/development/intents/planned/itd-2610141646454347-an-ideate-verdict-proves-its-own-legs.md b/.abcd/development/intents/planned/itd-2610141646454347-an-ideate-verdict-proves-its-own-legs.md deleted file mode 100644 index 5dbbe393f9..0000000000 --- a/.abcd/development/intents/planned/itd-2610141646454347-an-ideate-verdict-proves-its-own-legs.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -id: itd-2610141646454347 -slug: an-ideate-verdict-proves-its-own-legs -spec_id: spc-2610142138047122 -kind: standalone -bundle: null -suggested_kind: null -reclassification_history: - - { date: 2026-10-14, from: bundle-member, to: standalone, reason: "The product thinker ruled in the planning interview that this fix ships as soon as it is ready, outside bundle ideate-librarian: it needs nothing from the librarian, and its major issue blocks every release cut while it is open." } -builds_on: [] -related_issues: [iss-2610141647304982, iss-2610141647307807, iss-2610141647308358, iss-2610141647315880, iss-2610141647313018] -severity: major -impact: breaking -origin: researcher-authored -production_mode: hand-written -target_release: next ---- - -# An ideate verdict proves its own legs - -## Press Release - -> **An ideate verdict proves every leg it records, not just the record grill.** -> Every record id in every leg resolves, and a primary source must be a web -> address, a filed source or a record. The grill says what it searched and -> where, so an empty grill can be told from a broken search. A reframing has its -> own field. Two reviewers are recorded reviewer by reviewer, and a split -> between them carries the ruling that settled it. A claim checked only against -> the idea's own source reads "quoted", never "verified". -> -> "I read a verdict that killed an idea," said Bob, a staff engineer. "I could -> see which searches came back empty, that both reviewers agreed, and that the -> claim it leaned on was only checked as quoted. I trusted the kill because I -> could check it." - -## Why This Matters - -`abcd ideate record` checks citations only in the grill leg. A lab hand-run of -2026-10-14 recorded, with exit 0, a verdict that cited a non-existent intent in -leg 3 and another in leg 1 and named "a remark heard in passing" as a verified -primary source (iss-2610141647304982). The same lab found that a broken grill -and an empty one write the same record (iss-2610141647307807); that two -reviewers and a disagreement between them have no place in the record -(iss-2610141647308358, iss-2610141647315880); and that "verified" cannot say -"verified as quoted" (iss-2610141647313018). Each is a missing field or a -missing check, and the binary already checks the grill leg's ids, so the same -kind of check extends to the rest. - -Breaking, for anything that records a verdict: a payload with an unresolved id -in any leg, a free-text primary source, a reframed verdict without its -reframing, or schema version 1 is refused where it was accepted. - -## What's In Scope - -- Schema version 2 of the ideate verdict, its validation, and its rendering. -- The command page `commands/ideate.md` updated to the new shape in the same - change. - -## What's Out of Scope - -- Private verdicts and citing them (itd-2610141646456128). -- Ideate starting from a source, and citing pack items (itd-2610141646452213). -- Judging any claim: the binary checks shapes and citations, never the truth of - a claim. - -## Mechanism - -We expect no verdict recorded after this change to carry an unresolved id, an -unlabelled self-quotation or an unrecorded split, because each is refused at -record time by a check on the payload's shape and citations. Shown wrong if a -version 2 verdict on record carries an id that does not resolve, or a claim -marked verified whose only primary source is one of the idea's own origin -sources. - -## Scope Conditions - -- The writer accepts version 2 only; a version 1 payload is refused with a - message naming version 2. Verdicts already on record are rendered Markdown - and are not parsed or backfilled. -- The command page and the binary ship together; a host reading an older page - gets the version-1 refusal message. -- The binary checks shapes and citations. Whether a claim is true, or how a - reviewer reached a verdict, is the host's and the person's judgement. - -## Acceptance Criteria - -- **Given** a verdict citing a record id that does not resolve, in any leg, - **when** it is recorded, **then** it is refused, naming the leg and the id. -- **Given** a claim whose primary source is not a web address, a filed source - key or a record id, **when** the verdict is recorded, **then** it is refused. -- **Given** a grill leg with no hits and no search record (the terms searched, - the places searched, and the count each returned), **when** the verdict is - recorded, **then** it is refused. -- **Given** a verdict marked reframed with no reframing field, **when** it is - recorded, **then** it is refused. -- **Given** two reviewers, **when** the verdict is recorded, **then** each - reviewer's kill attempts, verdict and consulted sources are kept under that - reviewer. -- **Given** two reviewers whose verdicts differ, **when** the verdict is - recorded without a ruling (who ruled, and the ruling), **then** it is - refused. -- **Given** a claim marked verified whose only primary source is listed among - the verdict's origin sources, **when** it is recorded, **then** it is - refused; marked quoted, it is accepted. -- **Given** a version 1 payload, **when** it is recorded, **then** it is - refused with a message naming version 2. -- **Given** the updated command page, **when** it is read, **then** it tells the - runner to give each reviewer one idea per context. - -## Prior Art - -- itd-104 (shipped): the ideate admission gauntlet and `ideate record`, whose - grill-leg citation check this extends to every leg. -- itd-1 (discipline): every audit verifies acceptance criteria; the verdict's - own legs get the same treatment. -- iss-2608230748418054 (open): the grill's structural blind spot over the - research notes, which the search record makes visible. - -## SOTA - -- Structured-output validation of model-produced records (JSON Schema - validators, Pydantic-style models): mature; validates shape, not truth. -- Citation resolution in research tooling (DOI and URL resolvers): mature for - external references; abcd's record ids need abcd's own resolver. - -**Path:** 2. A native floor: abcd's own resolver and schema check, since the ids -are abcd's own; an external validator adds nothing the Go check lacks. - -## Open Questions - -_None._ - -## Audit Notes - -_Empty. Populated by intent-auditor when intent moves to shipped/._ - -## Grounds - -- pursued: an ideate verdict on record today can cite things that do not exist and still pass; we expect that after this change no verdict on record carries an unresolved citation or an unlabelled self-quotation, and the fix also unblocks release cuts, which refuse while its serious issue is open. Shown wrong if a verdict recorded after it still carries one. diff --git a/.abcd/development/intents/shipped/itd-2610141646454347-an-ideate-verdict-proves-its-own-legs.md b/.abcd/development/intents/shipped/itd-2610141646454347-an-ideate-verdict-proves-its-own-legs.md new file mode 100644 index 0000000000..f19c57e764 --- /dev/null +++ b/.abcd/development/intents/shipped/itd-2610141646454347-an-ideate-verdict-proves-its-own-legs.md @@ -0,0 +1,233 @@ +--- +id: itd-2610141646454347 +slug: an-ideate-verdict-proves-its-own-legs +spec_id: spc-2610142138047122 +kind: standalone +bundle: null +suggested_kind: null +reclassification_history: + - { date: 2026-10-14, from: bundle-member, to: standalone, reason: "The product thinker ruled in the planning interview that this fix ships as soon as it is ready, outside bundle ideate-librarian: it needs nothing from the librarian, and its major issue blocks every release cut while it is open." } +builds_on: [] +related_issues: [iss-2610141647304982, iss-2610141647307807, iss-2610141647308358, iss-2610141647315880, iss-2610141647313018] +severity: major +impact: breaking +origin: researcher-authored +production_mode: hand-written +--- + +# An ideate verdict proves its own legs + +## Press Release + +> **An ideate verdict proves every leg it records, not just the record grill.** +> Every record id in every leg resolves, and a primary source must be a web +> address, a filed source or a record. The grill says what it searched and +> where, so an empty grill can be told from a broken search. A reframing has its +> own field. Two reviewers are recorded reviewer by reviewer, and a split +> between them carries the ruling that settled it. A claim checked only against +> the idea's own source reads "quoted", never "verified". +> +> "I read a verdict that killed an idea," said Bob, a staff engineer. "I could +> see which searches came back empty, that both reviewers agreed, and that the +> claim it leaned on was only checked as quoted. I trusted the kill because I +> could check it." + +## Why This Matters + +`abcd ideate record` checks citations only in the grill leg. A lab hand-run of +2026-10-14 recorded, with exit 0, a verdict that cited a non-existent intent in +leg 3 and another in leg 1 and named "a remark heard in passing" as a verified +primary source (iss-2610141647304982). The same lab found that a broken grill +and an empty one write the same record (iss-2610141647307807); that two +reviewers and a disagreement between them have no place in the record +(iss-2610141647308358, iss-2610141647315880); and that "verified" cannot say +"verified as quoted" (iss-2610141647313018). Each is a missing field or a +missing check, and the binary already checks the grill leg's ids, so the same +kind of check extends to the rest. + +Breaking, for anything that records a verdict: a payload with an unresolved id +in any leg, a free-text primary source, a reframed verdict without its +reframing, or schema version 1 is refused where it was accepted. + +## What's In Scope + +- Schema version 2 of the ideate verdict, its validation, and its rendering. +- The command page `commands/ideate.md` updated to the new shape in the same + change. + +## What's Out of Scope + +- Private verdicts and citing them (itd-2610141646456128). +- Ideate starting from a source, and citing pack items (itd-2610141646452213). +- Judging any claim: the binary checks shapes and citations, never the truth of + a claim. + +## Mechanism + +We expect no verdict recorded after this change to carry an unresolved id, an +unlabelled self-quotation or an unrecorded split, because each is refused at +record time by a check on the payload's shape and citations. Shown wrong if a +version 2 verdict on record carries an id that does not resolve, or a claim +marked verified whose only primary source is one of the idea's own origin +sources. + +## Scope Conditions + +- The writer accepts version 2 only; a version 1 payload is refused with a + message naming version 2. Verdicts already on record are rendered Markdown + and are not parsed or backfilled. +- The command page and the binary ship together; a host reading an older page + gets the version-1 refusal message. +- The binary checks shapes and citations. Whether a claim is true, or how a + reviewer reached a verdict, is the host's and the person's judgement. + +## Acceptance Criteria + +- **Given** a verdict citing a record id that does not resolve, in any leg, + **when** it is recorded, **then** it is refused, naming the leg and the id. +- **Given** a claim whose primary source is not a web address, a filed source + key or a record id, **when** the verdict is recorded, **then** it is refused. +- **Given** a grill leg with no hits and no search record (the terms searched, + the places searched, and the count each returned), **when** the verdict is + recorded, **then** it is refused. +- **Given** a verdict marked reframed with no reframing field, **when** it is + recorded, **then** it is refused. +- **Given** two reviewers, **when** the verdict is recorded, **then** each + reviewer's kill attempts, verdict and consulted sources are kept under that + reviewer. +- **Given** two reviewers whose verdicts differ, **when** the verdict is + recorded without a ruling (who ruled, and the ruling), **then** it is + refused. +- **Given** a claim marked verified whose only primary source is listed among + the verdict's origin sources, **when** it is recorded, **then** it is + refused; marked quoted, it is accepted. +- **Given** a version 1 payload, **when** it is recorded, **then** it is + refused with a message naming version 2. +- **Given** the updated command page, **when** it is read, **then** it tells the + runner to give each reviewer one idea per context. + +## Prior Art + +- itd-104 (shipped): the ideate admission gauntlet and `ideate record`, whose + grill-leg citation check this extends to every leg. +- itd-1 (discipline): every audit verifies acceptance criteria; the verdict's + own legs get the same treatment. +- iss-2608230748418054 (open): the grill's structural blind spot over the + research notes, which the search record makes visible. + +## SOTA + +- Structured-output validation of model-produced records (JSON Schema + validators, Pydantic-style models): mature; validates shape, not truth. +- Citation resolution in research tooling (DOI and URL resolvers): mature for + external references; abcd's record ids need abcd's own resolver. + +**Path:** 2. A native floor: abcd's own resolver and schema check, since the ids +are abcd's own; an external validator adds nothing the Go check lacks. + +## Open Questions + +_None._ + +## Audit Notes + + +Fidelity review — receipt rcp-dbe525f06247 (verifier intent-auditor claude-opus-5-5). + +Provenance: intent-auditor@claude-opus-5-5 · rubric_hash sha256:effa65b3e9e88ff29433b443ec2be159522a8b0b71cf1434526514aa61edb13e · prompt_hash sha256:4685a5a288619851380647e707ec06360e84954083144001c915d57ba4776be2 +Input attestations: diff:c66f9959f..9c77aae4f@sha256:4b98c145c87cfc93bf26e7c5a45b7d7c43d64d5cb7016abb7a79d01464953c07; diff:9c77aae4f..a5629f871@sha256:dc5d3e380f731baeca83aea1099ffd186462719a598cd33558337d9eb1a0b0fb; + +Acceptance rollup: MET 9 · MET_WITH_CONCERNS 0 · NOT_MET 0 · INCONCLUSIVE 0 + +Per-criterion verdicts: +- ac-1 — MET: resolveCitations scans claims, primary sources, grill notes, kill attempts, reviewer sources, rejected alternatives, origins and the reframing, and the fix 9c77aae4f..a5629f871 adds reviewer labels and the ruling's 'by' beside its reason; CitationError names each part and id, tested per field including the two new cases. The grill's search terms and places stay unchecked by documented design: a search term records what was searched for, not a citation, so it falls outside 'citing a record id' + evidence: ac1-fix.diff:35 — "+ add(adversarial, r.Label)" + evidence: ac1-fix.diff:43 — "+ add(adversarial, v.ruling.By, v.ruling.Reason)" + evidence: ac1-fix.diff:55 — ""reviewer label": {func(p map[string]any) { reviewer(p, 0)["label"] = "the reviewer of an unfiled id" }, "adversarial-review", "an unfiled id"}," + evidence: ac1-fix.diff:57 — "leg(p, 2)["ruling"] = map[string]any{"by": "the advisor named in an unfiled id", "verdict": "survives", "reason": "both agree"}" + evidence: internal/core/ideate/record.go:1006 — "for _, c := range v.claims { add(research, c.Claim, c.PrimarySource)" + evidence: internal/core/ideate/record.go:100 — "parts[i] = c.Where + " cites " + c.ID" + evidence: internal/core/ideate/record.go:990 — "nor are the grill's search // terms and places, where a searched-for id that names nothing is the finding." + evidence: internal/core/ideate/v2_test.go:38 — "func TestV2RefusesAnUnresolvedIDInAnyLeg(t *testing.T) {" +- ac-2 — MET: validateResearch refuses a primary source that sourceShaped rejects; sourceShaped accepts only an https:// address with a host, source:< key> in the corpus key grammar, or a record id, and a key is resolved against the default corpus when one exists; tested with 'a remark heard in passing', file://, http:// and malformed keys + evidence: internal/core/ideate/record.go:572 — "if !sourceShaped(src) {" + evidence: internal/core/ideate/record.go:600 — "func sourceShaped(src string) bool {" + evidence: internal/core/ideate/record.go:1194 — "the verdict cites sources the sources corpus does not carry — nothing was written" + evidence: internal/core/ideate/v2_test.go:96 — "func TestV2RefusesAPrimarySourceOfNoKnownShape(t *testing.T) {" + evidence: internal/surface/cli/ideate.go:190 — "if dir, err := source.DefaultDir(); err == nil {" +- ac-3 — MET: validateGrill refuses no hits together with no searches, and validateSearches requires each search's terms, places and a non-nil, non-negative count; tested, including the accepted empty grill with a recorded search + evidence: internal/core/ideate/record.go:685 — "if len(leg.Hits) == 0 && len(leg.Searches) == 0 {" + evidence: internal/core/ideate/record.go:738 — "if s.Count == nil {" + evidence: internal/core/ideate/v2_test.go:188 — "func TestV2RefusesAGrillWithNoHitsAndNoSearches(t *testing.T) {" +- ac-4 — MET: validateReframing refuses a reframed verdict whose cleaned reframing is empty (absent or blank), and refuses a reframing beside any other verdict; tested both ways + evidence: internal/core/ideate/record.go:970 — "case p.Verdict == VerdictReframed && reframing == "":" + evidence: internal/core/ideate/v2_test.go:239 — "func TestV2RefusesAReframedVerdictWithoutAReframing(t *testing.T) {" +- ac-5 — MET: the Reviewer type holds label, kill_attempts, verdict and sources; Leg carries no top-level kill attempts; the renderer writes one '### Reviewer < label> — < verdict>' section holding that reviewer's attempts and sources; the test asserts each reviewer's attempt and source appear only in its own section + evidence: internal/core/ideate/ideate.go:296 — "type Reviewer struct {" + evidence: internal/core/ideate/render.go:134 — "fmt.Fprintf(b, "### Reviewer %s — %s\n\n", r.Label, r.Verdict)" + evidence: internal/core/ideate/v2_test.go:277 — "func TestV2KeepsEachReviewerUnderItsLabel(t *testing.T) {" +- ac-6 — MET: validateRuling refuses differing reviewer verdicts with no ruling, and a given ruling must name who ruled, a valid verdict and a reason; tested, including the rendered ruling + evidence: internal/core/ideate/record.go:880 — "if in == nil { for _, r := range reviewers[1:] { if r.Verdict != reviewers[0].Verdict {" + evidence: internal/core/ideate/record.go:898 — "case out.By == "":" + evidence: internal/core/ideate/v2_test.go:321 — "func TestV2RefusesASplitWithoutARuling(t *testing.T) {" +- ac-7 — MET: validateResearch refuses a verified claim whose normalised primary source is among the origin keys, and quoted is a registered status; tests cover alternate spellings, an origin with surrounding words, an omitted origin_sources list (refused) and an accepted quoted claim + evidence: internal/core/ideate/record.go:580 — "if c.Status == ClaimVerified && origin[sourceKey(src)] {" + evidence: internal/core/ideate/ideate.go:200 — "ClaimQuoted ClaimStatus = "quoted"" + evidence: internal/core/ideate/v2_test.go:367 — "func TestV2RefusesAVerifiedSelfQuotation(t *testing.T) {" +- ac-8 — MET: SchemaVersion is 2; decodePayload peeks schema_version before the strict decode and refuses an older version with olderSchemaError, which names version 2; tested in the core on a real v1-shaped document and at the CLI (exit 2, 'version 2') + evidence: internal/core/ideate/ideate.go:34 — "const SchemaVersion = 2" + evidence: internal/core/ideate/record.go:377 — "if json.Unmarshal(raw, &peek) == nil && peek.SchemaVersion > 0 && peek.SchemaVersion < SchemaVersion {" + evidence: internal/core/ideate/record.go:413 — "compose it as version %d, which adds origin_sources" + evidence: internal/core/ideate/v2_test.go:447 — "func TestV2RefusesAVersionOnePayload(t *testing.T) {" + evidence: internal/surface/cli/ideate_surface_test.go:158 — ""version-1-payload"," +- ac-9 — MET: the command page's leg 3 lists 'One idea per context' among the leg's properties, and a test reads the leg 3 section of the page for the phrase + evidence: commands/ideate.md:128 — "- **One idea per context.** Give each reviewer exactly one idea in its context," + evidence: internal/core/ideate/v2_test.go:488 — "func TestCommandPageGivesEachReviewerOneIdeaPerContext(t *testing.T) {" + +Gap audit: +- honoured: + - a primary source must be a web address, a filed source or a record + evidence: internal/core/ideate/record.go:600 — "func sourceShaped(src string) bool {" + - the grill says what it searched and where, so an empty grill can be told from a broken search + evidence: internal/core/ideate/record.go:685 — "if len(leg.Hits) == 0 && len(leg.Searches) == 0 {" + evidence: internal/core/ideate/render.go:115 — "b.WriteString("| Terms | Places | Count |\n|---|---|---|\n")" + - a reframing has its own field + evidence: internal/core/ideate/ideate.go:349 — "Reframing string `json:"reframing,omitempty"`" + evidence: internal/core/ideate/render.go:186 — "b.WriteString("## The reframing\n\n")" + - two reviewers are recorded reviewer by reviewer, and a split carries the ruling that settled it + evidence: internal/core/ideate/render.go:134 — "fmt.Fprintf(b, "### Reviewer %s — %s\n\n", r.Label, r.Verdict)" + evidence: internal/core/ideate/record.go:883 — "the reviewers' verdicts differ (%s) and the adversarial-review leg records no ruling — nothing was written" + - a claim checked only against the idea's own source reads quoted, never verified + evidence: internal/core/ideate/record.go:580 — "if c.Status == ClaimVerified && origin[sourceKey(src)] {" + - the command page is updated to the version 2 shape in the same change, and its example decodes as this binary's payload + evidence: commands/ideate.md:154 — ""schema_version": 2," + evidence: internal/core/ideate/v2_test.go:510 — "if err := dec.Decode(&p); err != nil {" +- diverged: + - every record id in every leg resolves: delivered over every prose, label and source field (reviewer labels and the ruling's 'by' added in a5629f871), with the grill's search terms and places exempt by design, as the brief, the command page and the code comment all state + evidence: internal/core/ideate/record.go:994 — "func citationPlaces(v verdictDoc) []citedText {" + evidence: ac1-fix.diff:25 — "part of the address). A search term naming an id is not checked, because" + evidence: .abcd/development/brief/04-surfaces/18-ideate.md:151 — "grill's search terms and places are not checked, because a searched-for id" + - a primary source may be a web address: delivered as https:// only, so an http:// address is refused + evidence: internal/core/ideate/record.go:602 — "case strings.HasPrefix(src, "https://"):" + evidence: internal/core/ideate/v2_test.go:100 — ""http://example.invalid/plain"," + - a filed source key is proved filed: only where a sources corpus exists; with none, a well-formed key is accepted on its shape alone + evidence: internal/core/ideate/record.go:1174 — "case errors.Is(err, source.ErrNoCorpus): return nil" +- missing: (none) + +Scope-condition dispositions: +- cond-2610142138048878 — survived: the writer refuses every version below 2 with a message naming version 2, and the ideate package has no non-test file read: existing verdict records are only ever created exclusively, never parsed or backfilled + evidence: internal/core/ideate/record.go:397 — "case p.SchemaVersion != SchemaVersion: return Payload{}, olderSchemaError(p.SchemaVersion)" + evidence: internal/core/ideate/record.go:243 — "if err := fsutil.CreateExclusiveIn(root, rel, []byte(body), 0o644); err != nil {" +- cond-2610142138047607 — survived: the command page and the binary change in the same delivered range, a test ties the page's example to the binary's payload type and version, and a v1 payload from an older page gets the refusal naming version 2 and pointing at the command page, at the core and at the CLI + evidence: internal/core/ideate/v2_test.go:513 — "if p.SchemaVersion != SchemaVersion {" + evidence: internal/core/ideate/record.go:414 — "and a reframing for a reframed verdict (the ideate command page carries the shape)" + evidence: internal/surface/cli/ideate_surface_test.go:158 — ""version-1-payload"," +- cond-2610142138040936 — survived: every new check is a shape, enum, presence, string-comparison or citation-resolution check; no source is fetched, a ruling is never checked against the overall verdict, and how a reviewer was run is left to the command page + evidence: internal/core/ideate/record.go:596 — "The // address is not fetched and the record id is resolved with every other // citation (resolveCitations); this is the shape alone" + evidence: internal/core/ideate/record.go:876 — "Whether the ruling was // right is the person's judgement, and the binary never checks it against the // verdict the whole gauntlet reached." + evidence: internal/core/ideate/record.go:787 — "conduct is the orchestrating prompt's obligation — the binary cannot observe // how an agent was run" + + +## Grounds + +- pursued: an ideate verdict on record today can cite things that do not exist and still pass; we expect that after this change no verdict on record carries an unresolved citation or an unlabelled self-quotation, and the fix also unblocks release cuts, which refuse while its serious issue is open. Shown wrong if a verdict recorded after it still carries one. diff --git a/.abcd/development/specs/open/spc-2610142138047122-an-ideate-verdict-proves-its-own-legs.md b/.abcd/development/specs/closed/spc-2610142138047122-an-ideate-verdict-proves-its-own-legs.md similarity index 100% rename from .abcd/development/specs/open/spc-2610142138047122-an-ideate-verdict-proves-its-own-legs.md rename to .abcd/development/specs/closed/spc-2610142138047122-an-ideate-verdict-proves-its-own-legs.md diff --git a/.abcd/work/issues/open/iss-2610141647304982-ideate-record-validates-citations-only.md b/.abcd/work/issues/resolved/iss-2610141647304982-ideate-record-validates-citations-only.md similarity index 75% rename from .abcd/work/issues/open/iss-2610141647304982-ideate-record-validates-citations-only.md rename to .abcd/work/issues/resolved/iss-2610141647304982-ideate-record-validates-citations-only.md index b440b7d6bf..1d3c726b6b 100644 --- a/.abcd/work/issues/open/iss-2610141647304982-ideate-record-validates-citations-only.md +++ b/.abcd/work/issues/resolved/iss-2610141647304982-ideate-record-validates-citations-only.md @@ -11,6 +11,12 @@ production_mode: hand-written found_at: "internal/core/ideate/record.go" remedy: "Deliver itd-2610141646454347: every record id in every leg is resolved, and a primary source must be a URL, a filed source key or a record id." related_intents: [itd-2610141646454347] +resolution: "a version-2 verdict resolves every record id in every leg, naming the leg, and a primary source must be an https URL, a filed source key or a record id" +impact: breaking +resolved_by: + intent: "itd-2610141646454347" + spec: "spc-2610142138047122" + commit: "81baf3f6a" --- ideate record validates citations only in the record-grill leg: a verdict citing a non-existent intent id in leg 3 and another in leg 1, with 'a remark heard in passing' as a verified primary source, was recorded with exit 0; in the real run a surviving kill attempt rested on a draft presented as shipped (lab finding F-7). diff --git a/.abcd/work/issues/open/iss-2610141647307807-a-grill-whose-search-silently-failed.md b/.abcd/work/issues/resolved/iss-2610141647307807-a-grill-whose-search-silently-failed.md similarity index 78% rename from .abcd/work/issues/open/iss-2610141647307807-a-grill-whose-search-silently-failed.md rename to .abcd/work/issues/resolved/iss-2610141647307807-a-grill-whose-search-silently-failed.md index e2884e9e61..80d3797468 100644 --- a/.abcd/work/issues/open/iss-2610141647307807-a-grill-whose-search-silently-failed.md +++ b/.abcd/work/issues/resolved/iss-2610141647307807-a-grill-whose-search-silently-failed.md @@ -11,6 +11,12 @@ production_mode: hand-written found_at: "internal/core/ideate/ideate.go" remedy: "Deliver itd-2610141646454347: the grill leg records each search (terms, paths, hit counts), and an empty grill without that trace is refused." related_intents: [itd-2610141646454347] +resolution: "the grill leg records each search as terms, places and count, and a grill with no hits and no searches is refused" +impact: breaking +resolved_by: + intent: "itd-2610141646454347" + spec: "spc-2610142138047122" + commit: "81baf3f6a" --- A grill whose search silently failed writes the same ideate record as an honest empty grill: a search loop that zsh did not word-split returned zero hits for every term (90 files mention one of them), and the verdict schema carries hits only, with nothing saying what was searched or where (lab finding F-4). diff --git a/.abcd/work/issues/open/iss-2610141647308358-ideate-s-leg-3-asks-for-one-fresh.md b/.abcd/work/issues/resolved/iss-2610141647308358-ideate-s-leg-3-asks-for-one-fresh.md similarity index 75% rename from .abcd/work/issues/open/iss-2610141647308358-ideate-s-leg-3-asks-for-one-fresh.md rename to .abcd/work/issues/resolved/iss-2610141647308358-ideate-s-leg-3-asks-for-one-fresh.md index 155db544b8..4f85f89df4 100644 --- a/.abcd/work/issues/open/iss-2610141647308358-ideate-s-leg-3-asks-for-one-fresh.md +++ b/.abcd/work/issues/resolved/iss-2610141647308358-ideate-s-leg-3-asks-for-one-fresh.md @@ -11,6 +11,12 @@ production_mode: hand-written found_at: "commands/ideate.md" remedy: "Deliver itd-2610141646454347: the verdict records a reviewer pair per reviewer, ideas reach reviewers one per context, and each reviewer's own research is recorded." related_intents: [itd-2610141646454347] +resolution: "each reviewer is recorded under its own label with its own kill attempts, verdict and consulted sources, and the command page gives each reviewer one idea per context" +impact: breaking +resolved_by: + intent: "itd-2610141646454347" + spec: "spc-2610142138047122" + commit: "81baf3f6a" --- ideate's leg 3 asks for one fresh evaluator, which conflicts with the standing Opus+Sonnet review-pair rule under the machine-wide sub-agent cap; fitting the pair into the cap meant batching several ideas into one evaluator context, and one evaluator made a single tool call and never opened the repository it judged (lab finding F-6). diff --git a/.abcd/work/issues/open/iss-2610141647313018-ideate-s-claim-status-cannot-say-whether.md b/.abcd/work/issues/resolved/iss-2610141647313018-ideate-s-claim-status-cannot-say-whether.md similarity index 73% rename from .abcd/work/issues/open/iss-2610141647313018-ideate-s-claim-status-cannot-say-whether.md rename to .abcd/work/issues/resolved/iss-2610141647313018-ideate-s-claim-status-cannot-say-whether.md index 58c48eab53..a22564baa9 100644 --- a/.abcd/work/issues/open/iss-2610141647313018-ideate-s-claim-status-cannot-say-whether.md +++ b/.abcd/work/issues/resolved/iss-2610141647313018-ideate-s-claim-status-cannot-say-whether.md @@ -11,6 +11,12 @@ production_mode: hand-written found_at: "internal/core/ideate/ideate.go" remedy: "Deliver itd-2610141646454347: claim status separates quoted from verified, and the primary source has a checked shape." related_intents: [itd-2610141646454347] +resolution: "claim status separates quoted from verified, a verified claim resting only on the idea's own origin source is refused, and the primary source has a checked shape" +impact: breaking +resolved_by: + intent: "itd-2610141646454347" + spec: "spc-2610142138047122" + commit: "81baf3f6a" --- ideate's claim status cannot say whether a claim was verified as quoted or as true: a write-up's own experience can only be checked as quotation, and the three statuses carry no such distinction; any non-empty string is accepted as a primary source (lab finding F-9). diff --git a/.abcd/work/issues/open/iss-2610141647315880-nothing-defines-how-two-ideate.md b/.abcd/work/issues/resolved/iss-2610141647315880-nothing-defines-how-two-ideate.md similarity index 73% rename from .abcd/work/issues/open/iss-2610141647315880-nothing-defines-how-two-ideate.md rename to .abcd/work/issues/resolved/iss-2610141647315880-nothing-defines-how-two-ideate.md index c0448b5e89..f21b5affd7 100644 --- a/.abcd/work/issues/open/iss-2610141647315880-nothing-defines-how-two-ideate.md +++ b/.abcd/work/issues/resolved/iss-2610141647315880-nothing-defines-how-two-ideate.md @@ -11,6 +11,12 @@ production_mode: hand-written found_at: "internal/core/ideate/ideate.go" remedy: "Deliver itd-2610141646454347: a reframing field, per-reviewer attempts, and the rule that a split pair goes to an advisor, recorded in the verdict." related_intents: [itd-2610141646454347] +resolution: "a reframed verdict carries its reframing, attempts are kept per reviewer, and a split between reviewers is refused unless the ruling that settled it names who ruled, the verdict and the reason" +impact: breaking +resolved_by: + intent: "itd-2610141646454347" + spec: "spc-2610142138047122" + commit: "81baf3f6a" --- Nothing defines how two ideate evaluators are merged or where a reframing is recorded: the schema has one adversarial leg, one verdict and no reframing field, although the page says a graduation carries the reframing; runs encode it as a final 'survived' kill attempt by unwritten convention (lab finding F-8). diff --git a/commands/ideate.md b/commands/ideate.md index 0c2fa6c61e..4325f4cf42 100644 --- a/commands/ideate.md +++ b/commands/ideate.md @@ -17,9 +17,9 @@ mistake to warn about. Reach for it when an idea is big and unproven enough that being wrong about it would be expensive. **The binary does no judging.** The three legs below are yours to run. `abcd -ideate record` validates what they produced, proves every citation names a real -record, and writes the durable verdict. It never fetches a source and never -forms an opinion. +ideate record` validates what they produced, proves every record id the legs +cite names a real record, and writes the durable verdict. It never fetches a source and +never forms an opinion. ## First, ask: private or public? @@ -39,6 +39,13 @@ decides who can ever read the verdict. `` is the repository's root commit, which the binary resolves itself. +## Where the idea came from + +Before the first leg, note the sources the idea came from — the post, the paper, +the conversation — as the verdict's `origin_sources`. An idea with nothing behind +it but its own words takes an empty list; the list is never left out, because +leg 1 is checked against it. + ## The three legs, in order The order is not cosmetic — each leg changes what the next one is looking at — @@ -56,10 +63,18 @@ A secondary citation is not a check. In the run this protocol comes from, three secondary-source claims were falsified once the primary was opened, one of them an extraction artefact that no source ever said. -Produce, for each claim: the claim, the primary source (a URL or a document -reference), and the finding — `verified`, `falsified`, or `unverifiable`. An -honest `unverifiable` is a result; a missing source is not, and the binary -refuses a claim that names none. +Produce, for each claim: the claim, the primary source, and the finding — +`verified`, `quoted`, `falsified`, or `unverifiable`. A primary source is one of +three things: an `https://` address, `source:` for a source filed in your +sources corpus (`/abcd:library`), or a record id. Anything else — a remark heard +in passing, a local file — is refused. Where a sources corpus exists, a +`source:` must name an entry in it. + +A claim checked only against the source the idea came from is `quoted`: the +source says what the idea says it says, which proves the quotation, not the +claim. The binary refuses a claim marked `verified` whose primary source is one of +the `origin_sources`. An honest `unverifiable` is a result; a missing source is +not, and the binary refuses a claim that names none. ### Leg 2 — Record grill @@ -77,20 +92,28 @@ verdict. Answer one question: does an entry already Every hit is **cited by record id** (`itd-N`, `spc-N`, `iss-N`, `adr-N`, `dcn-N`), and `abcd ideate record` refuses the whole verdict if an id does not -resolve in this repository. Check the id before you write it down. The brief, the principles, +resolve in this repository — in a hit, and in a claim or its primary source, a +note, a kill attempt, a reviewer's label or sources, the ruling, a rejected +alternative, an origin source or the reframing (an id inside a web address is +part of the address). A search term naming an id is not checked, because +searching for one is the finding. Check the id before you write it down. The brief, the principles, the research notes, and a frozen log's lines carry no per-entry id, so a hit that rests on one goes in the `note` field of the nearest citable record. A standing verdict on the submitted idea — recorded in a research note or a decision — is precisely the hit this leg exists to surface; missing it re-opens a decided question. -Finding **nothing** is a legitimate and common outcome — say so; do not -manufacture a hit. +Record **every search** you ran: the terms, the places you searched, and how many +entries came back. Finding **nothing** is a legitimate and common outcome — say +so; do not manufacture a hit. A grill with no hits must carry its searches, and +the binary refuses one that carries neither, because an empty grill and a broken +search would otherwise write the same record. ### Leg 3 — Adversarial review -Hand the idea to a **fresh evaluator whose only job is to kill it**. Three -properties make this leg worth running, and they are the whole point of it: +Hand the idea to **fresh evaluators whose only job is to kill it** — two +reviewers, each its own agent. Four properties make this leg worth running, and +they are the whole point of it: - **Fresh context.** The evaluator must not be the session that ran legs 1 and 2. Dispatch it as a separate agent with its own context. @@ -102,9 +125,17 @@ properties make this leg worth running, and they are the whole point of it: no mention of who wants this or how excited they are. The evaluator sees an artefact of unknown provenance. This is the only measured debiasing effect in the whole protocol; leaking authorship converts an adversary into a colleague. - -Ask it for concrete **kill attempts**, each with an outcome: `survived`, -`partial` (it landed but is not on its own fatal), or `fatal`. +- **One idea per context.** Give each reviewer exactly one idea in its context, + and dispatch a new reviewer for the next idea. A reviewer that has already + judged another idea compares the two instead of attacking this one. + +Ask each reviewer for concrete **kill attempts**, each with an outcome: +`survived`, `partial` (it landed but is not on its own fatal), or `fatal`; for its +own verdict (`survives`, `killed` or `reframed`); and for the sources it +consulted, in the same three shapes as a primary source. Record each reviewer +under a label of its own. If the reviewers' verdicts differ, settle the split and +record the **ruling**: who ruled, the verdict they reached, and why. The binary +refuses a split with no ruling. ## Record the verdict @@ -120,21 +151,34 @@ refuses a run with neither or both, writing nothing. ```json { - "schema_version": 1, + "schema_version": 2, "prompt_version": "1.0.0", "idea": "the idea in the words it entered with", + "origin_sources": ["https://… the post the idea came from"], "legs": [ {"kind": "research", "claims": [ - {"claim": "…", "primary_source": "https://…", "status": "verified|falsified|unverifiable"} - ]}, - {"kind": "record-grill", "hits": [ - {"record": "itd-104", "relation": "covered|contradicted|superseded", "note": "optional"} + {"claim": "…", "primary_source": "https://…", "status": "verified|quoted|falsified|unverifiable"} ]}, - {"kind": "adversarial-review", "kill_attempts": [ - {"attempt": "…", "outcome": "survived|partial|fatal"} - ]} + {"kind": "record-grill", + "hits": [ + {"record": "itd-104", "relation": "covered|contradicted|superseded", "note": "optional"} + ], + "searches": [ + {"terms": ["…"], "places": [".abcd/development/intents/"], "count": 1} + ]}, + {"kind": "adversarial-review", + "reviewers": [ + {"label": "reviewer-a", "verdict": "survives|killed|reframed", + "kill_attempts": [{"attempt": "…", "outcome": "survived|partial|fatal"}], + "sources": ["https://…"]}, + {"label": "reviewer-b", "verdict": "survives|killed|reframed", + "kill_attempts": [{"attempt": "…", "outcome": "survived|partial|fatal"}], + "sources": []} + ], + "ruling": {"by": "who ruled", "verdict": "survives|killed|reframed", "reason": "…"}} ], "verdict": "survives|killed|reframed", + "reframing": "only with a reframed verdict: the shape of the idea that survives", "rejected_alternatives": [ {"alternative": "…", "why_rejected": "…"} ] @@ -145,6 +189,11 @@ refuses a run with neither or both, writing nothing. record as a surviving one — that record is what stops the idea being talked back into six months from now. +`ruling` is needed only when the reviewers' verdicts differ, and `reframing` only +with a `reframed` verdict, which the binary refuses without one; a reframing beside +any other verdict is refused too. The binary records version 2 only: a payload of +`"schema_version": 1` is refused with a message naming version 2. + **Rejected alternatives are not optional.** If genuinely none were considered, say so explicitly with `"no_rejected_alternatives": true` and an empty list; the binary refuses an empty list that arrives by omission, because silence and "we @@ -152,9 +201,10 @@ weighed nothing" are indistinguishable to a later reader. ## What the binary does with it -It writes one record, `-ideate-.md` — the idea, the three legs, the -verdict, and the rejected alternatives, rendered for a human — to the store the -answer names: +It writes one record, `-ideate-.md` — the idea and where it came +from, the three legs (with the grill's searches, each reviewer's section and any +ruling), the verdict, the rejected alternatives and any reframing, rendered for a +human — to the store the answer names: | Answer | Record | Decision | |---|---|---| @@ -184,7 +234,8 @@ alone; relay that line. A public verdict is committed, and a private one is a file on disk, so in both stores every free-text field of the verdict is scanned and redacted before anything is written: a token, an email, or an absolute home path in -the idea, a claim, a note, a kill attempt, or a rejected alternative is rewritten +the idea, a source, a claim, a note, a search, a kill attempt, the ruling, a +rejected alternative, or the reframing is rewritten to a placeholder, and the count comes back as `redactions`. A repository whose scanner config cannot be read refuses the run outright rather than record under a detector it cannot trust — nothing is written, and the same payload records once @@ -193,14 +244,19 @@ the config is repaired. Exit codes: - **0** — recorded. Report the `visibility`, the `verdict`, the `path`, and the `cited_records` - (the ids the grill cited, every one proved to resolve). Show the user the + (the ids the grill cited, every one proved to resolve, as is every id cited + elsewhere in the verdict, apart from the idea text and the grill's searches). Show the user the written record. If `redactions` is non-zero, say so — the record no longer matches what was composed, and whoever composed it needs to know a credential or an identity was in the text. - **2** — refused, and **nothing was written**. Relay the diagnostic: a run with neither `--private` nor `--public`, or both, is refused (ask the product thinker, then re-run with the answer), an - unresolvable citation names the id, a payload fault names the field, a + unresolvable citation names the leg and the id, a primary source of no known + shape, a `source:` the corpus does not carry, an empty grill with no + searches, a split between reviewers with no ruling, a reframed verdict with no + reframing, a `verified` claim resting on the idea's own source, and a version 1 + payload are each refused naming what to fix, a payload fault names the field, a verdict record that already exists under today's date is refused rather than overwritten (give the re-run its own slug, or keep the first record), a public verdict in a repository with no shared tier (`.abcd/work/`) is @@ -214,7 +270,7 @@ Exit codes: path, `abcd intent ""`. Ideate mints no intent itself; graduation stays a deliberate act you take with the user. - **killed** — stop. Do not file an intent, and do not soften the record. -- **reframed** — any graduation carries the reframing recorded in leg 3, not the +- **reframed** — any graduation carries the verdict's `reframing`, not the original wording. **Binary resolution.** Run `"${CLAUDE_PLUGIN_ROOT}/abcd"` — a plugin install diff --git a/internal/core/ideate/enum_refusal_test.go b/internal/core/ideate/enum_refusal_test.go index be9105b53c..b0149e2bee 100644 --- a/internal/core/ideate/enum_refusal_test.go +++ b/internal/core/ideate/enum_refusal_test.go @@ -41,8 +41,17 @@ func TestEnumRefusalsDoNotEchoTheValue(t *testing.T) { legs(p, 1)["hits"].([]any)[0].(map[string]any)["relation"] = enumLeak }, "grill hit 1"}, "kill outcome": {func(p map[string]any) { - legs(p, 2)["kill_attempts"].([]any)[0].(map[string]any)["outcome"] = enumLeak + legs(p, 2)["reviewers"].([]any)[0].(map[string]any)["kill_attempts"].([]any)[0].(map[string]any)["outcome"] = enumLeak }, "kill attempt 1"}, + "reviewer verdict": {func(p map[string]any) { + legs(p, 2)["reviewers"].([]any)[1].(map[string]any)["verdict"] = enumLeak + }, "reviewer 2"}, + "ruling verdict": {func(p map[string]any) { + legs(p, 2)["ruling"] = map[string]any{"by": "the product thinker", "verdict": enumLeak, "reason": "a reason"} + }, "ruling"}, + "primary source": {func(p map[string]any) { + legs(p, 0)["claims"].([]any)[0].(map[string]any)["primary_source"] = shortLeak + }, "claim 1"}, } for name, tc := range cases { t.Run(name, func(t *testing.T) { diff --git a/internal/core/ideate/ideate.go b/internal/core/ideate/ideate.go index 6dd77942ca..a7835f1e9e 100644 --- a/internal/core/ideate/ideate.go +++ b/internal/core/ideate/ideate.go @@ -31,7 +31,7 @@ import ( // SchemaVersion stamps the verdict payload. It is versioned independently of // every other abcd artifact, so a future breaking change to the shape is // detectable rather than silently misread by a host composing against the old one. -const SchemaVersion = 1 +const SchemaVersion = 2 // MaxPayloadBytes caps the untrusted verdict document, mirroring the synthesis and // release-ingest ceilings. It is exported so a front door can bound its READ at @@ -85,6 +85,12 @@ type Store struct { Visibility Visibility RootSHA string Home string + // SourcesDir is the sources corpus a `source:` citation is resolved + // against (internal/core/source). It is passed in rather than found, so a + // test never reads the machine's own corpus; empty, or a directory where no + // corpus exists, means there is nothing to resolve against, and a well-formed + // key is accepted on its shape alone. + SourcesDir string } const ( @@ -92,8 +98,22 @@ const ( maxClaims = 200 // maxHits caps the record grill's hits. maxHits = 200 - // maxKillAttempts caps the adversarial leg's attempts. + // maxSearches caps the record grill's search record. + maxSearches = 200 + // maxSearchTerms and maxSearchPlaces cap one search's terms and places. + maxSearchTerms = 50 + maxSearchPlaces = 50 + // maxKillAttempts caps one reviewer's attempts. maxKillAttempts = 200 + // maxReviewers caps the adversarial leg's reviewers. + maxReviewers = 16 + // maxReviewerSources caps the sources one reviewer names. + maxReviewerSources = 50 + // maxOriginSources caps the sources the idea came from. + maxOriginSources = 50 + // maxLabelBytes caps a reviewer's label and a ruling's "by", which name a + // reviewer or a person rather than say anything. + maxLabelBytes = 128 // maxRejected caps the rejected-alternatives list. maxRejected = 200 // maxProseBytes caps one prose field after cleaning (mirrors the synthesis @@ -101,8 +121,8 @@ const ( maxProseBytes = 4096 // maxIdeaBytes caps the idea text — a longer-form field than a single claim. maxIdeaBytes = 8192 - // maxSourceBytes caps a claim's primary-source reference (a URL or a document - // citation, never a document). + // maxSourceBytes caps a source reference (an address, a filed source key or a + // record id, never a document), and one search term or place. maxSourceBytes = 1024 // maxRecordIDBytes bounds a cited id BEFORE it is matched or reported. The id // grammar admits unbounded digits, so without a cap "itd-" followed by a @@ -171,8 +191,13 @@ var legOrder = []LegKind{LegResearch, LegRecordGrill, LegAdversarialReview} type ClaimStatus string const ( - // ClaimVerified: the primary source supports the claim. + // ClaimVerified: the primary source supports the claim, and it is not one of + // the sources the idea came from. ClaimVerified ClaimStatus = "verified" + // ClaimQuoted: the claim was checked against the idea's own source and + // found to say what the idea says it says. That proves the quotation, not + // the claim, so it is never verified. + ClaimQuoted ClaimStatus = "quoted" // ClaimFalsified: the primary source contradicts it. Three of these killed an // idea in the proven manual run. ClaimFalsified ClaimStatus = "falsified" @@ -181,7 +206,7 @@ const ( ClaimUnverifiable ClaimStatus = "unverifiable" ) -var claimStatusEnum = map[ClaimStatus]bool{ClaimVerified: true, ClaimFalsified: true, ClaimUnverifiable: true} +var claimStatusEnum = map[ClaimStatus]bool{ClaimVerified: true, ClaimQuoted: true, ClaimFalsified: true, ClaimUnverifiable: true} // Relation is how an existing record entry bears on the idea. type Relation string @@ -244,10 +269,45 @@ type KillAttempt struct { // field is a refusal — a payload that smuggles grill hits into the research leg is // not a leg abcd knows how to read. type Leg struct { - Kind LegKind `json:"kind"` - Claims []Claim `json:"claims,omitempty"` - Hits []GrillHit `json:"hits,omitempty"` - KillAttempts []KillAttempt `json:"kill_attempts,omitempty"` + Kind LegKind `json:"kind"` + Claims []Claim `json:"claims,omitempty"` + Hits []GrillHit `json:"hits,omitempty"` + Searches []Search `json:"searches,omitempty"` + // Reviewers and Ruling are the adversarial leg's evidence. Kill attempts + // live under the reviewer that made them, never on the leg itself, so a + // record of two reviewers cannot blur into one. + Reviewers []Reviewer `json:"reviewers,omitempty"` + Ruling *Ruling `json:"ruling,omitempty"` +} + +// Search is one search the record grill ran: the terms it searched for, the +// places it searched, and how many entries came back. It is what lets an empty +// grill be told from a broken one — "searched these places for these terms and +// found nothing" is a finding, and a grill that cannot say so recorded nothing. +// Count is a pointer so an omitted count is refused rather than read as zero. +type Search struct { + Terms []string `json:"terms"` + Places []string `json:"places"` + Count *int `json:"count"` +} + +// Reviewer is one adversarial evaluator's whole contribution, kept under its +// own label: what it tried, what it concluded, and what it consulted. +type Reviewer struct { + Label string `json:"label"` + KillAttempts []KillAttempt `json:"kill_attempts"` + Verdict Verdict `json:"verdict"` + Sources []string `json:"sources"` +} + +// Ruling settles a split between reviewers: who ruled, what they ruled, and +// why. It is required exactly when the reviewers' verdicts differ, because a +// record that shows a split and no ruling leaves a later reader to guess which +// reviewer the verdict followed. +type Ruling struct { + By string `json:"by"` + Verdict Verdict `json:"verdict"` + Reason string `json:"reason"` } // RejectedAlternative is one option considered and turned down, with the reason. @@ -267,6 +327,11 @@ type Payload struct { PromptVersion string `json:"prompt_version"` // Idea is the idea as captured, in the words it entered the gauntlet with. Idea string `json:"idea"` + // OriginSources are the sources the idea came from. The list is REQUIRED + // and may be empty: a claim checked only against one of these is quoted, + // not verified, and a verdict that left them out would skip that check by + // omission. An empty list says the idea has no source behind it. + OriginSources []string `json:"origin_sources"` // Legs are the three legs, IN ORDER. An array rather than three named fields // precisely so the order is a property of the document the binary can check. Legs []Leg `json:"legs"` @@ -278,18 +343,23 @@ type Payload struct { // admissible only with this set; omission is not, and setting it beside a // populated list is a contradiction the document is refused for. NoRejectedAlternatives bool `json:"no_rejected_alternatives,omitempty"` + // Reframing is the shape of the idea that survives a reframed verdict. It is + // required for that verdict and refused beside any other, so what graduates + // from a reframed idea is a field of the record, not a reading of its legs. + Reframing string `json:"reframing,omitempty"` } // ClaimTally counts the research leg's outcomes, so a reader of the result knows // what the claims table found without re-reading the record. type ClaimTally struct { Verified int `json:"verified"` + Quoted int `json:"quoted"` Falsified int `json:"falsified"` Unverifiable int `json:"unverifiable"` } // Total reports how many claims the research leg carried. -func (t ClaimTally) Total() int { return t.Verified + t.Falsified + t.Unverifiable } +func (t ClaimTally) Total() int { return t.Verified + t.Quoted + t.Falsified + t.Unverifiable } // Result is the transport-agnostic outcome of `ideate record`. It is returned only // when the record was written; every refusal is an error, so a caller holding a @@ -318,7 +388,11 @@ type Result struct { // them proved to resolve — the citation gate's proof, so a reviewer can check // the record without re-running the resolution. CitedRecords []string `json:"cited_records"` - // KillAttempts counts the adversarial leg's attempts. + // Searches counts the record grill's searches. + Searches int `json:"searches"` + // Reviewers counts the adversarial leg's reviewers. + Reviewers int `json:"reviewers"` + // KillAttempts counts the adversarial leg's attempts, across every reviewer. KillAttempts int `json:"kill_attempts"` // RejectedAlternatives counts the recorded alternatives. RejectedAlternatives int `json:"rejected_alternatives"` diff --git a/internal/core/ideate/record.go b/internal/core/ideate/record.go index b50ea0f2ec..58e2050e66 100644 --- a/internal/core/ideate/record.go +++ b/internal/core/ideate/record.go @@ -25,18 +25,22 @@ import ( "encoding/json" "errors" "fmt" + "net/url" "os" "path" "path/filepath" + "regexp" "sort" "strings" "time" + "unicode" "github.com/intentdriven/abcd/internal/abcdhome" "github.com/intentdriven/abcd/internal/adapter/scanner" "github.com/intentdriven/abcd/internal/core/decisions" "github.com/intentdriven/abcd/internal/core/homeignore" "github.com/intentdriven/abcd/internal/core/recordid" + "github.com/intentdriven/abcd/internal/core/source" "github.com/intentdriven/abcd/internal/core/update" "github.com/intentdriven/abcd/internal/fsutil" "github.com/intentdriven/abcd/internal/gitutil" @@ -56,18 +60,48 @@ func (e *SlugError) Error() string { "and carries no path separator or dot segment", termsafe.Sanitize(e.Slug), e.Reason, MaxSlugLen) } -// CitationError is the record-grill citation gate's refusal: a hit names a record -// id that does not resolve in this repository, so the grill is reporting on -// something that is not there. It names every offending id, because a composer -// handed one id at a time re-runs the whole gauntlet once per mistake. +// CitationError is the citation gate's refusal: the verdict names a record id +// that does not resolve in this repository, in a grill hit or anywhere else in +// its legs, so it is reporting on something that is not there. It names every +// offending id and where each was written, because a composer handed one id at +// a time re-runs the whole gauntlet once per mistake. type CitationError struct { + // Unresolved is the sorted, deduped set of ids that name no record. Unresolved []string + // At is each unresolved id with the part of the verdict it was written in, + // in the order the verdict carries them. + At []UnresolvedCitation +} + +// UnresolvedCitation is one id that names no record, and the leg (or the other +// part of the verdict) it was written in. +type UnresolvedCitation struct { + Where string + ID string +} + +// newCitationError builds the refusal from the citations that did not resolve. +func newCitationError(at []UnresolvedCitation) *CitationError { + seen := map[string]bool{} + var ids []string + for _, c := range at { + if !seen[c.ID] { + seen[c.ID] = true + ids = append(ids, c.ID) + } + } + sort.Strings(ids) + return &CitationError{Unresolved: ids, At: at} } func (e *CitationError) Error() string { - return "the record grill cites records that do not exist in this repository — nothing was written: " + - strings.Join(e.Unresolved, ", ") + - "\nevery grill hit is cited by id, and an id that names no record is a hit on nothing; re-run the grill against the live record" + parts := make([]string, len(e.At)) + for i, c := range e.At { + parts[i] = c.Where + " cites " + c.ID + } + return "the verdict cites records that do not exist in this repository — nothing was written: " + + strings.Join(parts, "; ") + + "\nevery record id in the verdict must name a record, and an id that names none cites nothing; check each against the live record" } // ExistsError is the no-overwrite refusal: a verdict record for this slug already @@ -140,7 +174,7 @@ func Record(repoRoot, slug string, store Store, raw []byte, at time.Time) (Resul if err != nil { return Result{}, err } - v, err := validate(repoRoot, red, p) + v, err := validate(repoRoot, store.SourcesDir, red, p) if err != nil { return Result{}, err } @@ -334,6 +368,15 @@ func decodePayload(repoRoot string, raw []byte) (Payload, error) { if len(raw) > MaxPayloadBytes { return Payload{}, fmt.Errorf("verdict payload exceeds the %d-byte cap", MaxPayloadBytes) } + // An older shape is refused before the strict decode, which would stop at + // the first field version 2 moved (a version-1 adversarial leg's + // kill_attempts) and never name the version to compose. + var peek struct { + SchemaVersion int `json:"schema_version"` + } + if json.Unmarshal(raw, &peek) == nil && peek.SchemaVersion > 0 && peek.SchemaVersion < SchemaVersion { + return Payload{}, olderSchemaError(peek.SchemaVersion) + } dec := json.NewDecoder(bytes.NewReader(raw)) dec.DisallowUnknownFields() var p Payload @@ -349,8 +392,10 @@ func decodePayload(repoRoot string, raw []byte) (Payload, error) { case p.SchemaVersion > SchemaVersion: return Payload{}, update.TooNew("verdict", p.SchemaVersion, SchemaVersion) - case p.SchemaVersion != SchemaVersion: + case p.SchemaVersion < 0: return Payload{}, fmt.Errorf("unsupported verdict schema_version %d", p.SchemaVersion) + case p.SchemaVersion != SchemaVersion: + return Payload{}, olderSchemaError(p.SchemaVersion) } if !promptVersionRe.MatchString(p.PromptVersion) { return Payload{}, errors.New("verdict payload is missing a semver prompt_version") @@ -358,17 +403,32 @@ func decodePayload(repoRoot string, raw []byte) (Payload, error) { return p, nil } +// olderSchemaError refuses an older shape, which is never upgraded: version 1 +// had nowhere to put a search, a second reviewer or a reframing, so reading it +// as version 2 would invent what each leg proved. The message names the +// version to compose, because the host that sent it is reading an older +// command page. +func olderSchemaError(got int) error { + return fmt.Errorf("the verdict payload is schema_version %d, which this abcd no longer records — nothing was written; "+ + "compose it as version %d, which adds origin_sources, the grill's searches, the adversarial leg's reviewers and ruling, "+ + "and a reframing for a reframed verdict (the ideate command page carries the shape)", got, SchemaVersion) +} + // verdictDoc is the validated, cleaned document — the only shape the renderer and // the result builder ever see, so neither can reach a raw untrusted field. type verdictDoc struct { slug string idea string verdict Verdict + origins []string claims []Claim hits []GrillHit - kills []KillAttempt + searches []Search + reviewers []Reviewer + ruling *Ruling rejected []RejectedAlternative noneRejected bool + reframing string cited []string // redactions counts the secret/PII spans stage one rewrote out of the free // text above, so the surface can say what happened instead of the record @@ -378,8 +438,8 @@ type verdictDoc struct { // validate runs every gate over the decoded payload and returns the cleaned // document. The order is deliberate: cheap structural checks first, and the -// citation gate — the only one that touches the filesystem — last, so a malformed -// document never costs a record scan. +// citation gate and the sources check — the only ones that touch the +// filesystem — last, so a malformed document never costs a record scan. // // red is stage one of the store-before-commit gate (redact.go). Every free-text // field is redacted on the way in, ahead of termsafe's cleaning and well ahead of @@ -389,7 +449,9 @@ type verdictDoc struct { // relation, a kill outcome, a cited record id — are validated shapes with nothing // to redact, and running a matcher over them would corrupt a value the record is // keyed on rather than protect anything. -func validate(repoRoot string, red *recordRedactor, p Payload) (verdictDoc, error) { +// +// sourcesDir is the corpus a `source:` is resolved against (Store.SourcesDir). +func validate(repoRoot, sourcesDir string, red *recordRedactor, p Payload) (verdictDoc, error) { var v verdictDoc idea, n := red.field(p.Idea) @@ -415,30 +477,71 @@ func validate(repoRoot string, red *recordRedactor, p Payload) (verdictDoc, erro } } var err error - var n1, n2, n3, n4 int - if v.claims, n1, err = validateResearch(red, p.Legs[0]); err != nil { + var n0, n1, n2, n3, n4, n5 int + if v.origins, n0, err = validateOrigins(red, p.OriginSources); err != nil { return v, err } - if v.hits, n2, err = validateGrill(red, p.Legs[1]); err != nil { + if v.claims, n1, err = validateResearch(red, p.Legs[0], v.origins); err != nil { return v, err } - if v.kills, n3, err = validateAdversarial(red, p.Legs[2]); err != nil { + if v.hits, v.searches, n2, err = validateGrill(red, p.Legs[1]); err != nil { + return v, err + } + if v.reviewers, v.ruling, n3, err = validateAdversarial(red, p.Legs[2]); err != nil { return v, err } if v.rejected, v.noneRejected, n4, err = validateRejected(red, p); err != nil { return v, err } - v.redactions += n1 + n2 + n3 + n4 - if v.cited, err = resolveCitations(repoRoot, v.hits); err != nil { + if v.reframing, n5, err = validateReframing(red, p); err != nil { + return v, err + } + v.redactions += n0 + n1 + n2 + n3 + n4 + n5 + if v.cited, err = resolveCitations(repoRoot, v); err != nil { + return v, err + } + if err := resolveSourceKeys(sourcesDir, v); err != nil { return v, err } return v, nil } +// validateOrigins checks the sources the idea came from. The list is required +// but may be empty: the self-quotation check (validateResearch) compares every +// verified claim against it, and a list that could be left out would let that +// check be skipped by omission, the failure the rejected-alternatives marker +// exists for. An origin is free text, because an idea can come from a +// conversation as well as a page; it is never a primary source, so its shape is +// not one. +func validateOrigins(red *recordRedactor, origins []string) ([]string, int, error) { + if origins == nil { + return nil, 0, errors.New("the verdict does not say where the idea came from — nothing was written; " + + "list the sources it came from in \"origin_sources\", or [] for an idea with no source behind it " + + "(a claim checked only against the idea's own source is quoted, not verified, and that check reads the list)") + } + if len(origins) > maxOriginSources { + return nil, 0, fmt.Errorf("the verdict carries %d origin sources (max %d)", len(origins), maxOriginSources) + } + redactions := 0 + out := make([]string, 0, len(origins)) + for i, o := range origins { + text, n := red.field(o) + redactions += n + clean := termsafe.CleanProseLine(text, maxSourceBytes) + if clean == "" { + return nil, 0, fmt.Errorf("origin source %d is empty", i+1) + } + out = append(out, clean) + } + return out, redactions, nil +} + // validateResearch checks the first leg: at least one claim, each with prose, a -// named primary source, and a registered status. -func validateResearch(red *recordRedactor, leg Leg) ([]Claim, int, error) { - if len(leg.Hits) > 0 || len(leg.KillAttempts) > 0 { +// primary source of a known shape, and a registered status. A claim marked +// verified whose primary source is one of the idea's own origins is refused: +// checked against the source the idea came from, it was quoted, not verified. +func validateResearch(red *recordRedactor, leg Leg, origins []string) ([]Claim, int, error) { + if len(leg.Hits) > 0 || len(leg.Searches) > 0 || len(leg.Reviewers) > 0 || leg.Ruling != nil { return nil, 0, errors.New("the research leg carries another leg's evidence; each leg carries only its own") } if len(leg.Claims) == 0 { @@ -447,13 +550,14 @@ func validateResearch(red *recordRedactor, leg Leg) ([]Claim, int, error) { if len(leg.Claims) > maxClaims { return nil, 0, fmt.Errorf("the research leg carries %d claims (max %d)", len(leg.Claims), maxClaims) } + origin := originKeys(origins) redactions := 0 out := make([]Claim, 0, len(leg.Claims)) for i, c := range leg.Claims { at := i + 1 - // A primary source is free text too: the field holds a URL by convention, - // but nothing constrains it, and a file:// path to the researcher's own - // working copy is the commonest thing a research leg reaches for. + // A primary source is free text until its shape is checked, and a + // file:// path to the researcher's own working copy is the commonest + // thing a research leg reaches for, so it is redacted like any prose. claim, n := red.field(c.Claim) source, m := red.field(c.PrimarySource) redactions += n + m @@ -465,24 +569,123 @@ func validateResearch(red *recordRedactor, leg Leg) ([]Claim, int, error) { if src == "" { return nil, 0, fmt.Errorf("claim %d names no primary source; a claim checked against nothing is an assertion", at) } + if !sourceShaped(src) { + return nil, 0, fmt.Errorf("claim %d names a primary source of no known shape (%s); %s", + at, termsafe.DescribeRefused(src), sourceShapes) + } if !claimStatusEnum[c.Status] { - return nil, 0, fmt.Errorf("claim %d has an out-of-enum status (%s); a claim is %s, %s, or %s", - at, termsafe.DescribeRefused(string(c.Status)), ClaimVerified, ClaimFalsified, ClaimUnverifiable) + return nil, 0, fmt.Errorf("claim %d has an out-of-enum status (%s); a claim is %s, %s, %s, or %s", + at, termsafe.DescribeRefused(string(c.Status)), ClaimVerified, ClaimQuoted, ClaimFalsified, ClaimUnverifiable) + } + if c.Status == ClaimVerified && origin[sourceKey(src)] { + return nil, 0, fmt.Errorf("claim %d is marked %s, but its primary source is one of the idea's own origin sources — nothing was written; "+ + "a claim checked only against the source the idea came from is %s, not %s", at, ClaimVerified, ClaimQuoted, ClaimVerified) } out = append(out, Claim{Claim: text, PrimarySource: src, Status: c.Status}) } return out, redactions, nil } +// sourceShapes says in words what sourceShaped accepts, for every refusal of a +// source to quote, so the refusals cannot drift from the check. +var sourceShapes = "a source is an https:// address, source: for a source filed in the sources corpus, or a record id (" + + recordid.CitedIDForms() + ")" + +// sourceShaped reports whether a cleaned source is one of the three shapes a +// source may take: an https:// address with a host and no whitespace, a +// `source:` whose key has the corpus's key grammar, or a record id. The +// address is not fetched and the record id is resolved with every other +// citation (resolveCitations); this is the shape alone, so a remark heard in +// passing cannot stand where a source does. +func sourceShaped(src string) bool { + switch { + case strings.HasPrefix(src, "https://"): + if strings.IndexFunc(src, unicode.IsSpace) >= 0 { + return false + } + u, err := url.Parse(src) + return err == nil && u.Host != "" + case strings.HasPrefix(src, sourceKeyPrefix): + return source.ValidKey(strings.TrimPrefix(src, sourceKeyPrefix)) + } + return recordid.CitedIDRe.MatchString(src) +} + +// sourceKeyPrefix marks a source filed in the sources corpus. +const sourceKeyPrefix = "source:" + +// sourceKey is the spelling the self-quotation check compares sources by, so +// one source written two ways is still one source: an address loses its +// fragment and a trailing slash, its host is lower-cased and http and https +// meet, and a record id is folded as the resolver folds it (`ADR-0035` is +// adr-35). Anything else is compared as written. +func sourceKey(s string) string { + if lower := strings.ToLower(s); strings.HasPrefix(lower, "https://") || strings.HasPrefix(lower, "http://") { + if u, err := url.Parse(s); err == nil && u.Host != "" { + k := "https://" + strings.ToLower(u.Host) + strings.TrimSuffix(u.EscapedPath(), "/") + if u.RawQuery != "" { + k += "?" + u.RawQuery + } + return k + } + } + if c := recordid.CanonCitedID(s); c != "" { + return c + } + return s +} + +// originKeys is every source the idea's origins name, by sourceKey. An origin +// is free text, so besides the whole entry each word of it that is an address +// or a source key counts, as written and with surrounding punctuation trimmed +// ("https://… the post the idea came from" names the post, and so does +// "(https://…)"). A record id counts only as a whole entry: one mentioned in +// passing is not where the idea came from, and checking it is a primary check. +func originKeys(origins []string) map[string]bool { + keys := make(map[string]bool, len(origins)) + for _, o := range origins { + keys[sourceKey(o)] = true + for _, w := range strings.Fields(o) { + for _, s := range []string{w, strings.Trim(w, "()<>[]{},.;\"'")} { + if lower := strings.ToLower(s); strings.HasPrefix(lower, "https://") || strings.HasPrefix(lower, "http://") || + strings.HasPrefix(s, sourceKeyPrefix) { + keys[sourceKey(s)] = true + } + } + } + } + return keys +} + +// addressRe matches a web address written in running text, up to the next +// space, quote, bracket, comma or semicolon, so an id written straight after +// one (`…/thread;itd-9`) is still read as a citation. +var addressRe = regexp.MustCompile(`(?i)\bhttps?://[^\s,;<>"'()\[\]]+`) + +// withoutAddresses blanks every web address in text, so an id-shaped segment +// of a path (`…/adr-0007-use-markdown.md`) is read as part of the address it +// sits in, not as a citation of a record in this repository. +func withoutAddresses(text string) string { + return addressRe.ReplaceAllString(text, " ") +} + // validateGrill checks the second leg. An EMPTY hit list is valid and meaningful: // "no entry in the record touches this idea" is a finding, and the commonest one -// for a genuinely new idea. What is not valid is a hit that cites nothing usable. -func validateGrill(red *recordRedactor, leg Leg) ([]GrillHit, int, error) { - if len(leg.Claims) > 0 || len(leg.KillAttempts) > 0 { - return nil, 0, errors.New("the record-grill leg carries another leg's evidence; each leg carries only its own") +// for a genuinely new idea — but only when the grill says what it searched. A +// grill with no hits and no searches is indistinguishable from one whose search +// never ran, so it is refused. What is never valid is a hit that cites nothing +// usable, or a search that cannot say what it searched or what came back. +func validateGrill(red *recordRedactor, leg Leg) ([]GrillHit, []Search, int, error) { + if len(leg.Claims) > 0 || len(leg.Reviewers) > 0 || leg.Ruling != nil { + return nil, nil, 0, errors.New("the record-grill leg carries another leg's evidence; each leg carries only its own") } if len(leg.Hits) > maxHits { - return nil, 0, fmt.Errorf("the record grill carries %d hits (max %d)", len(leg.Hits), maxHits) + return nil, nil, 0, fmt.Errorf("the record grill carries %d hits (max %d)", len(leg.Hits), maxHits) + } + if len(leg.Hits) == 0 && len(leg.Searches) == 0 { + return nil, nil, 0, errors.New("the record grill found no hits and records no searches — nothing was written; " + + "an empty grill says what it searched: list each search under \"searches\" as {terms, places, count}, " + + "so a grill that found nothing can be told from one that never looked") } redactions := 0 out := make([]GrillHit, 0, len(leg.Hits)) @@ -491,14 +694,14 @@ func validateGrill(red *recordRedactor, leg Leg) ([]GrillHit, int, error) { // Bounded BEFORE the id is matched or quoted, so an over-long one is // described rather than echoed. if len(h.Record) > maxRecordIDBytes { - return nil, 0, fmt.Errorf("grill hit %d cites a %d-byte record id (max %d)", at, len(h.Record), maxRecordIDBytes) + return nil, nil, 0, fmt.Errorf("grill hit %d cites a %d-byte record id (max %d)", at, len(h.Record), maxRecordIDBytes) } if !recordid.CitedIDRe.MatchString(h.Record) { - return nil, 0, fmt.Errorf("grill hit %d cites %s, which is not a record id (want %s)", + return nil, nil, 0, fmt.Errorf("grill hit %d cites %s, which is not a record id (want %s)", at, termsafe.DescribeRefused(h.Record), recordid.CitedIDForms()) } if !relationEnum[h.Relation] { - return nil, 0, fmt.Errorf("grill hit %d (%s) has an out-of-enum relation (%s); a hit is %s, %s, or %s", + return nil, nil, 0, fmt.Errorf("grill hit %d (%s) has an out-of-enum relation (%s); a hit is %s, %s, or %s", at, h.Record, termsafe.DescribeRefused(string(h.Relation)), RelationCovered, RelationContradicted, RelationSuperseded) } // The cited id is NOT redacted: CitedIDRe has just proved it is a record @@ -512,42 +715,208 @@ func validateGrill(red *recordRedactor, leg Leg) ([]GrillHit, int, error) { Note: termsafe.CleanProseLine(note, maxProseBytes), }) } + searches, n, err := validateSearches(red, leg.Searches) + if err != nil { + return nil, nil, 0, err + } + return out, searches, redactions + n, nil +} + +// validateSearches checks the grill's search record: each search names at least +// one term and one place, and says how many entries came back. A count of zero +// is the finding an empty grill rests on; an omitted count is refused, because +// read as zero it would report a search that never returned as one that found +// nothing. +func validateSearches(red *recordRedactor, in []Search) ([]Search, int, error) { + if len(in) > maxSearches { + return nil, 0, fmt.Errorf("the record grill carries %d searches (max %d)", len(in), maxSearches) + } + redactions := 0 + out := make([]Search, 0, len(in)) + for i, s := range in { + at := i + 1 + if s.Count == nil { + return nil, 0, fmt.Errorf("grill search %d records no count; a search that cannot say what came back records nothing", at) + } + if *s.Count < 0 { + return nil, 0, fmt.Errorf("grill search %d records a negative count", at) + } + terms, n, err := cleanList(red, s.Terms, maxSearchTerms, fmt.Sprintf("grill search %d", at), "term") + if err != nil { + return nil, 0, err + } + places, m, err := cleanList(red, s.Places, maxSearchPlaces, fmt.Sprintf("grill search %d", at), "place") + if err != nil { + return nil, 0, err + } + redactions += n + m + count := *s.Count + out = append(out, Search{Terms: terms, Places: places, Count: &count}) + } + return out, redactions, nil +} + +// cleanList redacts and line-cleans a required, bounded list of short values, +// refusing an empty list or an empty member. owner and noun name the list in a +// refusal ("grill search 2 names no place"). +func cleanList(red *recordRedactor, in []string, max int, owner, noun string) ([]string, int, error) { + if len(in) == 0 { + return nil, 0, fmt.Errorf("%s names no %s", owner, noun) + } + if len(in) > max { + return nil, 0, fmt.Errorf("%s names %d %ss (max %d)", owner, len(in), noun, max) + } + redactions := 0 + out := make([]string, 0, len(in)) + for j, raw := range in { + text, n := red.field(raw) + redactions += n + clean := termsafe.CleanProseLine(text, maxSourceBytes) + if clean == "" { + return nil, 0, fmt.Errorf("%s %s %d is empty", owner, noun, j+1) + } + out = append(out, clean) + } return out, redactions, nil } -// validateAdversarial checks the third leg: at least one attempt, each with prose -// and a registered outcome. The leg's fresh-context, off-policy, unknown-authorship -// conduct is the orchestrating prompt's obligation — the binary cannot observe how -// an agent was run, and pretending to check it would be theatre. -func validateAdversarial(red *recordRedactor, leg Leg) ([]KillAttempt, int, error) { - if len(leg.Claims) > 0 || len(leg.Hits) > 0 { - return nil, 0, errors.New("the adversarial-review leg carries another leg's evidence; each leg carries only its own") +// validateAdversarial checks the third leg: at least one reviewer, each under a +// label of its own with at least one kill attempt, a registered verdict, and the +// sources it consulted; and, where the reviewers' verdicts differ, the ruling +// that settled the split. The leg's fresh-context, off-policy, unknown-authorship +// conduct is the orchestrating prompt's obligation — the binary cannot observe +// how an agent was run, and pretending to check it would be theatre. +func validateAdversarial(red *recordRedactor, leg Leg) ([]Reviewer, *Ruling, int, error) { + if len(leg.Claims) > 0 || len(leg.Hits) > 0 || len(leg.Searches) > 0 { + return nil, nil, 0, errors.New("the adversarial-review leg carries another leg's evidence; each leg carries only its own") + } + if len(leg.Reviewers) == 0 { + return nil, nil, 0, errors.New("the adversarial-review leg carries no reviewers; an adversary that was never run reviewed nothing") + } + if len(leg.Reviewers) > maxReviewers { + return nil, nil, 0, fmt.Errorf("the adversarial review carries %d reviewers (max %d)", len(leg.Reviewers), maxReviewers) + } + redactions := 0 + labels := map[string]int{} + out := make([]Reviewer, 0, len(leg.Reviewers)) + for i, r := range leg.Reviewers { + at := i + 1 + raw, n := red.field(r.Label) + redactions += n + label := termsafe.CleanProseLine(raw, maxLabelBytes) + if label == "" { + return nil, nil, 0, fmt.Errorf("reviewer %d has no label; each reviewer's findings are kept under its label", at) + } + if first, dup := labels[label]; dup { + return nil, nil, 0, fmt.Errorf("reviewer %d repeats the label of reviewer %d; each reviewer is kept under a label of its own", at, first) + } + labels[label] = at + if !r.Verdict.Valid() { + return nil, nil, 0, fmt.Errorf("reviewer %d has an out-of-enum verdict (%s); a reviewer's verdict is one of %s|%s|%s", + at, termsafe.DescribeRefused(string(r.Verdict)), VerdictSurvives, VerdictKilled, VerdictReframed) + } + kills, n, err := validateKillAttempts(red, at, r.KillAttempts) + if err != nil { + return nil, nil, 0, err + } + redactions += n + if len(r.Sources) > maxReviewerSources { + return nil, nil, 0, fmt.Errorf("reviewer %d names %d sources (max %d)", at, len(r.Sources), maxReviewerSources) + } + sources := make([]string, 0, len(r.Sources)) + for j, s := range r.Sources { + text, n := red.field(s) + redactions += n + src := termsafe.CleanProseLine(text, maxSourceBytes) + if !sourceShaped(src) { + return nil, nil, 0, fmt.Errorf("reviewer %d source %d is of no known shape (%s); %s", + at, j+1, termsafe.DescribeRefused(src), sourceShapes) + } + sources = append(sources, src) + } + out = append(out, Reviewer{Label: label, KillAttempts: kills, Verdict: r.Verdict, Sources: sources}) } - if len(leg.KillAttempts) == 0 { - return nil, 0, errors.New("the adversarial-review leg carries no kill attempts; an adversary that tried nothing reviewed nothing") + ruling, n, err := validateRuling(red, leg.Ruling, out) + if err != nil { + return nil, nil, 0, err + } + return out, ruling, redactions + n, nil +} + +// validateKillAttempts checks one reviewer's attempts: at least one, each with +// prose and a registered outcome. +func validateKillAttempts(red *recordRedactor, reviewer int, in []KillAttempt) ([]KillAttempt, int, error) { + if len(in) == 0 { + return nil, 0, fmt.Errorf("reviewer %d carries no kill attempts; an adversary that tried nothing reviewed nothing", reviewer) } - if len(leg.KillAttempts) > maxKillAttempts { - return nil, 0, fmt.Errorf("the adversarial review carries %d attempts (max %d)", len(leg.KillAttempts), maxKillAttempts) + if len(in) > maxKillAttempts { + return nil, 0, fmt.Errorf("reviewer %d carries %d attempts (max %d)", reviewer, len(in), maxKillAttempts) } redactions := 0 - out := make([]KillAttempt, 0, len(leg.KillAttempts)) - for i, k := range leg.KillAttempts { + out := make([]KillAttempt, 0, len(in)) + for i, k := range in { at := i + 1 attempt, n := red.field(k.Attempt) redactions += n text := termsafe.CleanProseLine(attempt, maxProseBytes) if text == "" { - return nil, 0, fmt.Errorf("kill attempt %d has no text", at) + return nil, 0, fmt.Errorf("reviewer %d kill attempt %d has no text", reviewer, at) } if !killOutcomeEnum[k.Outcome] { - return nil, 0, fmt.Errorf("kill attempt %d has an out-of-enum outcome (%s); an attempt is %s, %s, or %s", - at, termsafe.DescribeRefused(string(k.Outcome)), KillSurvived, KillPartial, KillFatal) + return nil, 0, fmt.Errorf("reviewer %d kill attempt %d has an out-of-enum outcome (%s); an attempt is %s, %s, or %s", + reviewer, at, termsafe.DescribeRefused(string(k.Outcome)), KillSurvived, KillPartial, KillFatal) } out = append(out, KillAttempt{Attempt: text, Outcome: k.Outcome}) } return out, redactions, nil } +// validateRuling checks the ruling against the reviewers it settles. It is +// required when their verdicts differ, and optional otherwise; given, it names +// who ruled, a registered verdict, and the reason. Whether the ruling was +// right is the person's judgement, and the binary never checks it against the +// verdict the whole gauntlet reached. +func validateRuling(red *recordRedactor, in *Ruling, reviewers []Reviewer) (*Ruling, int, error) { + if in == nil { + for _, r := range reviewers[1:] { + if r.Verdict != reviewers[0].Verdict { + return nil, 0, fmt.Errorf("the reviewers' verdicts differ (%s) and the adversarial-review leg records no ruling — nothing was written; "+ + "a split between reviewers is recorded with the ruling that settled it: \"ruling\": {by, verdict, reason}", + reviewerVerdicts(reviewers)) + } + } + return nil, 0, nil + } + by, n := red.field(in.By) + reason, m := red.field(in.Reason) + out := Ruling{ + By: termsafe.CleanProseLine(by, maxLabelBytes), + Verdict: in.Verdict, + Reason: termsafe.CleanProseLine(reason, maxProseBytes), + } + switch { + case out.By == "": + return nil, 0, errors.New("the ruling does not say who ruled") + case !out.Verdict.Valid(): + return nil, 0, fmt.Errorf("the ruling has an out-of-enum verdict (%s); a ruling is one of %s|%s|%s", + termsafe.DescribeRefused(string(in.Verdict)), VerdictSurvives, VerdictKilled, VerdictReframed) + case out.Reason == "": + return nil, 0, errors.New("the ruling records no reason; the reason is what a later reader weighs the split by") + } + return &out, n + m, nil +} + +// reviewerVerdicts lists each reviewer's verdict by its position, for the +// split refusal. Only core-owned values reach it: a position and a registered +// verdict, never a label. +func reviewerVerdicts(reviewers []Reviewer) string { + parts := make([]string, len(reviewers)) + for i, r := range reviewers { + parts[i] = fmt.Sprintf("reviewer %d %s", i+1, r.Verdict) + } + return strings.Join(parts, ", ") +} + // validateRejected enforces the explicit-empty-marker rule. Omission is refused // because the whole point of the record is that a later session can see WHAT was // turned down; a silently empty list is indistinguishable from a composer that @@ -590,39 +959,241 @@ func validateRejected(red *recordRedactor, p Payload) ([]RejectedAlternative, bo return out, false, redactions, nil } -// resolveCitations is AC 4's gate: every record id the grill cited must name a -// record that exists in this repository. It returns the sorted, deduped set as the -// proof the result carries. +// validateReframing checks the reframing against the verdict: a reframed +// verdict carries a non-empty one, and any other verdict carries none, since a +// reframing beside a surviving or a killed idea leaves a reader to guess which +// of the two the record means. +func validateReframing(red *recordRedactor, p Payload) (string, int, error) { + text, n := red.field(p.Reframing) + reframing := termsafe.CleanProse(text, maxIdeaBytes) + switch { + case p.Verdict == VerdictReframed && reframing == "": + return "", 0, errors.New("the verdict is reframed and carries no reframing — nothing was written; " + + "a reframed verdict records the idea's surviving shape in \"reframing\", because that, not the original wording, is what may graduate") + case p.Verdict != VerdictReframed && reframing != "": + return "", 0, fmt.Errorf("the verdict carries a reframing beside a %s verdict; only a reframed verdict has one, and the two contradict each other", p.Verdict) + } + return reframing, n, nil +} + +// citedText is one place in the verdict a record id may be written, and the +// part of the verdict it belongs to, which a refusal names. +type citedText struct { + where string + text string +} + +// citationPlaces lists every place in the cleaned document a record id may be +// written, each with the part of the verdict it belongs to: every leg's prose +// and sources, the rejected alternatives, the origin sources and the +// reframing. The idea's own words are not among them — it is the subject of +// the record, in the words it entered with — and nor are the grill's search +// terms and places, where a searched-for id that names nothing is the finding. +// A grill hit's record is not here either: it is resolved on its own, exactly, +// because it is the grill's citation proof. +func citationPlaces(v verdictDoc) []citedText { + const ( + research = "the research leg" + grill = "the record-grill leg" + adversarial = "the adversarial-review leg" + ) + var out []citedText + add := func(where string, texts ...string) { + for _, t := range texts { + out = append(out, citedText{where: where, text: t}) + } + } + for _, c := range v.claims { + add(research, c.Claim, c.PrimarySource) + } + for _, h := range v.hits { + add(grill, h.Note) + } + for _, r := range v.reviewers { + add(adversarial, r.Label) + for _, k := range r.KillAttempts { + add(adversarial, k.Attempt) + } + add(adversarial, r.Sources...) + } + if v.ruling != nil { + add(adversarial, v.ruling.By, v.ruling.Reason) + } + for _, r := range v.rejected { + add("the rejected alternatives", r.Alternative, r.WhyRejected) + } + add("the origin sources", v.origins...) + add("the reframing", v.reframing) + return out +} + +// proseIDRe finds a record id of a resolvable family anywhere in running text, +// in any case and with any zero padding. Its families are recordid's cited +// families, so prose is held to exactly the ids a grill hit may cite. The token +// boundaries are checked in code (idsIn), because Go's regexp has no lookbehind. +var proseIDRe = regexp.MustCompile(`(?i)(?:` + citedAlternation() + `)-[0-9]+`) + +// citedAlternation is recordid.CitedFamilies as an alternation. +func citedAlternation() string { + fams := recordid.CitedFamilies() + for i, f := range fams { + fams[i] = regexp.QuoteMeta(f) + } + return strings.Join(fams, "|") +} + +// idsIn returns every record id written in text as a whole token: not preceded +// by a letter, digit or hyphen (`xitd-1`, `pre-itd-1` name nothing), and not +// followed by a letter or digit. A hyphen after the number does not stop it +// being a citation, so `itd-104-ideate` names itd-104, as record-lint reads +// the same shape. +func idsIn(text string) []string { + var out []string + for _, loc := range proseIDRe.FindAllStringIndex(text, -1) { + if loc[0] > 0 && isIDByte(text[loc[0]-1], true) { + continue + } + if loc[1] < len(text) && isIDByte(text[loc[1]], false) { + continue + } + out = append(out, text[loc[0]:loc[1]]) + } + return out +} + +// isIDByte reports whether b would make a matched id part of a longer token: a +// letter or digit on either side, and a hyphen before it. +func isIDByte(b byte, before bool) bool { + switch { + case b >= 'a' && b <= 'z', b >= 'A' && b <= 'Z', b >= '0' && b <= '9': + return true + case b == '-': + return before + } + return false +} + +// resolveCitations is criterion 1's gate (and AC 4's before it): every record id +// the verdict cites, in any leg, must name a record that exists in this +// repository. It returns the sorted, deduped set of the GRILL's citations as the +// proof the result carries; every other id it resolved is proved by the record +// having been written at all. // // It builds ONE resolver for the whole document, so every citation is judged -// against the same view of the record. -func resolveCitations(repoRoot string, hits []GrillHit) ([]string, error) { - if len(hits) == 0 { +// against the same view of the record, and only when the document cites +// something: a verdict that names no id costs no scan. +// +// A prose id is folded to the spelling the resolver keys on (recordid.CanonCitedID) +// before it is looked up, so `ADR-0035` in a kill attempt names adr-35; an id +// that folds to nothing (an all-zero number) names no record and is refused. +func resolveCitations(repoRoot string, v verdictDoc) ([]string, error) { + type found struct{ where, id string } + var prose []found + for _, place := range citationPlaces(v) { + for _, raw := range idsIn(withoutAddresses(place.text)) { + prose = append(prose, found{where: place.where, id: raw}) + } + } + if len(v.hits) == 0 && len(prose) == 0 { return nil, nil } r, err := recordid.NewResolver(repoRoot) if err != nil { return nil, err } - seen := map[string]bool{} - var cited, unresolved []string - for _, h := range hits { - if seen[h.Record] { - continue + var missing []UnresolvedCitation + seen := map[UnresolvedCitation]bool{} + miss := func(where, id string) { + c := UnresolvedCitation{Where: where, ID: id} + if !seen[c] { + seen[c] = true + missing = append(missing, c) } - seen[h.Record] = true + } + cited := map[string]bool{} + for _, h := range v.hits { if !r.Has(h.Record) { - unresolved = append(unresolved, h.Record) + miss("the record-grill leg", h.Record) continue } - cited = append(cited, h.Record) + cited[h.Record] = true + } + for _, f := range prose { + canon := recordid.CanonCitedID(f.id) + switch { + case len(f.id) > maxRecordIDBytes: + // Bounded before it is echoed: the prose cap admits an id of + // thousands of digits, and the refusal describes it instead. + miss(f.where, termsafe.DescribeRefused(f.id)) + case canon == "": + miss(f.where, strings.ToLower(f.id)) + case !r.Has(canon): + miss(f.where, canon) + } + } + if len(missing) > 0 { + return nil, newCitationError(missing) + } + out := make([]string, 0, len(cited)) + for id := range cited { + out = append(out, id) + } + sort.Strings(out) + return out, nil +} + +// resolveSourceKeys checks every `source:` the verdict cites — a claim's +// primary source or a reviewer's — against the sources corpus at dir. Where +// there is no corpus (dir is empty, or names a directory with none in it) there +// is nothing to resolve against, and the key's shape, already checked, is all +// that can be proved; a corpus that exists and cannot be read refuses the run, +// because a key it might not carry would otherwise be recorded as filed. +func resolveSourceKeys(dir string, v verdictDoc) error { + type cite struct{ where, key string } + var keys []cite + for i, c := range v.claims { + if k, ok := strings.CutPrefix(c.PrimarySource, sourceKeyPrefix); ok { + keys = append(keys, cite{where: fmt.Sprintf("claim %d", i+1), key: k}) + } + } + for i, r := range v.reviewers { + for _, s := range r.Sources { + if k, ok := strings.CutPrefix(s, sourceKeyPrefix); ok { + keys = append(keys, cite{where: fmt.Sprintf("reviewer %d", i+1), key: k}) + } + } + } + if len(keys) == 0 || dir == "" { + return nil + } + want := make([]string, len(keys)) + for i, k := range keys { + want[i] = k.key + } + unfiled, err := source.Unfiled(dir, want) + switch { + case errors.Is(err, source.ErrNoCorpus): + return nil + case err != nil: + return fmt.Errorf("the verdict cites a filed source, and the sources corpus cannot be read to resolve it — nothing was written: %w", err) } - if len(unresolved) > 0 { - sort.Strings(unresolved) - return nil, &CitationError{Unresolved: unresolved} + if len(unfiled) == 0 { + return nil + } + missing := map[string]bool{} + for _, k := range unfiled { + missing[k] = true } - sort.Strings(cited) - return cited, nil + var parts []string + for _, k := range keys { + if missing[k.key] { + // The key has the corpus's key grammar (sourceShaped), so it is + // safe to quote, and it is the one handle a refusal names a source by. + parts = append(parts, k.where+" cites "+sourceKeyPrefix+k.key) + } + } + return fmt.Errorf("the verdict cites sources the sources corpus does not carry — nothing was written: %s; "+ + "file each with `abcd library add`, or cite it by its https:// address", strings.Join(parts, ", ")) } // result builds the transport-agnostic outcome from the validated document. @@ -635,14 +1206,20 @@ func (v verdictDoc) result(slug, rel string) Result { Path: rel, GrillHits: len(v.hits), CitedRecords: v.cited, - KillAttempts: len(v.kills), + Searches: len(v.searches), + Reviewers: len(v.reviewers), RejectedAlternatives: len(v.rejected), Redactions: v.redactions, } + for _, r := range v.reviewers { + res.KillAttempts += len(r.KillAttempts) + } for _, c := range v.claims { switch c.Status { case ClaimVerified: res.Claims.Verified++ + case ClaimQuoted: + res.Claims.Quoted++ case ClaimFalsified: res.Claims.Falsified++ case ClaimUnverifiable: diff --git a/internal/core/ideate/record_test.go b/internal/core/ideate/record_test.go index 79c342e4b7..f07a1796c9 100644 --- a/internal/core/ideate/record_test.go +++ b/internal/core/ideate/record_test.go @@ -54,13 +54,15 @@ func gitRepo(t *testing.T) string { } // validPayload is the reference payload: three legs in order, a resolvable -// citation, a verdict, and rejected alternatives. Tests mutate a copy of it so -// each refusal is one deviation from a known-good document. +// citation, a grill that says what it searched, two reviewers who agree, the +// idea's origin sources, a verdict, and rejected alternatives. Tests mutate a +// copy of it so each refusal is one deviation from a known-good document. func validPayload() map[string]any { return map[string]any{ "schema_version": SchemaVersion, "prompt_version": "1.0.0", "idea": "abcd should gate a new idea before it becomes a record entry", + "origin_sources": []any{"https://example.invalid/origin-post"}, "legs": []any{ map[string]any{ "kind": "research", @@ -83,12 +85,33 @@ func validPayload() map[string]any { map[string]any{"record": "itd-104", "relation": "covered", "note": "the same admission gate, already filed"}, map[string]any{"record": "adr-35", "relation": "contradicted"}, }, + "searches": []any{ + map[string]any{ + "terms": []any{"admission gate", "ideate"}, + "places": []any{".abcd/development/intents/", ".abcd/development/decisions/adrs/"}, + "count": 2, + }, + }, }, map[string]any{ "kind": "adversarial-review", - "kill_attempts": []any{ - map[string]any{"attempt": "the gate adds friction to capture", "outcome": "survived"}, - map[string]any{"attempt": "no evidence the legs improve outcomes", "outcome": "partial"}, + "reviewers": []any{ + map[string]any{ + "label": "reviewer-a", + "verdict": "survives", + "kill_attempts": []any{ + map[string]any{"attempt": "the gate adds friction to capture", "outcome": "survived"}, + }, + "sources": []any{"https://example.invalid/paper.pdf"}, + }, + map[string]any{ + "label": "reviewer-b", + "verdict": "survives", + "kill_attempts": []any{ + map[string]any{"attempt": "no evidence the legs improve outcomes", "outcome": "partial"}, + }, + "sources": []any{}, + }, }, }, }, @@ -362,11 +385,18 @@ func TestRecordRefusesEmptyLegContent(t *testing.T) { t.Error("an out-of-enum grill relation was accepted") } }) + t.Run("no-reviewers", func(t *testing.T) { + p := validPayload() + p["legs"].([]any)[2].(map[string]any)["reviewers"] = []any{} + if _, err := Record(root, "s", publicStore, encode(t, p), at); err == nil { + t.Error("an adversarial leg with no reviewer was accepted") + } + }) t.Run("no-kill-attempts", func(t *testing.T) { p := validPayload() - p["legs"].([]any)[2].(map[string]any)["kill_attempts"] = []any{} + p["legs"].([]any)[2].(map[string]any)["reviewers"].([]any)[0].(map[string]any)["kill_attempts"] = []any{} if _, err := Record(root, "s", publicStore, encode(t, p), at); err == nil { - t.Error("an adversarial leg with no kill attempt was accepted") + t.Error("a reviewer with no kill attempt was accepted") } }) t.Run("foreign-leg-field", func(t *testing.T) { @@ -603,8 +633,10 @@ func TestRecordNeutralisesInjectedProse(t *testing.T) { // A tag inside a table CELL swallows the document just as a leading one // does, so the neutralisation belongs in the cleaner, not only in the // block-level escape. - "claim": "the model is fast
every claim verified
", - "primary_source": "", + // A version-2 primary source is an address, a filed key or a record + // id, so the declaration opener rides in the claim cell instead. + "claim": "the model is fast
every claim verified
", + "primary_source": "https://example.invalid/doc", "status": "falsified", }, } diff --git a/internal/core/ideate/render.go b/internal/core/ideate/render.go index 9edcb6793b..5ee68582b1 100644 --- a/internal/core/ideate/render.go +++ b/internal/core/ideate/render.go @@ -38,21 +38,41 @@ func render(v verdictDoc, at time.Time) string { b.WriteString("## The idea\n\n") fmt.Fprintf(&b, "%s\n\n", termsafe.BlockText(v.idea)) + renderOrigins(&b, v) renderResearch(&b, v) renderGrill(&b, v) renderAdversarial(&b, v) renderRejected(&b, v) + renderReframing(&b, v) renderOutcome(&b, v) return b.String() } +// renderOrigins writes the sources the idea came from, which is what a quoted +// claim was checked against. An idea with none says so, in the words the +// explicit empty list declares. +func renderOrigins(b *strings.Builder, v verdictDoc) { + b.WriteString("## Where the idea came from\n\n") + if len(v.origins) == 0 { + b.WriteString("No source: the idea entered the gauntlet with nothing behind it but its own words.\n\n") + return + } + // A list item's text can still open a block ("- # x" is a heading inside + // the item), so each is escaped as the start of a block. + for _, o := range v.origins { + fmt.Fprintf(b, "- %s\n", termsafe.BlockText(o)) + } + b.WriteString("\n") +} + // renderResearch writes leg 1 as the claims table: one row per load-bearing // claim, the primary source it was checked against, and what that established. func renderResearch(b *strings.Builder, v verdictDoc) { b.WriteString("## Leg 1 — Primary-source research\n\n") b.WriteString("Every load-bearing claim checked against its primary source, never a\n") - b.WriteString("secondary citation.\n\n") + b.WriteString("secondary citation. A claim marked quoted was checked only against the\n") + b.WriteString("source the idea came from, which proves the quotation and not the claim.\n\n") b.WriteString("| Claim | Primary source | Finding |\n|---|---|---|\n") for _, c := range v.claims { fmt.Fprintf(b, "| %s | %s | %s |\n", termsafe.TableCell(c.Claim), termsafe.TableCell(c.PrimarySource), c.Status) @@ -69,15 +89,33 @@ func renderGrill(b *strings.Builder, v verdictDoc) { b.WriteString("every id resolved in this repository when the verdict was recorded.\n\n") if len(v.hits) == 0 { b.WriteString("No entry in the record covers, contradicts, or supersedes this idea.\n\n") + } else { + b.WriteString("| Record | Relation | Note |\n|---|---|---|\n") + for _, h := range v.hits { + note := h.Note + if note == "" { + note = "—" + } + fmt.Fprintf(b, "| %s | %s | %s |\n", h.Record, h.Relation, termsafe.TableCell(note)) + } + b.WriteString("\n") + } + renderSearches(b, v) +} + +// renderSearches writes what the grill searched, so a reader can tell an empty +// grill from a broken one: the terms, the places, and how many entries each +// search returned. A grill with hits may keep no search record, and says so. +func renderSearches(b *strings.Builder, v verdictDoc) { + b.WriteString("### Searches\n\n") + if len(v.searches) == 0 { + b.WriteString("The grill recorded no searches; its findings are the cited hits above.\n\n") return } - b.WriteString("| Record | Relation | Note |\n|---|---|---|\n") - for _, h := range v.hits { - note := h.Note - if note == "" { - note = "—" - } - fmt.Fprintf(b, "| %s | %s | %s |\n", h.Record, h.Relation, termsafe.TableCell(note)) + b.WriteString("| Terms | Places | Count |\n|---|---|---|\n") + for _, s := range v.searches { + fmt.Fprintf(b, "| %s | %s | %d |\n", + termsafe.TableCell(strings.Join(s.Terms, "; ")), termsafe.TableCell(strings.Join(s.Places, "; ")), *s.Count) } b.WriteString("\n") } @@ -89,11 +127,36 @@ func renderAdversarial(b *strings.Builder, v verdictDoc) { b.WriteString("## Leg 3 — Adversarial review\n\n") b.WriteString("Conducted fresh-context and off-policy by an evaluator that did not carry\n") b.WriteString("out the research and received the idea as an artefact of unknown\n") - b.WriteString("authorship — the evaluator-outside-the-loop principle applied to ideas.\n\n") - for _, k := range v.kills { - fmt.Fprintf(b, "- **%s** — %s\n", k.Outcome, k.Attempt) + b.WriteString("authorship — the evaluator-outside-the-loop principle applied to ideas.\n") + b.WriteString("Each reviewer's attempts, verdict and sources are kept under its own label.\n\n") + // The label sits after a fixed prefix, so it cannot open a block of its own. + for _, r := range v.reviewers { + fmt.Fprintf(b, "### Reviewer %s — %s\n\n", r.Label, r.Verdict) + for _, k := range r.KillAttempts { + fmt.Fprintf(b, "- **%s** — %s\n", k.Outcome, k.Attempt) + } + if len(r.Sources) == 0 { + b.WriteString("\nSources consulted: none named.\n\n") + } else { + fmt.Fprintf(b, "\nSources consulted: %s.\n\n", strings.Join(r.Sources, ", ")) + } } - b.WriteString("\n") + renderRuling(b, v) +} + +// renderRuling writes the ruling that settled a split between reviewers. Where +// the reviewers agreed and nobody ruled, the agreement is stated, so a reader +// can tell "no split" from "a split nobody settled" (which the validator +// refuses). +func renderRuling(b *strings.Builder, v verdictDoc) { + if v.ruling == nil { + if len(v.reviewers) > 1 { + b.WriteString("The reviewers agree, so no ruling was needed.\n\n") + } + return + } + b.WriteString("### Ruling\n\n") + fmt.Fprintf(b, "**%s**, ruled by %s — %s\n\n", v.ruling.Verdict, v.ruling.By, v.ruling.Reason) } // renderRejected writes the alternatives turned down. A deliberate absence is @@ -114,6 +177,16 @@ func renderRejected(b *strings.Builder, v verdictDoc) { b.WriteString("\n") } +// renderReframing writes a reframed idea's surviving shape in a section of its +// own, which is what the outcome below tells a later reader to graduate. +func renderReframing(b *strings.Builder, v verdictDoc) { + if v.verdict != VerdictReframed { + return + } + b.WriteString("## The reframing\n\n") + fmt.Fprintf(b, "%s\n\n", termsafe.BlockText(v.reframing)) +} + // renderOutcome states what the verdict means for the idea's future, so the // record answers the question its next reader arrives with. func renderOutcome(b *strings.Builder, v verdictDoc) { diff --git a/internal/core/ideate/v2_test.go b/internal/core/ideate/v2_test.go new file mode 100644 index 0000000000..965372c153 --- /dev/null +++ b/internal/core/ideate/v2_test.go @@ -0,0 +1,524 @@ +package ideate + +// v2_test.go pins spc-2610142138047122 (itd-2610141646454347): schema version +// 2 of the verdict, in which every leg proves what it records. Each test below +// is one of the intent's acceptance criteria, in order; the criterion number is +// in the test's comment. + +import ( + "bytes" + "encoding/json" + "errors" + "os" + "path/filepath" + "regexp" + "strings" + "testing" + + "github.com/intentdriven/abcd/internal/abcdhome" + "github.com/intentdriven/abcd/internal/core/source" +) + +// leg returns the i-th leg of a test payload for mutation. +func leg(p map[string]any, i int) map[string]any { return p["legs"].([]any)[i].(map[string]any) } + +// reviewer returns the i-th reviewer of a test payload's adversarial leg. +func reviewer(p map[string]any, i int) map[string]any { + return leg(p, 2)["reviewers"].([]any)[i].(map[string]any) +} + +// claim returns the i-th claim of a test payload's research leg. +func claim(p map[string]any, i int) map[string]any { + return leg(p, 0)["claims"].([]any)[i].(map[string]any) +} + +// TestV2RefusesAnUnresolvedIDInAnyLeg is criterion 1: a record id that does +// not resolve, written anywhere in the verdict's legs or its alternatives, is +// refused, and the refusal names where it was and the id. +func TestV2RefusesAnUnresolvedIDInAnyLeg(t *testing.T) { + root := seedRepo(t) + cases := map[string]struct { + mutate func(p map[string]any) + where string + id string + }{ + "primary source": {func(p map[string]any) { claim(p, 0)["primary_source"] = "itd-9999" }, "research", "itd-9999"}, + "claim text": {func(p map[string]any) { claim(p, 1)["claim"] = "spc-9999 already says so" }, "research", "spc-9999"}, + "grill record": {func(p map[string]any) { + leg(p, 1)["hits"] = []any{map[string]any{"record": "iss-9999", "relation": "covered"}} + }, "record-grill", "iss-9999"}, + "grill note": {func(p map[string]any) { + leg(p, 1)["hits"].([]any)[0].(map[string]any)["note"] = "as adr-9999 put it" + }, "record-grill", "adr-9999"}, + "kill attempt": {func(p map[string]any) { + reviewer(p, 1)["kill_attempts"].([]any)[0].(map[string]any)["attempt"] = "itd-9998 already killed this" + }, "adversarial-review", "itd-9998"}, + "id glued to an address": {func(p map[string]any) { + reviewer(p, 1)["kill_attempts"].([]any)[0].(map[string]any)["attempt"] = "already settled (https://example.invalid/thread;itd-9997)" + }, "adversarial-review", "itd-9997"}, + "reviewer source": {func(p map[string]any) { reviewer(p, 0)["sources"] = []any{"spc-9998"} }, "adversarial-review", "spc-9998"}, + "reviewer label": {func(p map[string]any) { reviewer(p, 0)["label"] = "the reviewer of itd-9996" }, "adversarial-review", "itd-9996"}, + "ruling by": {func(p map[string]any) { + leg(p, 2)["ruling"] = map[string]any{"by": "the advisor named in dcn-9995", "verdict": "survives", "reason": "both agree"} + }, "adversarial-review", "dcn-9995"}, + "rejected alternative": {func(p map[string]any) { + p["rejected_alternatives"] = []any{map[string]any{"alternative": "fold it into iss-9998", "why_rejected": "a reason"}} + }, "rejected alternatives", "iss-9998"}, + "why rejected": {func(p map[string]any) { + p["rejected_alternatives"] = []any{map[string]any{"alternative": "something", "why_rejected": "ADR-9998 rules it out"}} + }, "rejected alternatives", "adr-9998"}, + } + for name, tc := range cases { + t.Run(name, func(t *testing.T) { + p := validPayload() + tc.mutate(p) + _, err := Record(root, "an-uncited-idea", publicStore, encode(t, p), at) + var cite *CitationError + if !errors.As(err, &cite) { + t.Fatalf("Record error = %v, want a *CitationError", err) + } + msg := err.Error() + if !strings.Contains(msg, tc.where) || !strings.Contains(strings.ToLower(msg), tc.id) { + t.Errorf("the refusal does not name the leg %q and the id %q: %s", tc.where, tc.id, msg) + } + }) + } + assertNothingWritten(t, root) + + // Every spelling the record writes a resolvable id in is resolved, not + // refused: a padded, upper-case ADR handle in a kill attempt names adr-35. + p := validPayload() + reviewer(p, 0)["kill_attempts"].([]any)[0].(map[string]any)["attempt"] = "ADR-0035 and itd-104-ideate both bear on this" + claim(p, 0)["primary_source"] = "itd-104" + if _, err := Record(root, "a-cited-idea", publicStore, encode(t, p), at); err != nil { + t.Errorf("resolvable ids in prose were refused: %v", err) + } +} + +// TestV2RefusesAPrimarySourceOfNoKnownShape is criterion 2: a primary source +// is an https:// address, a filed source key, or a record id, and nothing else. +func TestV2RefusesAPrimarySourceOfNoKnownShape(t *testing.T) { + root := seedRepo(t) + for _, bad := range []string{ + "a remark heard in passing", + "http://example.invalid/plain", + "file:///tmp/notes.md", + "https://", + "https://example.invalid/a remark", + "https://a remark heard in passing", + "https://example.invalid/a remark", + "source:", + "source:Not A Key", + "itd-104 and more", + } { + p := validPayload() + claim(p, 0)["primary_source"] = bad + _, err := Record(root, "a-sourced-idea", publicStore, encode(t, p), at) + if err == nil { + t.Errorf("primary source %q was accepted", bad) + continue + } + if !strings.Contains(err.Error(), "claim 1") || !strings.Contains(err.Error(), "https://") { + t.Errorf("primary source %q: the refusal does not name the claim and the accepted shapes: %v", bad, err) + } + } + assertNothingWritten(t, root) + + // With no corpus on the machine a well-formed key is accepted: there is + // nothing to resolve it against, and its shape is still checked. + // An address whose path happens to hold an id-shaped segment is an + // address, not a citation of a record here. + for i, good := range []string{"https://example.invalid/paper.pdf", "itd-104", "adr-35", "source:survey2026", + "https://example.invalid/org/adr-tools/doc/adr/adr-0007-use-markdown.md"} { + p := validPayload() + claim(p, 0)["primary_source"] = good + if _, err := Record(root, "a-sourced-idea-"+string(rune('a'+i)), publicStore, encode(t, p), at); err != nil { + t.Errorf("primary source %q was refused: %v", good, err) + } + } +} + +// TestV2ResolvesASourceKeyAgainstThePresentCorpus is criterion 2's corpus +// half: where the machine has a sources corpus, a source: must name an +// entry in it. +func TestV2ResolvesASourceKeyAgainstThePresentCorpus(t *testing.T) { + root := seedRepo(t) + corpus := filedCorpus(t, "survey2026") + store := publicStore + store.SourcesDir = corpus + + p := validPayload() + claim(p, 0)["primary_source"] = "source:survey2026" + if _, err := Record(root, "a-filed-source", store, encode(t, p), at); err != nil { + t.Fatalf("a filed source key was refused: %v", err) + } + + p = validPayload() + claim(p, 0)["primary_source"] = "source:never-filed" + _, err := Record(root, "an-unfiled-source", store, encode(t, p), at) + if err == nil { + t.Fatal("a key the corpus does not carry was accepted") + } + if !strings.Contains(err.Error(), "source:never-filed") || !strings.Contains(err.Error(), "claim 1") { + t.Errorf("the refusal does not name the claim and the key: %v", err) + } +} + +// filedCorpus initialises a sources corpus under a temporary home with one +// public entry, key, and returns its directory. +func filedCorpus(t *testing.T, key string) string { + t.Helper() + home := t.TempDir() + t.Setenv("GIT_AUTHOR_NAME", "Fixture") + t.Setenv("GIT_AUTHOR_EMAIL", "fixture@example.invalid") + t.Setenv("GIT_COMMITTER_NAME", "Fixture") + t.Setenv("GIT_COMMITTER_EMAIL", "fixture@example.invalid") + dir := abcdhome.Path(home, "sources") + if _, err := source.Init(dir); err != nil { + t.Fatalf("source.Init: %v", err) + } + doc := filepath.Join(t.TempDir(), "survey.md") + if err := os.WriteFile(doc, []byte("# survey\nbody\n"), 0o600); err != nil { + t.Fatal(err) + } + if _, err := source.Add(source.AddRequest{Corpus: dir, Key: key, Title: "A survey", Class: source.ClassPublic, Original: doc}); err != nil { + t.Fatalf("source.Add: %v", err) + } + return dir +} + +// TestV2RefusesAGrillWithNoHitsAndNoSearches is criterion 3: an empty grill +// must say what it searched, so it can be told from a broken one. +func TestV2RefusesAGrillWithNoHitsAndNoSearches(t *testing.T) { + root := seedRepo(t) + p := validPayload() + leg(p, 1)["hits"] = []any{} + delete(leg(p, 1), "searches") + _, err := Record(root, "an-unsearched-idea", publicStore, encode(t, p), at) + if err == nil { + t.Fatal("a grill with no hits and no searches was accepted") + } + if !strings.Contains(err.Error(), "searches") { + t.Errorf("the refusal does not name the missing searches: %v", err) + } + + for name, search := range map[string]map[string]any{ + "no-count": {"terms": []any{"gate"}, "places": []any{"intents"}}, + "negative-count": {"terms": []any{"gate"}, "places": []any{"intents"}, "count": -1}, + "no-terms": {"terms": []any{}, "places": []any{"intents"}, "count": 0}, + "blank-term": {"terms": []any{" "}, "places": []any{"intents"}, "count": 0}, + "no-places": {"terms": []any{"gate"}, "count": 0}, + } { + p := validPayload() + leg(p, 1)["searches"] = []any{search} + if _, err := Record(root, "a-bad-search", publicStore, encode(t, p), at); err == nil { + t.Errorf("%s: the search was accepted", name) + } + } + assertNothingWritten(t, root) + + // An empty grill that says what it searched is a finding, and the record + // shows the searches it rests on. + p = validPayload() + leg(p, 1)["hits"] = []any{} + leg(p, 1)["searches"] = []any{map[string]any{ + "terms": []any{"admission gate"}, "places": []any{".abcd/development/intents/"}, "count": 0, + }} + res, err := Record(root, "a-searched-idea", publicStore, encode(t, p), at) + if err != nil { + t.Fatalf("an empty grill with searches was refused: %v", err) + } + if res.Searches != 1 { + t.Errorf("Searches = %d, want 1", res.Searches) + } + body := readFile(t, root, res.Path) + for _, want := range []string{"| Terms | Places | Count |", "| admission gate | .abcd/development/intents/ | 0 |"} { + if !strings.Contains(body, want) { + t.Errorf("the record is missing %q:\n%s", want, body) + } + } +} + +// TestV2RefusesAReframedVerdictWithoutAReframing is criterion 4. +func TestV2RefusesAReframedVerdictWithoutAReframing(t *testing.T) { + root := seedRepo(t) + for name, reframing := range map[string]any{"absent": nil, "blank": " "} { + p := validPayload() + p["verdict"] = "reframed" + if reframing != nil { + p["reframing"] = reframing + } + _, err := Record(root, "a-reframed-idea", publicStore, encode(t, p), at) + if err == nil { + t.Errorf("%s: a reframed verdict with no reframing was accepted", name) + } else if !strings.Contains(err.Error(), "reframing") { + t.Errorf("%s: the refusal does not name the reframing: %v", name, err) + } + } + // A reframing beside a verdict that is not reframed contradicts it. + p := validPayload() + p["reframing"] = "gate only ideas that would mint an intent" + if _, err := Record(root, "a-contradicted-idea", publicStore, encode(t, p), at); err == nil { + t.Error("a reframing beside a surviving verdict was accepted") + } + assertNothingWritten(t, root) + + p = validPayload() + p["verdict"] = "reframed" + p["reframing"] = "gate only ideas that would mint an intent" + res, err := Record(root, "a-reframed-idea", publicStore, encode(t, p), at) + if err != nil { + t.Fatalf("a reframed verdict with its reframing was refused: %v", err) + } + body := readFile(t, root, res.Path) + if !strings.Contains(body, "## The reframing\n\ngate only ideas that would mint an intent\n") { + t.Errorf("the record does not carry the reframing in its own section:\n%s", body) + } +} + +// TestV2KeepsEachReviewerUnderItsLabel is criterion 5: two reviewers' kill +// attempts, verdicts and sources are each kept under that reviewer. +func TestV2KeepsEachReviewerUnderItsLabel(t *testing.T) { + root := seedRepo(t) + res, err := Record(root, "a-reviewed-idea", publicStore, encode(t, validPayload()), at) + if err != nil { + t.Fatalf("Record: %v", err) + } + if res.Reviewers != 2 || res.KillAttempts != 2 { + t.Errorf("Reviewers = %d, KillAttempts = %d; want 2 and 2", res.Reviewers, res.KillAttempts) + } + body := readFile(t, root, res.Path) + a := strings.Index(body, "### Reviewer reviewer-a — survives") + b := strings.Index(body, "### Reviewer reviewer-b — survives") + if a < 0 || b < 0 || b < a { + t.Fatalf("the record does not hold a section per reviewer, in order:\n%s", body) + } + first, second := body[a:b], body[b:] + for _, want := range []string{"the gate adds friction to capture", "https://example.invalid/paper.pdf"} { + if !strings.Contains(first, want) || strings.Contains(second, want) { + t.Errorf("%q is not kept under reviewer-a alone:\n%s", want, body) + } + } + if !strings.Contains(second, "no evidence the legs improve outcomes") || strings.Contains(first, "no evidence the legs improve outcomes") { + t.Errorf("reviewer-b's attempt is not kept under reviewer-b alone:\n%s", body) + } + if !strings.Contains(second, "Sources consulted: none named.") { + t.Errorf("reviewer-b's empty sources are not stated:\n%s", body) + } + + for name, mutate := range map[string]func(p map[string]any){ + "duplicate-label": func(p map[string]any) { reviewer(p, 1)["label"] = "reviewer-a" }, + "blank-label": func(p map[string]any) { reviewer(p, 0)["label"] = " " }, + "no-verdict": func(p map[string]any) { delete(reviewer(p, 0), "verdict") }, + "bad-source": func(p map[string]any) { reviewer(p, 0)["sources"] = []any{"what I remembered"} }, + } { + p := validPayload() + mutate(p) + if _, err := Record(root, "a-misreviewed-idea", publicStore, encode(t, p), at); err == nil { + t.Errorf("%s: the reviewer was accepted", name) + } + } +} + +// TestV2RefusesASplitWithoutARuling is criterion 6: reviewers whose verdicts +// differ need the ruling that settled them, naming who ruled. +func TestV2RefusesASplitWithoutARuling(t *testing.T) { + root := seedRepo(t) + split := func() map[string]any { + p := validPayload() + reviewer(p, 1)["verdict"] = "killed" + return p + } + _, err := Record(root, "a-split-idea", publicStore, encode(t, split()), at) + if err == nil { + t.Fatal("a split between reviewers with no ruling was accepted") + } + if !strings.Contains(err.Error(), "ruling") { + t.Errorf("the refusal does not name the ruling: %v", err) + } + for name, ruling := range map[string]map[string]any{ + "no-by": {"verdict": "survives", "reason": "the fatal attempt rested on a misread claim"}, + "no-reason": {"by": "the product thinker", "verdict": "survives"}, + "bad-verdict": {"by": "the product thinker", "verdict": "maybe", + "reason": "the fatal attempt rested on a misread claim"}, + } { + p := split() + leg(p, 2)["ruling"] = ruling + if _, err := Record(root, "a-split-idea", publicStore, encode(t, p), at); err == nil { + t.Errorf("%s: the ruling was accepted", name) + } + } + assertNothingWritten(t, root) + + p := split() + leg(p, 2)["ruling"] = map[string]any{ + "by": "the product thinker", "verdict": "survives", "reason": "the fatal attempt rested on a misread claim", + } + res, err := Record(root, "a-split-idea", publicStore, encode(t, p), at) + if err != nil { + t.Fatalf("a split with its ruling was refused: %v", err) + } + body := readFile(t, root, res.Path) + for _, want := range []string{"### Ruling", "**survives**, ruled by the product thinker", "the fatal attempt rested on a misread claim"} { + if !strings.Contains(body, want) { + t.Errorf("the record is missing %q:\n%s", want, body) + } + } +} + +// TestV2RefusesAVerifiedSelfQuotation is criterion 7: a claim checked only +// against the idea's own source is quoted, never verified. +func TestV2RefusesAVerifiedSelfQuotation(t *testing.T) { + root := seedRepo(t) + p := validPayload() + claim(p, 0)["primary_source"] = "https://example.invalid/origin-post" + _, err := Record(root, "a-self-quoted-idea", publicStore, encode(t, p), at) + if err == nil { + t.Fatal("a verified claim resting on the idea's own source was accepted") + } + if !strings.Contains(err.Error(), "claim 1") || !strings.Contains(err.Error(), "quoted") { + t.Errorf("the refusal does not name the claim and the quoted status: %v", err) + } + // The same address written another way is still the idea's own source. + for i, same := range []string{ + "https://example.invalid/origin-post/", + "https://EXAMPLE.invalid/origin-post", + "https://example.invalid/origin-post#the-claim", + } { + p = validPayload() + claim(p, 0)["primary_source"] = same + if _, err := Record(root, "a-self-quoted-idea-"+string(rune('a'+i)), publicStore, encode(t, p), at); err == nil { + t.Errorf("a verified claim resting on %q, the idea's own source written another way, was accepted", same) + } + } + // An origin is free text, so the source in it may come with words around + // it, and a record id in it may be padded. + for i, c := range []struct{ origin, src string }{ + {"https://example.invalid/origin-post the post the idea came from", "https://example.invalid/origin-post"}, + {"https://example.invalid/wiki/Mercury_(planet) the article", "https://example.invalid/wiki/Mercury_(planet)"}, + {"ADR-0035", "adr-35"}, + } { + p = validPayload() + p["origin_sources"] = []any{c.origin} + claim(p, 0)["primary_source"] = c.src + if _, err := Record(root, "a-self-quoted-origin-"+string(rune('a'+i)), publicStore, encode(t, p), at); err == nil { + t.Errorf("a verified claim resting on %q, named inside the origin %q, was accepted", c.src, c.origin) + } + } + // The origin sources are stated explicitly, so the check cannot be skipped + // by leaving them out. + p = validPayload() + delete(p, "origin_sources") + if _, err := Record(root, "a-sourceless-idea", publicStore, encode(t, p), at); err == nil { + t.Error("a verdict that does not say where the idea came from was accepted") + } + assertNothingWritten(t, root) + + p = validPayload() + claim(p, 0)["primary_source"] = "https://example.invalid/origin-post" + claim(p, 0)["status"] = "quoted" + res, err := Record(root, "a-quoted-idea", publicStore, encode(t, p), at) + if err != nil { + t.Fatalf("a quoted claim was refused: %v", err) + } + if res.Claims.Quoted != 1 || res.Claims.Verified != 0 || res.Claims.Total() != 2 { + t.Errorf("Claims = %+v; want one quoted, none verified, two in all", res.Claims) + } + body := readFile(t, root, res.Path) + for _, want := range []string{"| quoted |", "- https://example.invalid/origin-post"} { + if !strings.Contains(body, want) { + t.Errorf("the record is missing %q:\n%s", want, body) + } + } + + // A record mentioned in passing in an origin is not where the idea came + // from: checking the record itself is a primary check. + p = validPayload() + p["origin_sources"] = []any{"a hallway chat questioning adr-35"} + claim(p, 0)["primary_source"] = "adr-35" + if _, err := Record(root, "a-record-checked-idea", publicStore, encode(t, p), at); err != nil { + t.Errorf("a verified claim checked against a record an origin only mentions was refused: %v", err) + } + // An idea with no source behind it says so with an empty list. + p = validPayload() + p["origin_sources"] = []any{} + if _, err := Record(root, "an-unsourced-idea", publicStore, encode(t, p), at); err != nil { + t.Errorf("an explicitly empty origin_sources was refused: %v", err) + } +} + +// TestV2RefusesAVersionOnePayload is criterion 8. +func TestV2RefusesAVersionOnePayload(t *testing.T) { + if SchemaVersion != 2 { + t.Fatalf("SchemaVersion = %d, want 2", SchemaVersion) + } + root := seedRepo(t) + p := validPayload() + p["schema_version"] = 1 + _, err := Record(root, "an-old-idea", publicStore, encode(t, p), at) + if err == nil { + t.Fatal("a version-1 payload was accepted") + } + if !strings.Contains(err.Error(), "version 2") { + t.Errorf("the refusal does not name version 2: %v", err) + } + + // A payload an older command page composed is not a version-2 shape with + // its number changed: its adversarial leg carries kill_attempts directly + // and it has no origin_sources. It gets the same refusal, not a decoder + // error about a field this version moved. + v1 := `{"schema_version": 1, "prompt_version": "1.0.0", + "idea": "abcd should gate a new idea before it becomes a record entry", + "legs": [ + {"kind": "research", "claims": [{"claim": "fresh-context evaluation is the only measured debiasing effect", + "primary_source": "https://example.invalid/paper.pdf", "status": "verified"}]}, + {"kind": "record-grill", "hits": [{"record": "itd-104", "relation": "covered"}]}, + {"kind": "adversarial-review", "kill_attempts": [{"attempt": "the gate adds friction to capture", "outcome": "survived"}]}], + "verdict": "survives", + "rejected_alternatives": [{"alternative": "make ideate a blocking pre-capture gate", "why_rejected": "capture friction stays at one line"}]}` + _, err = Record(root, "an-old-idea", publicStore, []byte(v1), at) + if err == nil { + t.Fatal("a version-1 document was accepted") + } + if !strings.Contains(err.Error(), "version 2") || strings.Contains(err.Error(), "unknown field") { + t.Errorf("a version-1 document is not refused by the message naming version 2: %v", err) + } + assertNothingWritten(t, root) +} + +// TestCommandPageGivesEachReviewerOneIdeaPerContext is criterion 9, read off +// the command page itself: leg 3 tells the runner to give each reviewer one +// idea per context, and the payload example is the shape this binary decodes. +func TestCommandPageGivesEachReviewerOneIdeaPerContext(t *testing.T) { + data, err := os.ReadFile(filepath.Join("..", "..", "..", "commands", "ideate.md")) + if err != nil { + t.Fatal(err) + } + page := string(data) + start := strings.Index(page, "### Leg 3") + end := strings.Index(page, "## Record the verdict") + if start < 0 || end < start { + t.Fatal("the command page has no leg 3 section before its record section") + } + if !strings.Contains(strings.ToLower(page[start:end]), "one idea per context") { + t.Errorf("leg 3 does not tell the runner to give each reviewer one idea per context:\n%s", page[start:end]) + } + + m := regexp.MustCompile("(?s)```json\n(.*?)\n```").FindStringSubmatch(page) + if m == nil { + t.Fatal("the command page carries no JSON payload example") + } + dec := json.NewDecoder(bytes.NewReader([]byte(m[1]))) + dec.DisallowUnknownFields() + var p Payload + if err := dec.Decode(&p); err != nil { + t.Fatalf("the page's payload example does not decode as this binary's payload: %v", err) + } + if p.SchemaVersion != SchemaVersion { + t.Errorf("the page's example is schema_version %d, want %d", p.SchemaVersion, SchemaVersion) + } + if p.OriginSources == nil || len(p.Legs) != 3 || len(p.Legs[1].Searches) == 0 || + len(p.Legs[2].Reviewers) < 2 || p.Legs[2].Ruling == nil { + t.Errorf("the page's example does not show origin_sources, searches, two reviewers and a ruling: %+v", p) + } +} diff --git a/internal/core/source/source.go b/internal/core/source/source.go index eeecc99fc5..9bce37c7b9 100644 --- a/internal/core/source/source.go +++ b/internal/core/source/source.go @@ -242,6 +242,31 @@ func (c *Corpus) Lookup(key string) (Entry, bool) { return Entry{}, false } +// ValidKey reports whether key has a source key's shape. It is exported for a +// caller that names a source by key in a document of its own (an ideate +// verdict's `source:`), so that caller bounds the key by the same grammar +// `add` mints it in. +func ValidKey(key string) bool { return keyRe.MatchString(key) } + +// Unfiled returns the keys, of those given, that the corpus at dir carries no +// entry for, in the order given. An absent corpus is ErrNoCorpus, so a caller +// can tell "nothing to resolve against" from "not filed"; a corpus that exists +// and cannot be read is the error that says why. The corpus is read once for +// every key, and nothing is written or printed. +func Unfiled(dir string, keys []string) ([]string, error) { + c, err := loadCorpus(dir) + if err != nil { + return nil, err + } + var out []string + for _, k := range keys { + if _, ok := c.Lookup(k); !ok { + out = append(out, k) + } + } + return out, nil +} + // Class is the class a key's folder declares, or "" when it has none or two. func (c *Corpus) Class(key string) string { if f := c.folders[key]; len(f) == 1 { diff --git a/internal/surface/cli/ideate.go b/internal/surface/cli/ideate.go index 1c0ea9ad0a..5bd3f2f03a 100644 --- a/internal/surface/cli/ideate.go +++ b/internal/surface/cli/ideate.go @@ -24,6 +24,7 @@ import ( "github.com/intentdriven/abcd/internal/abcdhome" "github.com/intentdriven/abcd/internal/core/ideate" + "github.com/intentdriven/abcd/internal/core/source" "github.com/intentdriven/abcd/internal/gitutil" "github.com/intentdriven/abcd/internal/termsafe" "github.com/spf13/cobra" @@ -183,6 +184,12 @@ func ideateStore(repoRoot string, private, public bool) (ideate.Store, error) { "and a verdict is filed under the repository's root commit; commit once, then record (nothing written)"} } store := ideate.Store{Visibility: vis, RootSHA: sha} + // The corpus a `source:` is resolved against is the person's default + // one. A home that cannot be resolved leaves it empty, which the core reads + // as no corpus: a well-formed key is then accepted on its shape alone. + if dir, err := source.DefaultDir(); err == nil { + store.SourcesDir = dir + } if vis == ideate.VisibilityPrivate { // The error is not checked here: an unresolved home reaches the core as // "", and the core's refusal names it in the same words for every caller. @@ -216,14 +223,15 @@ func renderIdeateResult(res ideate.Result) string { } else { out += fmt.Sprintf(" pointer: %s\n", res.DecisionPath) } - out += fmt.Sprintf(" claims: %d (verified %d · falsified %d · unverifiable %d)\n", - res.Claims.Total(), res.Claims.Verified, res.Claims.Falsified, res.Claims.Unverifiable) + out += fmt.Sprintf(" claims: %d (verified %d · quoted %d · falsified %d · unverifiable %d)\n", + res.Claims.Total(), res.Claims.Verified, res.Claims.Quoted, res.Claims.Falsified, res.Claims.Unverifiable) if len(res.CitedRecords) == 0 { - out += " grill: no entry in the record covers, contradicts, or supersedes this\n" + out += fmt.Sprintf(" grill: no entry in the record covers, contradicts, or supersedes this (%d search(es) recorded)\n", res.Searches) } else { - out += fmt.Sprintf(" grill: %d hit(s), all cited ids resolved: %v\n", res.GrillHits, res.CitedRecords) + out += fmt.Sprintf(" grill: %d hit(s), all cited ids resolved: %v (%d search(es) recorded)\n", + res.GrillHits, res.CitedRecords, res.Searches) } - out += fmt.Sprintf(" adversary: %d kill attempt(s)\n", res.KillAttempts) + out += fmt.Sprintf(" adversary: %d reviewer(s), %d kill attempt(s)\n", res.Reviewers, res.KillAttempts) if res.RejectedAlternatives == 0 { out += " rejected: none, recorded explicitly\n" } else { diff --git a/internal/surface/cli/ideate_surface_test.go b/internal/surface/cli/ideate_surface_test.go index 415cb11167..4732c28c2d 100644 --- a/internal/surface/cli/ideate_surface_test.go +++ b/internal/surface/cli/ideate_surface_test.go @@ -12,6 +12,7 @@ import ( "github.com/intentdriven/abcd/internal/core/decisions" "github.com/intentdriven/abcd/internal/core/ideate" + "github.com/intentdriven/abcd/internal/core/source" "github.com/intentdriven/abcd/internal/gittest" ) @@ -50,20 +51,31 @@ func ideateRepo(t *testing.T) string { // ideateVerdictJSON is a valid verdict document; cited is the record id the grill // hit cites, so a test can swap in an id that does not resolve. func ideateVerdictJSON(cited string) string { + return ideateVerdictJSONFrom(cited, "https://example.invalid/paper") +} + +// ideateVerdictJSONFrom is ideateVerdictJSON with the research claim's primary +// source given too, so a test can cite a filed source key. +func ideateVerdictJSONFrom(cited, primary string) string { return `{ - "schema_version": 1, + "schema_version": 2, "prompt_version": "1.0.0", "idea": "abcd should gate a new idea before it becomes a record entry", + "origin_sources": [], "legs": [ {"kind": "research", "claims": [ {"claim": "fresh-context evaluation is the measured debiasing effect", - "primary_source": "https://example.invalid/paper", "status": "verified"} + "primary_source": "` + primary + `", "status": "verified"} ]}, {"kind": "record-grill", "hits": [ {"record": "` + cited + `", "relation": "covered", "note": "already filed"} + ], "searches": [ + {"terms": ["admission gate"], "places": [".abcd/development/intents/"], "count": 1} ]}, - {"kind": "adversarial-review", "kill_attempts": [ - {"attempt": "the gauntlet adds friction", "outcome": "survived"} + {"kind": "adversarial-review", "reviewers": [ + {"label": "reviewer-a", "verdict": "survives", "sources": [], "kill_attempts": [ + {"attempt": "the gauntlet adds friction", "outcome": "survived"} + ]} ]} ], "verdict": "survives", @@ -140,6 +152,14 @@ func TestIdeateRecordRefusalsExitTwo(t *testing.T) { []string{"ideate", "record", "../../../etc/passwd", "--public", "--verdict-json", writeVerdict(t, ideateVerdictJSON("itd-104"))}, "slug", }, + { + // A host reading an older command page composes version 1; the + // refusal names the version to compose instead. + "version-1-payload", + []string{"ideate", "record", "the-ideate-gate", "--public", "--verdict-json", + writeVerdict(t, strings.Replace(ideateVerdictJSON("itd-104"), `"schema_version": 2`, `"schema_version": 1`, 1))}, + "version 2", + }, { "verdict-json-required", []string{"ideate", "record", "the-ideate-gate", "--public"}, @@ -369,3 +389,45 @@ func TestIdeateRecordRefusesAShallowCheckout(t *testing.T) { t.Errorf("a refused run laid the private store: %v", err) } } + +// TestIdeateRecordResolvesAFiledSourceAgainstTheDefaultCorpus is the front +// door's half of spc-2610142138047122's source check: a `source:` primary +// source is resolved against the person's default sources corpus when one +// exists, and accepted on its shape when none does. +func TestIdeateRecordResolvesAFiledSourceAgainstTheDefaultCorpus(t *testing.T) { + repo := ideateRepo(t) + t.Chdir(repo) + + // No corpus under this HOME: a well-formed key is accepted. + runCLI(t, "ideate", "record", "a-filed-idea", "--public", + "--verdict-json", writeVerdict(t, ideateVerdictJSONFrom("itd-104", "source:survey2026")), "--json") + + for k, v := range map[string]string{ + "GIT_AUTHOR_NAME": "Fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", + "GIT_COMMITTER_NAME": "Fixture", "GIT_COMMITTER_EMAIL": "fixture@example.invalid", + } { + t.Setenv(k, v) + } + dir := abcdhome.Path(os.Getenv("HOME"), "sources") + if _, err := source.Init(dir); err != nil { + t.Fatalf("source.Init: %v", err) + } + doc := filepath.Join(t.TempDir(), "survey.md") + if err := os.WriteFile(doc, []byte("# survey\n"), 0o600); err != nil { + t.Fatal(err) + } + if _, err := source.Add(source.AddRequest{Corpus: dir, Key: "survey2026", Title: "A survey", Class: source.ClassPublic, Original: doc}); err != nil { + t.Fatalf("source.Add: %v", err) + } + runCLI(t, "ideate", "record", "a-second-filed-idea", "--public", + "--verdict-json", writeVerdict(t, ideateVerdictJSONFrom("itd-104", "source:survey2026")), "--json") + + out, err := runCLIErr(t, "ideate", "record", "an-unfiled-idea", "--public", + "--verdict-json", writeVerdict(t, ideateVerdictJSONFrom("itd-104", "source:never-filed"))) + if err == nil { + t.Fatalf("a key the default corpus does not carry was accepted\n%s", out) + } + if code := exitCodeOf(err); code != 2 || !strings.Contains(err.Error(), "source:never-filed") { + t.Errorf("exit %d, diagnostic %q; want 2 naming the key", code, err.Error()) + } +}