From 46ad4e8c2cc6e52713e5dd03348a620dd461c6bc Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 19:58:47 +0000 Subject: [PATCH 1/3] feat(intake): reconcile can see a prompt that shipped silently, and one filed twice MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two blind spots, both found by drift that reached the dashboard as pickable backlog, both invisible to every existing guard. **Leg 4 — the absence signal (`--repo`).** Leg 3 tests PRESENCE: "the prompt names things that exist upstream". smoke_install_stale_jax_pin.md shipped 2026-08-23 (autolens_workspace_test#266, PR #268) with no Mind-side trace at all, and both guards passed it — `lifecycle.py check` has no invariant for a prompt that was never `active/`, and `reconcile --repo autolens_workspace_test` pointed straight at the right repo found 0 suspects of 132. It named `smoke_install.sh`, which still exists. What it QUOTED — `pip install "jax<0.7" "jaxlib<0.7"` — did not. Verified against the real clone: the gone line flags, the still-present install line in the same fence does not. Three filters keep it off every prompt that quotes its own evidence, all three required: an explicit source-language fence (a traceback goes in a bare fence, and those lines are absent from every repo by construction); a named file that exists upstream, so "absent from what?" has an answer; and absence from the whole checkout, so a moved line reads as the refactor it is. Feeds `upstream_score` only — a quote can vanish for reasons other than shipping. **Leg 5 — duplicate candidates (offline).** Every other signal scores a prompt against the completion ARCHIVE; nothing scored live prompts against each other. That let two filings of the same finding — bug/workspaces/jax_likelihood_pins_ stale_by_1e4.md (08-14) and bug/autolens/jax_likelihood_smoke_pins_stale.md (08-19), same three scripts, same failing gate — both live until a drift audit found the survivor nine days later. Reconstructed from the two records' `## Original prompt` sections, the new leg pairs them at 12.0 on the shared script paths. Measured on the live backlog: 36 pairs -> 2. The filter that did almost all of that is dropping BARE BASENAMES — `start_here.py`, `modeling.py`, `no_run.yaml` are workspace-wide conventions, and sharing one is a convention, not a task. Mutual reference and a folder index naming both disqualify a pair: a phased parent and child, and the four health_fixes/ prompts split by cause whose README names all four, are declared series rather than accidents. Also fixes a real parsing bug the tests caught: `_PATH_RE` opened with `\b`, which cannot match before a leading dot, so `.github/workflows/release.yml` silently truncated to a path resolving against no file. Dot-directories are where CI recipes live, which is half of what these prompts are about. The `--repo` seam becomes `_UpstreamReader` — callable for leg 3, `.quotes` for leg 4, probed with getattr so every pre-existing lambda fake still exercises leg 3 alone. 13 new tests, mostly refusals; 582 pass. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vd6ptkoJsjAJMosPWsCDZz --- agents/conductors/intake/AGENTS.md | 51 +++- agents/conductors/intake/_intake.py | 389 ++++++++++++++++++++++++- tests/test_intake_reconcile_ranking.py | 233 +++++++++++++++ 3 files changed, 668 insertions(+), 5 deletions(-) diff --git a/agents/conductors/intake/AGENTS.md b/agents/conductors/intake/AGENTS.md index 2401a8e2..8b9dedc3 100644 --- a/agents/conductors/intake/AGENTS.md +++ b/agents/conductors/intake/AGENTS.md @@ -93,8 +93,8 @@ schema — light structure over free-form prose. | **census** | `intake census` | inventory every filed prompt (work-type/target/difficulty/status + hygiene flags); always read-only | | **dashboard** | `intake dashboard` | render the census as the Mind **task** page — picks, in flight, parked, planned, backlog, recent, epics; `--apply` writes `PyAutoMind/dashboard.md`, `--check` exits 1 on drift | | **formalise** | `intake formalise [prefix]` | retroactively header the prompts census flags — derive the missing fields, insert in place, prose untouched; `--apply` writes | -| **reconcile** | `intake reconcile [prefix]` | rank backlog prompts that look already-shipped (vs the `complete/` records / `active/`); always read-only — retiring stays human | -| **reconcile --repo** | `intake reconcile --repo [prefix]` | **also** read the target repo's source for identifiers the prompts name — the one signal that sees a prompt with no Mind-side trace. Opt-in; the default path is offline | +| **reconcile** | `intake reconcile [prefix]` | rank backlog prompts that look already-shipped (vs the `complete/` records / `active/`), and pair live prompts that look like the same work filed twice; always read-only — retiring stays human | +| **reconcile --repo** | `intake reconcile --repo [prefix]` | **also** read the target repo's source: identifiers the prompts name that exist upstream, and lines they quote that are **gone** — the two signals that see a prompt with no Mind-side trace. Opt-in; the default path is offline | **Recent** is the one section laid out by *date* rather than by state: the 50 newest events on the **work in hand** — issued, parked, filed — merged across @@ -151,6 +151,24 @@ are on PyAutoFit `main`, and the prompt is **not** shipped — the upstream catc wraps only the likelihood call while the raising line sits before the `try`. Presence of a name is not presence of the fix. +`--repo` also carries the **absence signal**, the inversion of the above: a +literal line a prompt *quotes* that is **gone** from a file it names. Presence +tests "the prompt names things that exist upstream" and is blind to a task that +shipped leaving no Mind-side trace while its files still exist — +`smoke_install_stale_jax_pin.md` shipped 2026-08-23 and both guards passed it +(`lifecycle.py check` has no invariant for a prompt that was never `active/`; +`reconcile --repo autolens_workspace_test` found 0 suspects of 132). It named +`smoke_install.sh`, which still exists. What it quoted — +`pip install "jax<0.7" "jaxlib<0.7"` — did not. + +Three filters keep it off every prompt that quotes its own evidence, and all +three must hold: the fence carries an explicit **source language** (a traceback +or pytest summary goes in a bare fence, and those lines are absent from every +repo by construction); the prompt **names a file that exists upstream** with +that extension, so "absent from what?" has an answer; and the line is absent +from the **whole checkout**, so a moved line reads as the refactor it is. Like +presence, it feeds only the upstream key and never retires anything. + **This is the only network access in PyAutoBrain.** Every other conductor and faculty is stdlib-only and offline, and the default `reconcile` path stays that way — a test detonates on any socket or subprocess use when `--repo` is absent. @@ -161,6 +179,32 @@ Clones are cached shallow (`--depth 1`) under `$PYAUTO_BRAIN_CACHE` (default `~/.pyauto-brain/upstream`), and the resolved sha is printed so a verdict is re-checkable. +### Duplicate candidates (offline) + +Every signal above scores a prompt against the completion **archive**. Nothing +scored the live prompts against **each other**, and near-duplicate filings are a +standing hazard of a backlog several independent sessions file into. +`bug/workspaces/jax_likelihood_pins_stale_by_1e4.md` (filed 08-14) and +`bug/autolens/jax_likelihood_smoke_pins_stale.md` (filed 08-19) named the same +three scripts from the same failing smoke gate; the 08-19 copy was verified and +retired on 08-26, and the 08-14 copy kept rendering as pickable backlog. + +`reconcile` therefore also emits a `duplicate-candidate` bucket, pair-wise, over +shared upstream **source paths**, rare **identifiers** and **tracking +references**. It runs on the default (offline) path — it reads only `draft/`. + +Three filters carry the precision, measured on the 2026-08-27 backlog (134 +prompts): **36 pairs → 2**. + +| Filter | Why | +|---|---| +| A path named by more than `_DUP_PATH_COMMON` prompts is dropped, and a **bare basename never counts** | `start_here.py`, `modeling.py`, `no_run.yaml` are workspace-wide conventions; sharing one is a convention, not a task. This filter alone removed 31 of the 36 | +| **Mutual reference disqualifies** | a phased parent and its child name each other — a series, not a duplicate | +| **A folder index naming both disqualifies** | the four `draft/bug/health_fixes/` prompts were split by *cause*, so they share their failing scripts and none names another; the folder's `README.md` names all four, and that is the declaration | + +A pair is something to **read together**, never a verdict: two prompts can share +files and still be different work. + ## Machine sources (one staging surface) Conception input increasingly arrives from the organism itself — research @@ -195,7 +239,8 @@ bin/pyauto-brain intake --apply dashboard # write PyAut bin/pyauto-brain intake dashboard --check # exit 1 if the committed page has drifted bin/pyauto-brain intake formalise # propose retroactive headers (dry-run) bin/pyauto-brain intake --apply formalise bug/ # write them, only under bug/ -bin/pyauto-brain intake reconcile # rank shipped-but-stale suspects (read-only) +bin/pyauto-brain intake reconcile # rank shipped-but-stale suspects + duplicate pairs (read-only) +bin/pyauto-brain intake reconcile --repo autolens_workspace_test # + upstream presence and quote-absence legs ``` **Writes only under `--apply`; dry-run is the default.** Exit codes: `0` produced diff --git a/agents/conductors/intake/_intake.py b/agents/conductors/intake/_intake.py index 5f67710a..e8ffee6c 100755 --- a/agents/conductors/intake/_intake.py +++ b/agents/conductors/intake/_intake.py @@ -1780,6 +1780,105 @@ def _idents(text: str) -> set: return set(_IDENT_RE.findall(text)) +# --- literal quotes and named paths (legs 4 and 5) ------------------------------ +#: Fence languages whose content is SOURCE, mapped to the file extension it +#: would live in. Deliberately narrow. A prompt quoting a traceback, a pytest +#: summary or a log excerpt writes it in a BARE fence or one tagged `text`, and +#: those lines are absent from every repo by construction — scoring them would +#: make leg 4 fire on every prompt that quotes its own evidence. Requiring an +#: explicit source language is what keeps the absence signal honest. +_FENCE_LANGS = { + "python": ".py", "py": ".py", + "bash": ".sh", "sh": ".sh", "shell": ".sh", "zsh": ".sh", + "yaml": ".yaml", "yml": ".yaml", + "toml": ".toml", "cfg": ".cfg", "ini": ".cfg", +} +#: A path a prompt names, e.g. `scripts/interferometer/jax_likelihood/mge.py`. +#: The left edge is a lookbehind, NOT `\b`: a word boundary cannot match before +#: a leading dot, so `\b` silently truncated `.github/workflows/release.yml` to +#: `github/workflows/release.yml` — a path that then resolves against no file in +#: the checkout. Dot-directories are exactly where CI recipes live, which is +#: half of what these prompts are about. +_PATH_RE = re.compile( + r"(? bool: + """Is this fenced line plausibly a line OF a source file? + + Length and punctuation only — the discrimination that matters is the fence + language above, and (for leg 4) that the prompt also names a file which + actually exists upstream. This just drops fence padding and one-word lines. + """ + s = line.strip() + return 8 <= len(s) <= 200 and any(ch in s for ch in "=(){}[]\"':/,") + + +def _quoted_source_lines(text: str) -> dict: + """`.ext` -> {literal line, …} for every source-tagged fence in a prompt.""" + out: dict = {} + inside, ext = False, None + for line in text.splitlines(): + m = re.match(r"^\s*```+\s*([A-Za-z0-9_+.-]*)\s*$", line) + if m: + inside, ext = (False, None) if inside else ( + True, _FENCE_LANGS.get(m.group(1).lower())) + continue + if inside and ext and _looks_like_source(line): + out.setdefault(ext, set()).add(line.strip()) + return out + + +def _named_paths(text: str, mind_internal: bool = False) -> set: + """Source paths a prompt names. Mind-internal paths dropped by default.""" + found = set(_PATH_RE.findall(text)) + if mind_internal: + return found + return {f for f in found if not any(f.startswith(d) for d in _MIND_DIRS)} + + +#: Files under `draft/` that describe a folder's prompts rather than being one. +_INDEX_NAMES = ("README.md", "index.md", "AGENTS.md") + + +def _indexed_together(mind: Path, a: str, b: str, base_a: str, base_b: str) -> bool: + """Does an index in these prompts' common folder name them both?""" + folder = a.rsplit("/", 1)[0] + if folder != b.rsplit("/", 1)[0]: + return False + for name in _INDEX_NAMES: + f = mind / folder / name + try: + text = f.read_text(encoding="utf-8", errors="replace") + except OSError: + continue + if base_a in text and base_b in text: + return True + return False + + +def _qualified_path(path: str) -> bool: + """Is this path specific enough to be evidence that two prompts share work? + + A bare basename is not. `start_here.py` and `modeling.py` are workspace-wide + conventions repeated across hundreds of example folders — a prompt naming + one has said nothing about which work it is. Requiring a directory component + is what separates `interferometer/jax_likelihood/mge.py` (one file in one + repo) from `mge.py` (dozens). `X.py` and friends are prose placeholders. + """ + return "/" in path and len(Path(path).stem) > 2 + + +def _issue_refs(text: str) -> set: + """`{'PyAutoFit#1473', …}`, repo name normalised so `#1473` alone is ignored.""" + return {f"{normalise_repo(repo)}#{num}" for repo, num in _ISSUE_RE.findall(text)} + + # --- the upstream read (--repo) ------------------------------------------------- # Leg 3 of the staleness work (PyAutoBrain#223). Everything above this line is # Mind-local: it cross-references `draft/` against `complete/` and `active/`. @@ -1808,6 +1907,78 @@ def _idents(text: str) -> set: # trap above would ride it there. _UPSTREAM_MIN_IDENTS = 2 # one shared name is a coincidence, not a signal +# --- leg 4: the absence signal -------------------------------------------------- +# Leg 3 above tests PRESENCE — "the prompt names things that exist upstream" — +# and that is structurally blind to the shape that motivated this leg. +# `smoke_install_stale_jax_pin.md` shipped on 2026-08-23 (autolens_workspace_test +# #266, PR #268) with NO Mind-side trace at all: no active.md entry, no active/ +# move, no complete/ record. It kept rendering as pickable backlog and was +# re-picked four days later. Both guards were run against it directly and both +# passed it: `lifecycle.py check` (its invariant is about active.md slugs, and a +# task that skips Mind state is in neither place) and `intake reconcile +# maintenance/ci --repo autolens_workspace_test`, pointed straight at the right +# repo — 0 suspects of 132 scanned. It named `smoke_install.sh`, which still +# exists, so presence said nothing. +# +# The defect it described was an ABSENCE. It quoted a literal line — +# `pip install "jax<0.7" "jaxlib<0.7"` — and that line is GONE upstream. A +# backlog prompt quoting a source line that no longer exists in a file it names +# is close to a proof the work shipped. +# +# Three filters keep it from firing on every prompt that quotes its own +# evidence, and all three must hold: +# +# 1. the fence carries an explicit SOURCE language (_FENCE_LANGS). A quoted +# traceback, pytest summary or log excerpt goes in a bare or `text` fence, +# and those are absent from every repo by construction. +# 2. the prompt names a file that EXISTS upstream with that extension. Without +# an anchor the question "absent from what?" has no answer. +# 3. the line is absent from the WHOLE tree, not just that file — otherwise a +# refactor that moved a line reads as a shipped fix. +# +# Like leg 3 it feeds `upstream_score`, never `score`: a quoted line can vanish +# for reasons other than the prompt shipping (an unrelated refactor, a reworded +# quote), so it ranks for review and never retires. +_W_QUOTE_ABSENT = 4.0 # per absent quoted line. Above _W_UPSTREAM (1.5): the + # prompt's own argument is that absence is the stronger + # evidence, and the ordering inside `needs-review` + # should say so. Still below _W_SHIPPED (7.0), which is + # a Mind-local band this leg must never reach. + +# --- leg 5: duplicate candidates ------------------------------------------------ +# Every signal above scores a prompt against the completion ARCHIVE. Nothing +# scored the live prompts against EACH OTHER, and near-duplicate filings are a +# standing hazard of a backlog this size that several independent sessions file +# into. +# +# Measured case: `bug/workspaces/jax_likelihood_pins_stale_by_1e4.md` (filed +# 08-14) and `bug/autolens/jax_likelihood_smoke_pins_stale.md` (filed 08-19) — +# same three scripts, same failing smoke gate, written five days apart by +# sessions that did not know about each other. The 08-19 copy was verified and +# retired on 08-26; the 08-14 copy kept rendering as pickable backlog until a +# drift audit found it on 08-27. No guard saw it: neither was ever `active/`, so +# no lifecycle invariant applied, and reconcile compares prompt text to RECORD +# text — the record covering it carried the twin's words, not its own. +# +# Shared upstream file paths are the load-bearing signal: two prompts naming the +# same source files are almost never independent work. Identifiers and issue +# references corroborate. Mutual reference is the precision filter — a phased +# parent and its child name each other, and that is a series, not a duplicate. +_W_DUP_PATH = 4.0 # per shared upstream source path +_W_DUP_IDENT = 1.0 # per shared rare identifier +_W_DUP_ISSUE = 2.0 # per shared tracking reference +_DUP_THRESHOLD = 8.0 +#: An identifier this many live prompts or more name is backlog vocabulary. +_DUP_IDENT_COMMON = 4 +#: Ditto for paths — and this one is load-bearing. Measured on the 2026-08-27 +#: backlog (134 prompts): scoring every shared path flagged 36 pairs, and the +#: noise was almost entirely BARE BASENAMES. `start_here.py`, `modeling.py`, +#: `simulator.py`, `no_run.yaml`, `smoke_tests.txt` are workspace-wide filenames +#: that dozens of unrelated prompts name; two prompts sharing `start_here.py` +#: share a convention, not a task. Same discrimination `_TOKEN_COMMON_DF` and +#: `_IDENT_COMMON_DF` already make against the records. +_DUP_PATH_COMMON = 4 + def _upstream_noise() -> set: """Identifiers whose presence upstream says nothing about a prompt. @@ -1955,6 +2126,93 @@ def _grep_source(root: Path, idents: set) -> dict: return hits +_QUOTE_SUFFIXES = (".py", ".pyi", ".sh", ".yaml", ".yml", ".toml", ".cfg") + + +def _check_quotes(root: Path, named: set, quoted: dict) -> dict: + """Leg 4: which lines a prompt quotes are GONE from the files it names. + + `named` are path fragments the prompt mentions; `quoted` is + `{'.ext': {line, …}}` from its source-tagged fences. Returns + `{"files": [rel, …], "absent": [(line, rel), …]}` — `files` is the anchor + (the named paths that actually exist upstream), `absent` the lines that + occur in NONE of them and nowhere else in the tree either. + + A line found elsewhere in the tree is reported as neither: it moved, and a + moved line is a refactor, not a shipped fix. + """ + resolved: dict = {} + for cand in sorted(named): + if len(resolved) >= 12: # a prompt naming more is a survey + break + for f in root.rglob("*"): + if not f.is_file() or ".git" in f.parts: + continue + if f.as_posix().endswith("/" + cand) or f.name == cand: + resolved[cand] = f + break + if not resolved: + return {"files": [], "absent": []} + + # One read of each resolved file, plus one lazy pass over the tree only for + # the lines that were not found in them. + bodies = {} + for cand, f in resolved.items(): + try: + bodies[cand] = f.read_text(encoding="utf-8", errors="replace") + except OSError: + bodies[cand] = "" + + missing: dict = {} + for ext, lines in quoted.items(): + anchors = {c: b for c, b in bodies.items() + if Path(c).suffix == ext} + if not anchors: + continue # nothing of that kind was named + for line in lines: + if not any(line in b for b in anchors.values()): + missing[line] = sorted(anchors)[0] + if missing: + # Absent from the named file is not enough — the line may have moved. + for f in root.rglob("*"): + if not missing: + break + if (not f.is_file() or ".git" in f.parts + or f.suffix not in _QUOTE_SUFFIXES): + continue + try: + text = f.read_text(encoding="utf-8", errors="replace") + except OSError: + continue + for line in [ln for ln in missing if ln in text]: + del missing[line] + # Two named candidates can resolve to one file (`smoke_install.sh` and + # `autolens_workspace_test/.github/scripts/smoke_install.sh` are the same + # anchor) — report the anchor once. + return {"files": sorted({f.relative_to(root).as_posix() + for f in resolved.values()}), + "absent": sorted(missing.items())} + + +class _UpstreamReader: + """The `--repo` seam: identifier presence (leg 3) and quote absence (leg 4). + + Callable for leg 3 so every existing caller and test fake keeps working; leg + 4 hangs off `.quotes`, and `reconcile` probes for it with `getattr`. A fake + that is a plain lambda therefore exercises leg 3 alone, which is what the + hermetic tests written before this leg existed expect. + """ + + def __init__(self, root: Path): + self._root = root + + def __call__(self, idents: set) -> dict: + return _grep_source(self._root, idents) + + def quotes(self, named: set, quoted: dict) -> dict: + return _check_quotes(self._root, named, quoted) + + def upstream_reader(mind: Path, target: str, cache: Path = None): """Build the `source_reader` seam for `reconcile(repo=…)`. @@ -1971,7 +2229,97 @@ def upstream_reader(mind: Path, target: str, cache: Path = None): root, sha = _clone_upstream(slug, cache) except Exception as exc: # network/git failure return None, "", slug, f"could not read {slug}: {exc}" - return (lambda idents: _grep_source(root, idents)), sha, slug, "" + return _UpstreamReader(root), sha, slug, "" + + +def duplicate_candidates(mind: Path, records: list, prefix: str = "") -> list: + """Leg 5: pairs of LIVE prompts that look like the same work filed twice. + + Offline and Mind-local. Scores every unordered pair on the artefacts both + name — upstream source paths, rare identifiers, tracking references — and + returns the pairs over `_DUP_THRESHOLD`, strongest first. + + Two filters carry the precision: + + * **Mutual reference disqualifies.** A phased parent and its child name + each other, as do deliberate siblings; that is a series, and reconcile + must not offer to merge one. The measured duplicate pair named neither + the other, which is exactly why it survived. + * **Rare identifiers only.** An identifier `_DUP_IDENT_COMMON` or more + live prompts share is backlog vocabulary, the same reasoning `ident_df` + applies against the records. + + Advisory, like every other reconcile output: merging or retiring a prompt + stays a human act. + """ + import itertools + + texts: dict = {} + for r in records: + if prefix and not _prefix_match(r["path"], prefix): + continue + if r["path"].rsplit("/", 1)[-1] in _INDEX_NAMES: + continue # an index is not a filing; it describes them + try: + texts[r["path"]] = (mind / r["path"]).read_text( + encoding="utf-8", errors="replace") + except OSError: + continue + + ident_pdf: dict = {} + path_pdf: dict = {} + feats: dict = {} + for path, text in texts.items(): + idents = _idents(text) + named = {f for f in _named_paths(text) if _qualified_path(f)} + feats[path] = { + "paths": named, + "idents": idents, + "issues": _issue_refs(text), + "base": path.rsplit("/", 1)[-1], + "text": text, + } + for i in idents: + ident_pdf[i] = ident_pdf.get(i, 0) + 1 + for f in named: + path_pdf[f] = path_pdf.get(f, 0) + 1 + + out = [] + for a, b in itertools.combinations(sorted(feats), 2): + fa, fb = feats[a], feats[b] + # Mutual reference: these two prompts know about each other. + if fa["base"] in fb["text"] or fb["base"] in fa["text"]: + continue + # A DECLARED series knows about itself without any member naming + # another. `draft/bug/health_fixes/` holds four prompts split out of one + # health run — they share their failing scripts because they were split + # by CAUSE, not by script — and the folder's own README.md names all + # four. That index IS the declaration, and it is the same reasoning + # leg 1 applies to a record that resolves a folder of prompts as a + # group. Without it the trio yields three pairs of already-known work. + if _indexed_together(mind, a, b, fa["base"], fb["base"]): + continue + shared_paths = sorted(f for f in fa["paths"] & fb["paths"] + if path_pdf.get(f, 0) <= _DUP_PATH_COMMON) + shared_idents = sorted(i for i in fa["idents"] & fb["idents"] + if ident_pdf.get(i, 0) <= _DUP_IDENT_COMMON) + shared_issues = sorted(fa["issues"] & fb["issues"]) + score = (_W_DUP_PATH * len(shared_paths) + + _W_DUP_IDENT * len(shared_idents) + + _W_DUP_ISSUE * len(shared_issues)) + if score < _DUP_THRESHOLD: + continue + shared = [] + if shared_paths: + shared.append(("shared-paths", ", ".join(shared_paths[:5]))) + if shared_idents: + shared.append(("shared-identifiers", ", ".join(shared_idents[:5]))) + if shared_issues: + shared.append(("shared-issues", ", ".join(shared_issues[:5]))) + out.append({"paths": [a, b], "score": round(score, 2), + "shared": [{"kind": k, "evidence": e} for k, e in shared]}) + out.sort(key=lambda d: (-d["score"], d["paths"])) + return out def reconcile(mind: Path, prefix: str = "", source_reader=None, @@ -2158,6 +2506,25 @@ def reconcile(mind: Path, prefix: str = "", source_reader=None, f"upstream: " + "; ".join( f"{i} ({up[i][0]})" for i in shown))) + # 4b. The absence signal: lines this prompt QUOTES that are gone + # from a file it names. The only leg that fires on a prompt + # which shipped leaving no Mind-side trace AND whose named + # files still exist — the shape leg 3 is blind to. + quotes = getattr(source_reader, "quotes", None) + if quotes is not None: + named = _named_paths(prompt_text) + quoted = _quoted_source_lines(prompt_text) + q = quotes(named, quoted) if (named and quoted) else {} + absent = q.get("absent") or [] + if absent: + upstream_score += _W_QUOTE_ABSENT * len(absent) + findings.append(( + "upstream-quote-absent", + f"{len(absent)} line(s) this prompt quotes are gone " + f"from the file(s) it names (" + + ", ".join(q["files"][:3]) + "): " + + "; ".join(repr(ln) for ln, _ in absent[:3]))) + # `Status:` alone is not evidence — it fired on every hand-set draft. Kept # as context on prompts something else already flagged, never as a reason. if (score > 0 or upstream_score > 0) and r["status"] not in ("-", "formalised"): @@ -2188,7 +2555,8 @@ def reconcile(mind: Path, prefix: str = "", source_reader=None, suspects.sort(key=lambda s: (order[s["confidence"]], -s["overlap_score"], -s.get("upstream_score", 0.0), s["path"])) return {"generated": _dt.date.today().isoformat(), "scanned": c["total"], - "upstream": upstream_meta or {}, "suspects": suspects} + "upstream": upstream_meta or {}, "suspects": suspects, + "duplicates": duplicate_candidates(mind, c["records"], prefix)} def emit_reconcile(res: dict): @@ -2215,8 +2583,25 @@ def emit_reconcile(res: dict): if len(ev) > 160: ev = ev[:157] + "…" print(f"{' ' * (width + 3)}{f['kind']}: {ev}") + dups = res.get("duplicates") or [] + if dups: + print(f"\n== Duplicate candidates: {len(dups)} pair(s) ==") + print(" Live prompts naming the same artefacts, neither referencing " + "the other.") + for d in dups: + print(f"[{d['score']:>6}] {d['paths'][0]}") + print(f"{' ' * 9}{d['paths'][1]}") + for s in d["shared"]: + ev = s["evidence"] + if len(ev) > 160: + ev = ev[:157] + "\u2026" + print(f"{' ' * 9}{s['kind']}: {ev}") print("\nRetiring a prompt stays human: verify against the target repo's " "git log / merged\nPRs, then retire it to the complete/ archive by hand.") + if dups: + print("A duplicate candidate is a PAIR to read together, not a verdict: " + "two prompts can\nshare files and still be different work. Merge " + "or retire by hand.") if up: print("`needs-review` means the prompt NAMES things that exist upstream " "— NOT that it\nshipped. A fix can land next to the name without " diff --git a/tests/test_intake_reconcile_ranking.py b/tests/test_intake_reconcile_ranking.py index 88e0adab..beb5bd41 100644 --- a/tests/test_intake_reconcile_ranking.py +++ b/tests/test_intake_reconcile_ranking.py @@ -333,3 +333,236 @@ def test_a_real_repo_target_resolves_to_its_slug(tmp_path): slug, err = _intake.resolve_repo(tmp_path, target) assert err == "" assert slug == "PyAutoLabs/PyAutoFit" + + +# --------------------------------------------------------------------------- # +# leg 4 — the absence signal (--repo), PyAutoMind#353 +# +# Leg 3 tests PRESENCE and is structurally blind to a prompt that shipped with +# NO Mind-side trace while the files it names still exist. +# `smoke_install_stale_jax_pin.md` shipped 2026-08-23 (autolens_workspace_test +# #266, PR #268) and both guards passed it: `lifecycle.py check` (its invariant +# is about active.md slugs, and a task that skips Mind state is in neither +# place) and `intake reconcile maintenance/ci --repo autolens_workspace_test`, +# pointed straight at the right repo — 0 suspects of 132. It named +# `smoke_install.sh`, which still exists. +# +# What it quoted was gone. These tests pin that, and the three filters that keep +# it from firing on every prompt that quotes its own evidence. +# --------------------------------------------------------------------------- # +def _tree(root: Path, files: dict) -> Path: + """A fake upstream checkout for the `.quotes` seam.""" + for rel, body in files.items(): + p = root / rel + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(body, encoding="utf-8") + return root + + +def _up(root: Path): + """The real reader over a fake tree — exercises the code `--repo` runs.""" + return _intake._UpstreamReader(root) + + +_GONE_PROMPT = """# The smoke install pins jax below 0.7 + +The install script still holds the pin: + +```bash +pip install "jax<0.7" "jaxlib<0.7" +``` + +It lives in `.github/scripts/smoke_install.sh`. +""" + + +def test_a_quoted_line_gone_from_a_named_file_is_surfaced(tmp_path): + """The motivating case: quoted line absent, named file present.""" + up = _tree(tmp_path / "up", { + ".github/scripts/smoke_install.sh": 'pip install "./PyAutoArray"\n'}) + root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) + res = _intake.reconcile(root, source_reader=_up(up)) + + s = [x for x in res["suspects"] if x["path"].endswith("smoke_pin.md")] + assert s, "a prompt quoting a line that no longer exists must be surfaced" + assert any(f["kind"] == "upstream-quote-absent" for f in s[0]["findings"]) + + +def test_a_quoted_line_still_present_is_not_evidence(tmp_path): + """The control. Same prompt, same file — but the line is still there, so + the prompt describes live work and must stay off the list.""" + up = _tree(tmp_path / "up", { + ".github/scripts/smoke_install.sh": 'pip install "jax<0.7" "jaxlib<0.7"\n'}) + root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) + assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() + + +def test_a_moved_line_is_not_a_shipped_fix(tmp_path): + """Absent from the named file is not enough. A refactor that moved the line + elsewhere in the tree is not the prompt shipping, so absence must be checked + against the WHOLE checkout before it counts.""" + up = _tree(tmp_path / "up", { + ".github/scripts/smoke_install.sh": "# moved out of here\n", + "ci/pins.sh": 'pip install "jax<0.7" "jaxlib<0.7"\n'}) + root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) + assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() + + +def test_quoted_output_is_not_treated_as_source(tmp_path): + """The precision filter that makes the leg usable. A prompt quoting its own + evidence — a traceback, a pytest summary, a log excerpt — writes it in a + BARE fence, and those lines are absent from every repo by construction. + Scoring them would fire leg 4 on most of the backlog.""" + body = ("# Three pins fail the smoke gate\n\n" + "It lives in `scripts/interferometer/rectangular.py`.\n\n" + "```\n" + "AssertionError: Not equal to tolerance rtol=0.0001, atol=0\n" + " [0]: -3163.8939270532364 (ACTUAL), -3164.286252 (DESIRED)\n" + "```\n") + up = _tree(tmp_path / "up", {"scripts/interferometer/rectangular.py": "x = 1\n"}) + root = _mind(tmp_path / "mind", {"bug/workspaces/pins.md": body}, {}) + assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() + + +def test_a_quote_with_no_named_file_has_nothing_to_be_absent_from(tmp_path): + """The anchor requirement. Without a file the prompt names that actually + exists upstream, 'absent' has no referent — the quote could belong to any + repo, or to none.""" + body = ("# Some pin somewhere\n\n```bash\n" + 'pip install "jax<0.7" "jaxlib<0.7"\n```\n') + up = _tree(tmp_path / "up", {"unrelated.py": "x = 1\n"}) + root = _mind(tmp_path / "mind", {"maintenance/ci/nameless.md": body}, {}) + assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() + + +def test_quote_absence_cannot_reach_a_mind_local_band(tmp_path): + """Same structural defence as leg 3: a quoted line can vanish for reasons + other than the prompt shipping (an unrelated refactor, a reworded quote), so + it ranks for review and can never produce a shipped verdict.""" + lines = "\n".join(f'pip install "pkg-{i}==1.0"' for i in range(9)) + body = ("# Many gone pins\n\nIn `ci/install.sh`.\n\n```bash\n" + + lines + "\n```\n") + up = _tree(tmp_path / "up", {"ci/install.sh": "# emptied\n"}) + root = _mind(tmp_path / "mind", {"maintenance/ci/many.md": body}, {}) + s = _intake.reconcile(root, source_reader=_up(up))["suspects"][0] + assert s["confidence"] == "needs-review" + assert s["overlap_score"] == 0.0 + assert s["upstream_score"] > 0.0 + + +def test_a_plain_lambda_reader_still_works(tmp_path): + """Backward compatibility of the seam. Leg 4 hangs off `.quotes`, probed + with getattr — a reader without it exercises leg 3 alone, which is what + every fake written before this leg existed is.""" + root = _mind(tmp_path, {"bug/gearbox/ties.md": "# Ties\n\n`_a_helper`\n`_b_helper`\n"}, {}) + res = _intake.reconcile(root, source_reader=_reader( + {"_a_helper": ["src/a.py:1"], "_b_helper": ["src/b.py:2"]})) + assert res["suspects"][0]["confidence"] == "needs-review" + + +# --------------------------------------------------------------------------- # +# leg 5 — duplicate candidates (offline), PyAutoMind#360 +# +# Every leg above scores a prompt against the completion ARCHIVE. Nothing scored +# the live prompts against EACH OTHER, and that blind spot let +# `bug/workspaces/jax_likelihood_pins_stale_by_1e4.md` (filed 08-14) and +# `bug/autolens/jax_likelihood_smoke_pins_stale.md` (filed 08-19) — the same +# three scripts, the same failing smoke gate, five days apart — both live. The +# 08-19 copy was verified and retired on 08-26; the 08-14 copy kept rendering as +# pickable backlog. +# --------------------------------------------------------------------------- # +def _dups(res) -> set: + return {tuple(d["paths"]) for d in res["duplicates"]} + + +def test_two_prompts_naming_the_same_scripts_are_paired(tmp_path): + """The measured case, in miniature: three shared script paths, in different + work-type/target folders, neither naming the other.""" + scripts = ("`interferometer/jax_likelihood/rectangular.py`\n" + "`interferometer/jax_likelihood/mge.py`\n" + "`multi_dataset/jax_likelihood/mge.py`\n") + root = _mind(tmp_path, { + "bug/workspaces/pins_stale_by_1e4.md": "# Three pins are stale\n\n" + scripts, + "bug/autolens/smoke_pins_stale.md": "# Four scripts fail smoke\n\n" + scripts, + }, {}) + assert _dups(_intake.reconcile(root)) == { + ("draft/bug/autolens/smoke_pins_stale.md", + "draft/bug/workspaces/pins_stale_by_1e4.md")} + + +def test_a_bare_basename_is_not_evidence_of_shared_work(tmp_path): + """The filter that took the live backlog from 36 pairs to 2. `start_here.py` + and `modeling.py` are workspace-wide conventions repeated across hundreds of + example folders — two prompts naming one share a convention, not a task.""" + common = "`start_here.py`\n`modeling.py`\n`simulator.py`\n`no_run.yaml`\n" + root = _mind(tmp_path, { + "docs/autolens/multi_galaxy.md": "# Multi galaxy docs\n\n" + common, + "docs/workspaces/cluster_narrative.md": "# Cluster narrative\n\n" + common, + }, {}) + assert _intake.reconcile(root)["duplicates"] == [] + + +def test_prompts_that_name_each_other_are_a_series_not_a_duplicate(tmp_path): + """A phased parent and its child share everything by design. The measured + duplicate pair named neither the other, which is exactly why it survived.""" + scripts = ("`interferometer/jax_likelihood/rectangular.py`\n" + "`interferometer/jax_likelihood/mge.py`\n" + "`multi_dataset/jax_likelihood/mge.py`\n") + root = _mind(tmp_path, { + "bug/workspaces/phase_one.md": "# Phase 1\n\n" + scripts, + "bug/autolens/phase_two.md": ("# Phase 2\n\nParent: `phase_one.md`\n\n" + + scripts), + }, {}) + assert _intake.reconcile(root)["duplicates"] == [] + + +def test_a_folder_index_naming_both_is_a_declared_series(tmp_path): + """`draft/bug/health_fixes/` holds four prompts split out of one health run + — split by CAUSE, not by script, so they share their failing scripts and no + two name each other. The folder's own README.md names all four, and that + index IS the declaration. Without this the trio yields three pairs of + already-known work.""" + scripts = ("`imaging/features/advanced/double_einstein_ring/chaining.py`\n" + "`config/build/profile_release.yaml`\n" + "`imaging/features/advanced/double_source_plane_lens/chaining.py`\n") + root = _mind(tmp_path, { + "bug/health_fixes/jax_runtime_and_parity.md": "# JAX runtime\n\n" + scripts, + "bug/health_fixes/jit_visualization_outputs.md": "# JIT viz\n\n" + scripts, + }, {}) + (root / "draft" / "bug" / "health_fixes" / "README.md").write_text( + "The split:\n- jax_runtime_and_parity.md\n- jit_visualization_outputs.md\n", + encoding="utf-8") + assert _intake.reconcile(root)["duplicates"] == [] + + +def test_one_shared_path_is_not_a_duplicate(tmp_path): + """The same bar every other leg sets: one shared artefact is a coincidence. + Two prompts touching one file is ordinary; it is the OVERLAP that speaks.""" + root = _mind(tmp_path, { + "bug/workspaces/one.md": "# One\n\n`interferometer/jax_likelihood/mge.py`\n", + "bug/autolens/two.md": "# Two\n\n`interferometer/jax_likelihood/mge.py`\n", + }, {}) + assert _intake.reconcile(root)["duplicates"] == [] + + +def test_duplicate_scan_stays_offline_and_never_writes(tmp_path, monkeypatch): + """Leg 5 is Mind-local by construction — it reads only `draft/`. The + offline guarantee the rest of the default path carries must hold here too.""" + import socket + import subprocess + + def _boom(*a, **k): + raise AssertionError("duplicate scan attempted network access") + + monkeypatch.setattr(subprocess, "run", _boom) + monkeypatch.setattr(socket, "socket", _boom) + scripts = ("`interferometer/jax_likelihood/rectangular.py`\n" + "`interferometer/jax_likelihood/mge.py`\n" + "`multi_dataset/jax_likelihood/mge.py`\n") + root = _mind(tmp_path, { + "bug/workspaces/one.md": "# One\n\n" + scripts, + "bug/autolens/two.md": "# Two\n\n" + scripts, + }, {}) + before = {p: p.read_bytes() for p in root.rglob("*.md")} + assert _intake.reconcile(root)["duplicates"] + assert {p: p.read_bytes() for p in root.rglob("*.md")} == before From 38c7ec951556298fefba29b9c0356dcb7bb6a950 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 20:03:15 +0000 Subject: [PATCH 2/3] fix(intake): the absence signal must corroborate its anchor, and know its repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Running leg 4 across the live backlog (134 prompts, read at autolens_workspace_test) surfaced six hits and every one was false. Two causes, two filters, 6 -> 0 — while the retired smoke_install_stale_jax_pin.md, restored into draft/ to reproduce, still fires. **Proposed code.** einstein_radius_jit_native_seed_finder.md quotes 23 lines of a JAX-native seed finder it wants WRITTEN. All 23 are absent because none has ever existed, and at 101.0 it outscored every true positive — the loudest thing in the output was the most wrong. delaunay_research.md is the same shape. The fix is to require the anchor to be CORROBORATED: at least one line the prompt quotes for that extension must still be PRESENT in the file it names. One present line proves the prompt is talking about this file, in this checkout, and only then does a sibling line's absence mean something happened. The motivating case passes it exactly — it quoted two lines from smoke_install.sh, the PyAuto install line still there and the jax pin gone. **The wrong repo.** A prompt read against a repo it is not about quotes lines absent from it by construction. mge_jit_regression_rebaseline.md is about autolens_workspace_developer and was being read at _test. The prompt that asked for this leg raised it as an open question ("consider whether the Repos: header should drive the upstream target"): the header had the answer all along while --repo had to be told by hand. Now a prompt that never mentions the repo under test is skipped, with the slug carried on the reader. The two positive fixtures were wrong in the same way the false positives were — they quoted only absent lines, which the real prompt did not. Made faithful, and both new refusals pinned. 584 pass. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vd6ptkoJsjAJMosPWsCDZz --- agents/conductors/intake/AGENTS.md | 25 ++++++++++---- agents/conductors/intake/_intake.py | 41 +++++++++++++++++++---- tests/test_intake_reconcile_ranking.py | 46 ++++++++++++++++++++++---- 3 files changed, 93 insertions(+), 19 deletions(-) diff --git a/agents/conductors/intake/AGENTS.md b/agents/conductors/intake/AGENTS.md index 8b9dedc3..12c40794 100644 --- a/agents/conductors/intake/AGENTS.md +++ b/agents/conductors/intake/AGENTS.md @@ -161,13 +161,24 @@ shipped leaving no Mind-side trace while its files still exist — `smoke_install.sh`, which still exists. What it quoted — `pip install "jax<0.7" "jaxlib<0.7"` — did not. -Three filters keep it off every prompt that quotes its own evidence, and all -three must hold: the fence carries an explicit **source language** (a traceback -or pytest summary goes in a bare fence, and those lines are absent from every -repo by construction); the prompt **names a file that exists upstream** with -that extension, so "absent from what?" has an answer; and the line is absent -from the **whole checkout**, so a moved line reads as the refactor it is. Like -presence, it feeds only the upstream key and never retires anything. +Five filters keep it off every prompt that quotes its own evidence, and all +five must hold: + +| Filter | The false positive it kills | +|---|---| +| the fence carries an explicit **source language** | a traceback or pytest summary goes in a bare fence, and those lines are absent from every repo by construction | +| the prompt **names a file that exists upstream** with that extension | without an anchor, *"absent from what?"* has no answer | +| the line is absent from the **whole checkout**, not just that file | a refactor that moved a line is not the prompt shipping | +| the prompt **mentions the repo being read** | a prompt read against a repo it is not about quotes lines absent by construction — its `Repos:` header had the answer all along, while `--repo` had to be told the target by hand | +| **the anchor is corroborated** — at least one quoted line of that kind is still *present* in the named file | **proposed code.** `einstein_radius_jit_native_seed_finder.md` quotes 23 lines of a seed finder it wants *written*; all 23 are absent because none ever existed, and it outscored every true positive. One line still present proves the prompt is talking about this file, in this checkout | + +Measured against the live backlog (134 prompts) read at `autolens_workspace_test`: +the last two filters took **6 hits → 0**, every one of the six a proposal or a +wrong-repo read, while the retired `smoke_install_stale_jax_pin.md` — restored +into `draft/` to reproduce — still fires. That prompt quoted *two* lines from +`smoke_install.sh`: the PyAuto install line, still there, and the jax pin, gone. + +Like presence, it feeds only the upstream key and never retires anything. **This is the only network access in PyAutoBrain.** Every other conductor and faculty is stdlib-only and offline, and the default `reconcile` path stays that diff --git a/agents/conductors/intake/_intake.py b/agents/conductors/intake/_intake.py index e8ffee6c..a8528c9d 100755 --- a/agents/conductors/intake/_intake.py +++ b/agents/conductors/intake/_intake.py @@ -2169,9 +2169,28 @@ def _check_quotes(root: Path, named: set, quoted: dict) -> dict: if Path(c).suffix == ext} if not anchors: continue # nothing of that kind was named - for line in lines: - if not any(line in b for b in anchors.values()): - missing[line] = sorted(anchors)[0] + gone = {ln for ln in lines + if not any(ln in b for b in anchors.values())} + # THE ANCHOR MUST BE CORROBORATED. If not one of the lines this prompt + # quotes for `ext` is present in the file(s) it names, the anchor is + # unproven and absence means nothing — measured against the live backlog, + # this is what separates the signal from its two loudest false positives: + # + # * PROPOSED code. `einstein_radius_jit_native_seed_finder.md` quotes + # 23 lines of a JAX-native seed finder it wants WRITTEN. All 23 are + # absent because none has ever existed, and it scored higher than + # every true positive. + # * The WRONG REPO. A prompt read against a repo it is not about + # quotes lines absent from it by construction. + # + # One quoted line still present proves the prompt is talking about this + # file, in this checkout — and then a sibling line's absence is a change + # that happened. The motivating case passes it exactly: the PyAuto + # install line is still in `smoke_install.sh`, and the jax pin is gone. + if len(gone) == len(lines): + continue + for line in gone: + missing[line] = sorted(anchors)[0] if missing: # Absent from the named file is not enough — the line may have moved. for f in root.rglob("*"): @@ -2203,8 +2222,9 @@ class _UpstreamReader: hermetic tests written before this leg existed expect. """ - def __init__(self, root: Path): + def __init__(self, root: Path, slug: str = ""): self._root = root + self.slug = slug def __call__(self, idents: set) -> dict: return _grep_source(self._root, idents) @@ -2229,7 +2249,7 @@ def upstream_reader(mind: Path, target: str, cache: Path = None): root, sha = _clone_upstream(slug, cache) except Exception as exc: # network/git failure return None, "", slug, f"could not read {slug}: {exc}" - return _UpstreamReader(root), sha, slug, "" + return _UpstreamReader(root, slug), sha, slug, "" def duplicate_candidates(mind: Path, records: list, prefix: str = "") -> list: @@ -2514,7 +2534,16 @@ def reconcile(mind: Path, prefix: str = "", source_reader=None, if quotes is not None: named = _named_paths(prompt_text) quoted = _quoted_source_lines(prompt_text) - q = quotes(named, quoted) if (named and quoted) else {} + # A prompt that never mentions the repo being read is not about + # it, and its quotes are absent from it by construction. The + # `Repos:` header had this answer all along, while `--repo` had + # to be told the target by hand. + slug = getattr(source_reader, "slug", "") + about = (not slug) or normalise_repo(slug.rsplit("/", 1)[-1]) in { + normalise_repo(x) for x in re.findall( + r"[@`/\s]([A-Za-z_][A-Za-z0-9_]*)", prompt_text)} + q = (quotes(named, quoted) + if (named and quoted and about) else {}) absent = q.get("absent") or [] if absent: upstream_score += _W_QUOTE_ABSENT * len(absent) diff --git a/tests/test_intake_reconcile_ranking.py b/tests/test_intake_reconcile_ranking.py index beb5bd41..05ac71ff 100644 --- a/tests/test_intake_reconcile_ranking.py +++ b/tests/test_intake_reconcile_ranking.py @@ -364,11 +364,15 @@ def _up(root: Path): return _intake._UpstreamReader(root) +# Faithful to the real prompt: it quoted TWO lines from the same file — the +# pin it was about, and the install line above it. That second line is what +# corroborates the anchor, and the leg requires it. _GONE_PROMPT = """# The smoke install pins jax below 0.7 The install script still holds the pin: ```bash +pip install "./PyAutoArray[optional]" pip install "jax<0.7" "jaxlib<0.7" ``` @@ -379,7 +383,7 @@ def _up(root: Path): def test_a_quoted_line_gone_from_a_named_file_is_surfaced(tmp_path): """The motivating case: quoted line absent, named file present.""" up = _tree(tmp_path / "up", { - ".github/scripts/smoke_install.sh": 'pip install "./PyAutoArray"\n'}) + ".github/scripts/smoke_install.sh": 'pip install "./PyAutoArray[optional]"\n'}) root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) res = _intake.reconcile(root, source_reader=_up(up)) @@ -391,8 +395,9 @@ def test_a_quoted_line_gone_from_a_named_file_is_surfaced(tmp_path): def test_a_quoted_line_still_present_is_not_evidence(tmp_path): """The control. Same prompt, same file — but the line is still there, so the prompt describes live work and must stay off the list.""" - up = _tree(tmp_path / "up", { - ".github/scripts/smoke_install.sh": 'pip install "jax<0.7" "jaxlib<0.7"\n'}) + up = _tree(tmp_path / "up", {".github/scripts/smoke_install.sh": + 'pip install "./PyAutoArray[optional]"\n' + 'pip install "jax<0.7" "jaxlib<0.7"\n'}) root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() @@ -402,7 +407,7 @@ def test_a_moved_line_is_not_a_shipped_fix(tmp_path): elsewhere in the tree is not the prompt shipping, so absence must be checked against the WHOLE checkout before it counts.""" up = _tree(tmp_path / "up", { - ".github/scripts/smoke_install.sh": "# moved out of here\n", + ".github/scripts/smoke_install.sh": 'pip install "./PyAutoArray[optional]"\n', "ci/pins.sh": 'pip install "jax<0.7" "jaxlib<0.7"\n'}) root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() @@ -441,8 +446,9 @@ def test_quote_absence_cannot_reach_a_mind_local_band(tmp_path): it ranks for review and can never produce a shipped verdict.""" lines = "\n".join(f'pip install "pkg-{i}==1.0"' for i in range(9)) body = ("# Many gone pins\n\nIn `ci/install.sh`.\n\n```bash\n" - + lines + "\n```\n") - up = _tree(tmp_path / "up", {"ci/install.sh": "# emptied\n"}) + 'pip install "./PyAutoArray[optional]"\n' + lines + "\n```\n") + up = _tree(tmp_path / "up", + {"ci/install.sh": 'pip install "./PyAutoArray[optional]"\n'}) root = _mind(tmp_path / "mind", {"maintenance/ci/many.md": body}, {}) s = _intake.reconcile(root, source_reader=_up(up))["suspects"][0] assert s["confidence"] == "needs-review" @@ -566,3 +572,31 @@ def _boom(*a, **k): before = {p: p.read_bytes() for p in root.rglob("*.md")} assert _intake.reconcile(root)["duplicates"] assert {p: p.read_bytes() for p in root.rglob("*.md")} == before + + +def test_proposed_code_is_not_an_absence(tmp_path): + """The loudest false positive on the live backlog, and the reason the anchor + must be corroborated. `einstein_radius_jit_native_seed_finder.md` quotes 23 + lines of a JAX-native seed finder it wants WRITTEN — all 23 absent because + none has ever existed — and it outscored every true positive. + + One quoted line still present proves the prompt is talking about this file + in this checkout. None present means the anchor is unproven: proposed code, + or the wrong repo. Absence then says nothing.""" + proposed = "\n".join(f" seed_{i} = grid_argmin(eigen_{i})" for i in range(6)) + body = ("# JAX-native seed finder\n\nAdd to `scripts/misc/util.py`:\n\n" + "```python\n" + proposed + "\n```\n") + up = _tree(tmp_path / "up", {"scripts/misc/util.py": "def existing(): ...\n"}) + root = _mind(tmp_path / "mind", {"refactor/autogalaxy/seed.md": body}, {}) + assert _paths(_intake.reconcile(root, source_reader=_up(up))) == set() + + +def test_a_prompt_not_about_the_repo_being_read_is_skipped(tmp_path): + """The `Repos:` header had the answer all along, while `--repo` had to be + told the target by hand. A prompt that never mentions the repo under test + is not about it, and its quotes are absent from it by construction.""" + up = _tree(tmp_path / "up", { + ".github/scripts/smoke_install.sh": 'pip install "./PyAutoArray[optional]"\n'}) + root = _mind(tmp_path / "mind", {"maintenance/ci/smoke_pin.md": _GONE_PROMPT}, {}) + reader = _intake._UpstreamReader(up, "PyAutoLabs/some_other_repo") + assert _paths(_intake.reconcile(root, source_reader=reader)) == set() From 5a189746cc38a0b9323098cb3952ead5ffd8ce26 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 20:06:03 +0000 Subject: [PATCH 3/3] fix(intake): keep the tenant's repo name out of organ code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI's tenant firewall caught what the tests could not: the new leg-4 comments and the test's section header named a concrete satellite repo, in .py files. The framework organs must stay adoptable as a config-diff fork, so an instance fact outside the declared config surfaces is drift — and growing FIREWALL_ALLOWLIST instead would mean one more file an adopting fork has to rewrite, for a name that carries none of the finding. The shape is the evidence, not the tenant: "a workspace-test repo, issue #266 / PR #268" says everything the comment needed. The dates, issue and PR numbers are not instance facts and stay. AGENTS.md keeps the concrete names deliberately — skills prose and AGENTS.md are out of the firewall's scope by design (production prompts, never genericised), and the runnable `--repo` example there is more useful naming a real repo. The test file was already violating its own header, which promises every fixture is fictional and that nothing in it names a real prompt or record. tenant firewall: OK. 584 pass. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Vd6ptkoJsjAJMosPWsCDZz --- agents/conductors/intake/_intake.py | 22 +++++++++++----------- tests/test_intake_reconcile_ranking.py | 17 +++++++++-------- 2 files changed, 20 insertions(+), 19 deletions(-) diff --git a/agents/conductors/intake/_intake.py b/agents/conductors/intake/_intake.py index a8528c9d..03167818 100755 --- a/agents/conductors/intake/_intake.py +++ b/agents/conductors/intake/_intake.py @@ -1801,7 +1801,7 @@ def _idents(text: str) -> set: #: half of what these prompts are about. _PATH_RE = re.compile( r"(? set: # --- leg 4: the absence signal -------------------------------------------------- # Leg 3 above tests PRESENCE — "the prompt names things that exist upstream" — # and that is structurally blind to the shape that motivated this leg. -# `smoke_install_stale_jax_pin.md` shipped on 2026-08-23 (autolens_workspace_test -# #266, PR #268) with NO Mind-side trace at all: no active.md entry, no active/ -# move, no complete/ record. It kept rendering as pickable backlog and was -# re-picked four days later. Both guards were run against it directly and both -# passed it: `lifecycle.py check` (its invariant is about active.md slugs, and a -# task that skips Mind state is in neither place) and `intake reconcile -# maintenance/ci --repo autolens_workspace_test`, pointed straight at the right -# repo — 0 suspects of 132 scanned. It named `smoke_install.sh`, which still -# exists, so presence said nothing. +# `smoke_install_stale_jax_pin.md` shipped on 2026-08-23 (a workspace-test +# repo, issue #266 / PR #268) with NO Mind-side trace at all: no active.md +# entry, no active/ move, no complete/ record. It kept rendering as pickable +# backlog and was re-picked four days later. Both guards were run against it +# directly and both passed it: `lifecycle.py check` (its invariant is about +# active.md slugs, and a task that skips Mind state is in neither place) and +# `intake reconcile maintenance/ci --repo `, pointed straight at it — +# 0 suspects of 132 scanned. It named `smoke_install.sh`, which still exists, so +# presence said nothing. # # The defect it described was an ABSENCE. It quoted a literal line — # `pip install "jax<0.7" "jaxlib<0.7"` — and that line is GONE upstream. A @@ -2206,7 +2206,7 @@ def _check_quotes(root: Path, named: set, quoted: dict) -> dict: for line in [ln for ln in missing if ln in text]: del missing[line] # Two named candidates can resolve to one file (`smoke_install.sh` and - # `autolens_workspace_test/.github/scripts/smoke_install.sh` are the same + # `/.github/scripts/smoke_install.sh` are the same # anchor) — report the anchor once. return {"files": sorted({f.relative_to(root).as_posix() for f in resolved.values()}), diff --git a/tests/test_intake_reconcile_ranking.py b/tests/test_intake_reconcile_ranking.py index 05ac71ff..91537c5f 100644 --- a/tests/test_intake_reconcile_ranking.py +++ b/tests/test_intake_reconcile_ranking.py @@ -340,15 +340,16 @@ def test_a_real_repo_target_resolves_to_its_slug(tmp_path): # # Leg 3 tests PRESENCE and is structurally blind to a prompt that shipped with # NO Mind-side trace while the files it names still exist. -# `smoke_install_stale_jax_pin.md` shipped 2026-08-23 (autolens_workspace_test -# #266, PR #268) and both guards passed it: `lifecycle.py check` (its invariant -# is about active.md slugs, and a task that skips Mind state is in neither -# place) and `intake reconcile maintenance/ci --repo autolens_workspace_test`, -# pointed straight at the right repo — 0 suspects of 132. It named -# `smoke_install.sh`, which still exists. +# `smoke_install_stale_jax_pin.md` shipped 2026-08-23 (a workspace-test repo, +# issue #266 / PR #268) and both guards passed it: `lifecycle.py check` (its +# invariant is about active.md slugs, and a task that skips Mind state is in +# neither place) and `intake reconcile maintenance/ci --repo `, +# pointed straight at it — 0 suspects of 132. It named `smoke_install.sh`, +# which still exists. # -# What it quoted was gone. These tests pin that, and the three filters that keep -# it from firing on every prompt that quotes its own evidence. +# What it quoted was gone. These tests pin that, and the five filters that keep +# it from firing on every prompt that quotes its own evidence — the last two +# added after a live run showed all six of its first hits were false. # --------------------------------------------------------------------------- # def _tree(root: Path, files: dict) -> Path: """A fake upstream checkout for the `.quotes` seam."""