Repository navigation
intake reconcile: see a prompt that shipped silently, and one filed twice - #307
Merged
Merged
Conversation
…ne filed twice Two blind spots, both found by drift that reached the dashboard as pickable backlog, both invisible to every existing guard. **Leg 4 — the absence signal (`--repo`).** Leg 3 tests PRESENCE: "the prompt names things that exist upstream". smoke_install_stale_jax_pin.md shipped 2026-08-23 (autolens_workspace_test#266, PR #268) with no Mind-side trace at all, and both guards passed it — `lifecycle.py check` has no invariant for a prompt that was never `active/`, and `reconcile --repo autolens_workspace_test` pointed straight at the right repo found 0 suspects of 132. It named `smoke_install.sh`, which still exists. What it QUOTED — `pip install "jax<0.7" "jaxlib<0.7"` — did not. Verified against the real clone: the gone line flags, the still-present install line in the same fence does not. Three filters keep it off every prompt that quotes its own evidence, all three required: an explicit source-language fence (a traceback goes in a bare fence, and those lines are absent from every repo by construction); a named file that exists upstream, so "absent from what?" has an answer; and absence from the whole checkout, so a moved line reads as the refactor it is. Feeds `upstream_score` only — a quote can vanish for reasons other than shipping. **Leg 5 — duplicate candidates (offline).** Every other signal scores a prompt against the completion ARCHIVE; nothing scored live prompts against each other. That let two filings of the same finding — bug/workspaces/jax_likelihood_pins_ stale_by_1e4.md (08-14) and bug/autolens/jax_likelihood_smoke_pins_stale.md (08-19), same three scripts, same failing gate — both live until a drift audit found the survivor nine days later. Reconstructed from the two records' `## Original prompt` sections, the new leg pairs them at 12.0 on the shared script paths. Measured on the live backlog: 36 pairs -> 2. The filter that did almost all of that is dropping BARE BASENAMES — `start_here.py`, `modeling.py`, `no_run.yaml` are workspace-wide conventions, and sharing one is a convention, not a task. Mutual reference and a folder index naming both disqualify a pair: a phased parent and child, and the four health_fixes/ prompts split by cause whose README names all four, are declared series rather than accidents. Also fixes a real parsing bug the tests caught: `_PATH_RE` opened with `\b`, which cannot match before a leading dot, so `.github/workflows/release.yml` silently truncated to a path resolving against no file. Dot-directories are where CI recipes live, which is half of what these prompts are about. The `--repo` seam becomes `_UpstreamReader` — callable for leg 3, `.quotes` for leg 4, probed with getattr so every pre-existing lambda fake still exercises leg 3 alone. 13 new tests, mostly refusals; 582 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vd6ptkoJsjAJMosPWsCDZz
… its repo
Running leg 4 across the live backlog (134 prompts, read at
autolens_workspace_test) surfaced six hits and every one was false. Two causes,
two filters, 6 -> 0 — while the retired smoke_install_stale_jax_pin.md, restored
into draft/ to reproduce, still fires.
**Proposed code.** einstein_radius_jit_native_seed_finder.md quotes 23 lines of
a JAX-native seed finder it wants WRITTEN. All 23 are absent because none has
ever existed, and at 101.0 it outscored every true positive — the loudest thing
in the output was the most wrong. delaunay_research.md is the same shape.
The fix is to require the anchor to be CORROBORATED: at least one line the
prompt quotes for that extension must still be PRESENT in the file it names. One
present line proves the prompt is talking about this file, in this checkout, and
only then does a sibling line's absence mean something happened. The motivating
case passes it exactly — it quoted two lines from smoke_install.sh, the PyAuto
install line still there and the jax pin gone.
**The wrong repo.** A prompt read against a repo it is not about quotes lines
absent from it by construction. mge_jit_regression_rebaseline.md is about
autolens_workspace_developer and was being read at _test. The prompt that asked
for this leg raised it as an open question ("consider whether the Repos: header
should drive the upstream target"): the header had the answer all along while
--repo had to be told by hand. Now a prompt that never mentions the repo under
test is skipped, with the slug carried on the reader.
The two positive fixtures were wrong in the same way the false positives were —
they quoted only absent lines, which the real prompt did not. Made faithful, and
both new refusals pinned. 584 pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vd6ptkoJsjAJMosPWsCDZz
CI's tenant firewall caught what the tests could not: the new leg-4 comments and the test's section header named a concrete satellite repo, in .py files. The framework organs must stay adoptable as a config-diff fork, so an instance fact outside the declared config surfaces is drift — and growing FIREWALL_ALLOWLIST instead would mean one more file an adopting fork has to rewrite, for a name that carries none of the finding. The shape is the evidence, not the tenant: "a workspace-test repo, issue #266 / PR #268" says everything the comment needed. The dates, issue and PR numbers are not instance facts and stay. AGENTS.md keeps the concrete names deliberately — skills prose and AGENTS.md are out of the firewall's scope by design (production prompts, never genericised), and the runnable `--repo` example there is more useful naming a real repo. The test file was already violating its own header, which promises every fixture is fictional and that nothing in it names a real prompt or record. tenant firewall: OK. 584 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vd6ptkoJsjAJMosPWsCDZz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two blind spots in
intake reconcile, both found by drift that reached the dashboard as pickable backlog, both invisible to every existing guard. Filed asdraft/feature/pyautobrain/intake_reconcile_absence_signal.md(PyAutoMind#353) anddraft/feature/pyautobrain/reconcile_duplicate_prompt_signal.md.Leg 4 — the absence signal (
--repo)Leg 3 tests PRESENCE: "the prompt names things that exist upstream". That is structurally blind to a task which shipped leaving no Mind-side trace while the files it names still exist.
smoke_install_stale_jax_pin.mdshipped 2026-08-23 (autolens_workspace_test#266, PR #268) with noactive.mdentry, noactive/move and nocomplete/record. Both guards passed it:lifecycle.py checkhas no invariant for a prompt that was neveractive/, andreconcile --repo autolens_workspace_test, pointed straight at the right repo, found 0 suspects of 132. It namedsmoke_install.sh, which still exists.What it quoted did not:
pip install "jax<0.7" "jaxlib<0.7".The filters, and how they were arrived at
The first cut had three filters and passed its tests. Then it was run against the live backlog (134 prompts, read at
autolens_workspace_test) and produced six hits, all six false — with the loudest the most wrong. Two more filters took that to 0, while the retired prompt, restored intodraft/to reproduce, still fires.mge_jit_regression_rebaseline.mdis aboutautolens_workspace_developerand was being read at_test. TheRepos:header had this answer all along, while--repohad to be told the target by hand — the open question the filing prompt raisedeinstein_radius_jit_native_seed_finder.mdquotes 23 lines of a seed finder it wants written; all 23 are absent because none ever existed, and at 101.0 it outscored every true positiveThe motivating case passes corroboration exactly: it quoted two lines from
smoke_install.sh— the PyAuto install line, still there, and the jax pin, gone.Feeds
upstream_scoreonly, neverscore: a quoted line can vanish for reasons other than the prompt shipping, so it ranks for review and never retires.Leg 5 — duplicate candidates (offline)
Every other signal scores a prompt against the completion archive. Nothing scored the live prompts against each other.
That let two filings of one finding both stay live:
bug/workspaces/jax_likelihood_pins_stale_by_1e4.md(filed 08-14) andbug/autolens/jax_likelihood_smoke_pins_stale.md(filed 08-19) — same three scripts, same failing smoke gate, five days apart, neither naming the other. The 08-19 copy was verified and retired on 08-26; the 08-14 copy kept rendering as pickable backlog until a drift audit found it on 08-27. Reconstructed from the two records'## Original promptsections, the new leg pairs them at 12.0 on the shared script paths.Measured on the live backlog: 36 pairs → 2.
start_here.py,modeling.py,no_run.yamlare workspace-wide conventions — sharing one is a convention, not a task. This alone removed 31 of the 36draft/bug/health_fixes/prompts were split by cause, so they share failing scripts and none names another; the folder'sREADME.mdnames all fourRuns on the default offline path — it reads only
draft/.Two bugs found on the way
_PATH_REopened with\b, which cannot match before a leading dot, so.github/workflows/release.ymlsilently truncated to a path resolving against no file. Dot-directories are where CI recipes live, which is half of what these prompts are about. Caught by a test..pyfiles. Genericised rather than allowlisted — growingFIREWALL_ALLOWLISTwould mean one more file an adopting fork must rewrite, for a name carrying none of the finding.AGENTS.mdkeeps the concrete names deliberately (prose is out of the firewall's scope by design). The test file was already violating its own header, which promises every fixture is fictional.The
--reposeam becomes_UpstreamReader— callable for leg 3,.quotesfor leg 4, probed withgetattrso every pre-existing lambda fake still exercises leg 3 alone.Testing
15 new tests, mostly refusals: quoted output, moved lines, an unanchored quote, proposed code, the wrong repo, one shared path, declared series, offline, never-writes. 584 pass, tenant firewall OK.
Two of the original positive fixtures were wrong in the same way the false positives were — they quoted only absent lines, which the real prompt never did. The live run caught that; the tests alone would not have.
Neither leg can produce a shipped verdict or retire anything: both stay advisory, as reconcile already promises.