Repository navigation
queue: line kinds — a heading is not a paper you can download - #62
Merged
Merged
Conversation
The backfill's 105 misses turned out not to be 105 papers arXiv couldn't find. A good share of them were never papers: `__Dark Matter__` and five other bold groupings, tracking-wrapped links pasted from email, stray prose. The board was rendering each one as a paper — a search link, three issue buttons, a slot in the "N waiting" count — and the nightly backfill was spending an arXiv lookup on each of them, every night, forever. So a queue line now has a kind, decided in `_parse_paper`: * `__Bold__` on its own line is a **subhead** — a grouping the human wrote. Renders as a heading: no link, no buttons, not counted. * `NOTE <text>` is a **note** — an annotation, sibling of the `DONE ` prefix (same shape, same never-delete rule). A bare non-arXiv link is one automatically, rendered as a link labelled by host rather than 700 characters of Outlook safelink. * Everything else is a **paper**, and only papers are counted, given buttons, or looked up on arXiv. `backfill_arxiv_refs.py --mark-unresolved` writes the `NOTE ` prefix in bulk for lines it could not identify. Deliberately opt-in and never used by the nightly: an unresolved title is more often a real paper whose line lost something on the way in (a LaTeX redshift, a wrapped subtitle) than a genuine non-paper, and those deserve re-trying once a human fixes the line. One fix falls out of the same parse: a line that is *only* an arXiv link was being read as a titleless paper with no ref, so it got a search box instead of the PDF it plainly names. It is now a paper labelled `arXiv:<id>`, with its abstract page and its 📄 button. On the live queue: 6 headings and 3 links stop pretending to be papers, the waiting count drops 232 → 223, and PDF coverage rises 127 → 133. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PjWdVFPn2WDzcqpMXTMp7G
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the backfill's misses actually were
The ref backfill (#61) resolved 125 of 230 lines. Reading the 105 misses, a good
share were never papers at all:
__Dark Matter__and five other bold groupings,tracking-wrapped links pasted out of Outlook, stray lines of prose. The board was
rendering each as a paper — a search link, three issue buttons, a slot in the
"N waiting" count — and the nightly backfill was spending an arXiv lookup on each
of them, every night, forever.
Line kinds
_parse_papernow decides what a line is:__Bold Text__alone on a lineNOTE <text>, or a bare non-arXiv linkNOTEis a deliberate sibling of the existingDONEprefix — same shape, samenever-delete rule.
The marker
backfill_arxiv_refs.py --mark-unresolvedwrites theNOTEprefix in bulk forlines it could not identify. Opt-in, and never used by the nightly run: an
unresolved title is more often a real paper whose line lost something on the way
in (a LaTeX redshift, a wrapped subtitle) than a genuine non-paper, and those
deserve re-trying once a human fixes the line. The nightly keeps asking; a human
who has read a dry run retires the ones that are genuinely prose.
A fix that falls out of the same parse
A line that is only an arXiv link was read as a titleless paper with no ref —
so it got a search box instead of the PDF it plainly names. It is now a paper
labelled
arXiv:<id>, with its abstract page and its 📄 button.Effect on the live queue
6 new tests; 113 pass,
make validateclean.Generated by Claude Code