From d6000587d201c7461d177840ecae10ec2b5bc090 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 10 Sep 2026 18:15:53 +0000 Subject: [PATCH] queue: act on label-less taps, sweep untouched issues, show filings awaiting merge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things were losing dashboard taps, and a third kept intake filings off main: - The issue form sometimes drops the `labels=` hint from the board's prefilled URL (five of twelve open issues on 2026-09-10 โ€” `queue add:` / `queue intake:` titles with no label). Both workflows gated on the label alone, so those issues sat open and silent. queue_actions.yml and queue_filing.yml now read the action from the title prefix when the label is missing (the body's `file:` tells an inbox add from an interests add), put the label back for the record, and hand a label-less ๐Ÿ“ฅ/๐Ÿ“‘ over to queue_filing.yml by dispatch. - queue_actions.yml shared one concurrency group across every issue, and GitHub cancels the older pending run when a newer arrives โ€” three โž• taps six seconds apart left the middle one cancelled (#85), before any failure step could comment. The group is now per issue; the push retry already serialises the queue commits. - A new nightly queue_sweep.yml re-dispatches every open request nothing has touched (no bot comment, no filing branch) through a `workflow_dispatch` door added to both workflows. - ๐Ÿ“ฅ/๐Ÿ“‘ filings land on `queue-filing/issue-` and the PR could not be opened (repo policy), so three gated filings sat unmerged for a week while their papers were re-tapped into the reading queue. The board now lists them first as "Filings awaiting merge" with a one-tap compare link, and the filing prompt handles a line that has since moved into the queue. A re-tap on an already-filed paper is answered with its branch, not filed twice. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Rhn9qWAjP54dP27dyEibG2 --- .github/workflows/queue_actions.yml | 183 +++++++++++++++++++++++----- .github/workflows/queue_filing.yml | 154 ++++++++++++++++++++--- .github/workflows/queue_sweep.yml | 89 ++++++++++++++ README.md | 11 ++ scripts/board.py | 78 ++++++++++++ tests/test_board.py | 53 ++++++++ 6 files changed, 517 insertions(+), 51 deletions(-) create mode 100644 .github/workflows/queue_sweep.yml diff --git a/.github/workflows/queue_actions.yml b/.github/workflows/queue_actions.yml index f655997..1f72e45 100644 --- a/.github/workflows/queue_actions.yml +++ b/.github/workflows/queue_actions.yml @@ -19,74 +19,194 @@ name: Queue actions # # Each commits the file it touched, re-renders the dashboard, and closes the # issue. ๐Ÿ“ฅ `queue-intake` and ๐Ÿ“‘ `queue-cite` are NOT handled here โ€” they stay -# open as filing work items and are driven by queue_filing.yml. +# open as filing work items and are driven by queue_filing.yml. When one of +# those arrives *without* its label (below), this workflow hands it over by +# dispatching queue_filing.yml on the issue number. +# +# The label is the intent, but it is not trusted to arrive. The dashboard's +# prefilled `issues/new?โ€ฆ&labels=x` URL is a hint to GitHub's issue form, and +# on some submissions (five of the twelve open issues on 2026-09-10 โ€” all of +# them tapped from a phone) the form dropped it, leaving an issue titled +# `queue add: โ€ฆ` with no label at all. A label-only gate then skipped both +# workflows and the issue sat open and silent for a week. So the gate reads +# the *title* prefix the board writes as well (`queue read:`, `queue add:`, +# `queue dismiss:`, `queue intake:`, `queue cite:`, `interests clear:`), +# and the resolve step puts the missing label back for the record, with the +# body's `file:` line telling an inbox add from an interests add. +# +# Two doors: the `opened` event (the dashboard path) and `workflow_dispatch` +# on an issue number (the nightly queue_sweep.yml, which re-dispatches every +# open request nothing has acted on โ€” and a human's `gh workflow run +# queue_actions.yml -f issue=N`). # # Only owner/member/collaborator-authored issues act: this is a public repo, # and a stranger's issue must never mutate the queue (they cannot apply the -# label either, but the gate does not rely on that). +# label either, but the gate does not rely on that). The dispatch door +# re-checks the association from the API, since the event carries none. on: issues: types: [opened] + workflow_dispatch: + inputs: + issue: + description: "Issue number to (re)process โ€” the sweep's door" + required: true + type: string -# contents: write โ†’ the queue commit; issues: write โ†’ comment + close; -# actions: write โ†’ re-dispatch the Dashboard workflow (token pushes do not -# retrigger `on: push` workflows). +# contents: write โ†’ the queue commit; issues: write โ†’ label + comment + close; +# actions: write โ†’ re-dispatch the Dashboard workflow and hand a label-less +# intake/cite over to queue_filing.yml (token pushes do not retrigger +# `on: push` workflows, and a token-applied label does not fire `labeled`). permissions: contents: write issues: write actions: write +# Per issue, not one group for the whole surface. GitHub keeps at most one +# *pending* run per concurrency group and cancels the older pending one when a +# newer arrives โ€” so three โž• taps six seconds apart left the middle one +# `cancelled` (#85, 2026-09-04), and a cancelled run never reaches its +# failure step: the issue stayed open with no comment, indistinguishable +# from "never swept". The push retry below is what serialises the queue +# commits on a moved main; nothing else needs a shared group. concurrency: - group: queue-actions + group: queue-actions-${{ github.event.issue.number || inputs.issue }} cancel-in-progress: false jobs: queue-action: - # One job, six labels. The gate is unchanged: this is a public repo, and - # a stranger's issue must never mutate the queue, the inbox or the - # interests list. + # One job, six labels โ€” and the six title prefixes those labels ride + # behind, for the submissions that lost the label on the way in. if: >- - (contains(github.event.issue.labels.*.name, 'queue-read') || - contains(github.event.issue.labels.*.name, 'queue-add') || - contains(github.event.issue.labels.*.name, 'queue-dismiss') || - contains(github.event.issue.labels.*.name, 'interests-add') || - contains(github.event.issue.labels.*.name, 'interests-dismiss') || - contains(github.event.issue.labels.*.name, 'interests-clear')) && - contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), - github.event.issue.author_association) + github.event_name == 'workflow_dispatch' || + (contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), + github.event.issue.author_association) && + (contains(github.event.issue.labels.*.name, 'queue-read') || + contains(github.event.issue.labels.*.name, 'queue-add') || + contains(github.event.issue.labels.*.name, 'queue-dismiss') || + contains(github.event.issue.labels.*.name, 'interests-add') || + contains(github.event.issue.labels.*.name, 'interests-dismiss') || + contains(github.event.issue.labels.*.name, 'interests-clear') || + startsWith(github.event.issue.title, 'queue read:') || + startsWith(github.event.issue.title, 'queue add:') || + startsWith(github.event.issue.title, 'queue dismiss:') || + startsWith(github.event.issue.title, 'queue intake:') || + startsWith(github.event.issue.title, 'queue cite:') || + startsWith(github.event.issue.title, 'interests clear:'))) runs-on: ubuntu-latest env: GH_TOKEN: ${{ github.token }} - ISSUE: ${{ github.event.issue.number }} - LABELS: ${{ join(github.event.issue.labels.*.name, ',') }} + ISSUE: ${{ github.event.issue.number || inputs.issue }} steps: - uses: actions/checkout@v4 + # Read the issue back from the API rather than the event: the dispatch + # door has no event payload, and one code path for both doors is the + # point. Everything downstream keys off `label` โ€” the one the issue + # carries, or the one its title says it should have carried. + - name: Resolve the request + id: resolve + run: | + issue_json=$(gh api "repos/${GITHUB_REPOSITORY}/issues/${ISSUE}") + state=$(jq -r '.state' <<<"$issue_json") + assoc=$(jq -r '.author_association' <<<"$issue_json") + title=$(jq -r '.title' <<<"$issue_json") + labels=$(jq -r '[.labels[].name] | join(",")' <<<"$issue_json") + # The body is untrusted text: to a file, never interpolated. + jq -r '.body // ""' <<<"$issue_json" > /tmp/issue_body.txt + if [ "$state" != "open" ]; then + echo "issue #${ISSUE} is ${state} โ€” nothing to do" + echo "label=" >> "$GITHUB_OUTPUT"; exit 0 + fi + case "$assoc" in + OWNER|MEMBER|COLLABORATOR) ;; + *) echo "issue #${ISSUE} author is ${assoc} โ€” not acting" + echo "label=" >> "$GITHUB_OUTPUT"; exit 0 ;; + esac + label="" + derived=no + case ",$labels," in + *,queue-read,*) label=queue-read ;; + *,queue-add,*) label=queue-add ;; + *,queue-dismiss,*) label=queue-dismiss ;; + *,interests-add,*) label=interests-add ;; + *,interests-dismiss,*) label=interests-dismiss ;; + *,interests-clear,*) label=interests-clear ;; + *,queue-intake,*) label=queue-intake ;; + *,queue-cite,*) label=queue-cite ;; + esac + if [ -z "$label" ]; then + # No action label survived the issue form. The title prefix is + # the board's own wording; the body's `file:` line says which + # suggestion tier an add/dismiss came from. + file=$(sed -n 's/^file:[[:space:]]*//p' /tmp/issue_body.txt | head -1) + tier=queue + [ "$file" = "arxiv-interests.md" ] && tier=interests + case "$title" in + "queue read:"*) label=queue-read ;; + "queue add:"*) label=${tier}-add ;; + "queue dismiss:"*) label=${tier}-dismiss ;; + "queue intake:"*) label=queue-intake ;; + "queue cite:"*) label=queue-cite ;; + "interests clear:"*) label=interests-clear ;; + esac + if [ -n "$label" ]; then + derived=yes + # For the record and for the label-filtered views; a + # token-applied label fires no `labeled` event, so the + # filing hand-over below is explicit. + gh issue edit "$ISSUE" --add-label "$label" || true + echo "issue #${ISSUE} carried no label โ€” read '${label}' from its title" + fi + fi + if [ -z "$label" ]; then + echo "issue #${ISSUE} is not a queue request (title: ${title}) โ€” nothing to do" + fi + echo "label=${label}" >> "$GITHUB_OUTPUT" + echo "derived=${derived}" >> "$GITHUB_OUTPUT" + + # ๐Ÿ“ฅ/๐Ÿ“‘ are filing work items, not mechanical actions: queue_filing.yml + # owns them. A labelled one already fired that workflow's `labeled` + # trigger on its own; a label-less one (or a sweep re-dispatch) has no + # such event, so this is the hand-over. + - name: Hand a filing request to queue_filing.yml + if: >- + contains(fromJSON('["queue-intake", "queue-cite"]'), + steps.resolve.outputs.label) && + (steps.resolve.outputs.derived == 'yes' || + github.event_name == 'workflow_dispatch') + run: | + gh workflow run queue_filing.yml --ref main -f "issue=${ISSUE}" + echo "dispatched queue_filing.yml for #${ISSUE} (${{ steps.resolve.outputs.label }})" + - name: Apply the action id: mark + if: >- + contains(fromJSON('["queue-read", "queue-add", "queue-dismiss", + "interests-add", "interests-dismiss", + "interests-clear"]'), + steps.resolve.outputs.label) env: - # env, never inline interpolation โ€” the body is untrusted text. - ISSUE_BODY: ${{ github.event.issue.body }} + LABEL: ${{ steps.resolve.outputs.label }} run: | - printf '%s' "$ISSUE_BODY" > /tmp/issue_body.txt # The label picks the script; the push retry below re-runs the same # command on a moved main, so the mapping is stated once and the # command travels as a step output. - case ",$LABELS," in - *,queue-add,*) + case "$LABEL" in + queue-add) action=add cmd="scripts/inbox_actions.py add" ;; - *,queue-dismiss,*) + queue-dismiss) action=dismiss cmd="scripts/inbox_actions.py dismiss" ;; - *,interests-add,*) + interests-add) action=interests-add cmd="scripts/interests_actions.py add" ;; - *,interests-dismiss,*) + interests-dismiss) action=interests-dismiss cmd="scripts/interests_actions.py dismiss" ;; - *,interests-clear,*) + interests-clear) action=interests-clear cmd="scripts/interests_actions.py clear" ;; *) @@ -171,6 +291,7 @@ jobs: run: gh workflow run knowledge_board.yml --ref main - name: Close or report on the issue + if: steps.mark.outcome == 'success' env: STATUS: ${{ steps.push.outputs.status || steps.mark.outputs.status }} ACTION: ${{ steps.mark.outputs.action }} @@ -199,7 +320,7 @@ jobs: "Dropped from the ${tier}. Git history still holds the line. Dashboard refresh dispatched." ;; already-gone) gh issue close "$ISSUE" --comment \ - "Not in the arXiv inbox or interests list any more โ€” nothing to change (it may already have lapsed or been cleared)." ;; + "Not in the arXiv inbox or interests list any more โ€” nothing to change (it may already have lapsed, been cleared, or been moved by an earlier tap)." ;; cleared:*) gh issue close "$ISSUE" --comment \ "Cleared ${STATUS#cleared:} paper(s) from the arXiv interests list โ€” the next day's batch is now on the dashboard. Git history still holds the batch. Dashboard refresh dispatched." ;; @@ -214,8 +335,8 @@ jobs: "Could not process this queue action (status: ${STATUS:-error}) โ€” the paper's line was not found. Handle manually; leaving the issue open." ;; esac - # The close step above is implicitly `if: success()`, so a died push - # used to leave the issue open and completely silent (#69 read as + # The close step above runs only after a successful apply, so a died + # push used to leave the issue open and completely silent (#69 read as # "never swept up" rather than "failed"). Same shape as # queue_filing.yml's failure report. - name: Report failure on the issue diff --git a/.github/workflows/queue_filing.yml b/.github/workflows/queue_filing.yml index 5ea3cf0..49ba05b 100644 --- a/.github/workflows/queue_filing.yml +++ b/.github/workflows/queue_filing.yml @@ -8,22 +8,46 @@ name: Queue filing # notes folded in), DONE-mark on the queue line โ€” and opens a reviewable PR # that closes the issue. Merge stays human. # -# Trigger is `labeled` only: GitHub fires it both for labels applied at -# creation (the dashboard path) and for labels added later, and using it alone -# avoids the double-run an `opened`+`labeled` pair would cause. Only -# owner/member/collaborator-authored issues act (public repo โ€” same gate as -# queue_actions.yml), and only triage+ users can apply labels at all. +# Two doors. `labeled` is the dashboard path: GitHub fires it both for labels +# applied at creation and for labels added later, and using it alone avoids +# the double-run an `opened`+`labeled` pair would cause. `workflow_dispatch` +# on an issue number is the hand-over from queue_actions.yml for an issue +# whose label the issue form dropped (a `queue intake: โ€ฆ` title with no +# label at all โ€” it happens on phone submissions), and the door the nightly +# queue_sweep.yml re-dispatches through. Only owner/member/collaborator- +# authored issues act (public repo โ€” same gate as queue_actions.yml), and +# only triage+ users can apply labels at all; the dispatch door re-checks +# the association from the API, since it has no event to read it from. # # The PR is created with the runner's GITHUB_TOKEN, so `pull_request` # workflows (validate.yml) do NOT run on it โ€” `make validate` + pytest run # here as the gate instead, before anything is pushed. # +# **The PR needs one repo setting.** GITHUB_TOKEN may open a PR only when +# "Allow GitHub Actions to create and approve pull requests" is on +# (Settings โ†’ Actions โ†’ General โ†’ Workflow permissions). Off, the filing +# still lands on its `queue-filing/issue-` branch, gated, and the issue +# gets a one-tap compare link instead of a PR โ€” but nothing reaches `main` +# until a human opens and merges it, and the board's *Filings awaiting +# merge* section is where those branches wait. Three did, unnoticed, between +# 2026-09-01 and 2026-09-10; the setting is the fix. +# +# A second tap on the same paper does not file it twice: a filing branch +# that already exists for this issue, or for another open issue naming the +# same paper, is answered with its compare link and the run stops clean. +# # Claude runs with tools scoped to workspace edits plus arXiv lookups; the # deterministic steps below stage explicit paths only โ€” never `git add -A`. on: issues: types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue number to file โ€” the hand-over and sweep door" + required: true + type: string # contents: write โ†’ push the filing branch; pull-requests/issues: write โ†’ the # PR + failure comment; id-token: write โ†’ claude-code-action's OIDC startup @@ -35,22 +59,23 @@ permissions: id-token: write concurrency: - group: queue-filing-${{ github.event.issue.number }} + group: queue-filing-${{ github.event.issue.number || inputs.issue }} cancel-in-progress: false jobs: file-paper: if: >- - contains(fromJSON('["queue-intake", "queue-cite"]'), - github.event.label.name) && - contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), - github.event.issue.author_association) + github.event_name == 'workflow_dispatch' || + (contains(fromJSON('["queue-intake", "queue-cite"]'), + github.event.label.name) && + contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), + github.event.issue.author_association)) runs-on: ubuntu-latest env: GH_TOKEN: ${{ github.token }} # gh needs explicit repo context in the pre-checkout secret-guard step. GH_REPO: ${{ github.repository }} - ISSUE: ${{ github.event.issue.number }} + ISSUE: ${{ github.event.issue.number || inputs.issue }} steps: - name: Require the Claude OAuth token env: @@ -87,16 +112,89 @@ jobs: - name: Install the test runner run: pip install pytest - - name: Stash the request for Claude - env: + # Read the issue back from the API rather than the event: the dispatch + # door has no event payload, and one code path for both doors is the + # point. `label` is the filing label the issue carries, or the one its + # `queue intake:` / `queue cite:` title says the form dropped. + - name: Resolve the request + id: resolve + run: | + issue_json=$(gh api "repos/${GITHUB_REPOSITORY}/issues/${ISSUE}") + state=$(jq -r '.state' <<<"$issue_json") + assoc=$(jq -r '.author_association' <<<"$issue_json") + title=$(jq -r '.title' <<<"$issue_json") + labels=$(jq -r '[.labels[].name] | join(",")' <<<"$issue_json") # env, never inline interpolation โ€” issue text is untrusted input. - ISSUE_BODY: ${{ github.event.issue.body }} - LABEL: ${{ github.event.label.name }} + jq -r '.body // ""' <<<"$issue_json" > /tmp/issue_body.txt + printf '%s' "$title" > /tmp/issue_title.txt + skip=no + if [ "$state" != "open" ]; then + echo "issue #${ISSUE} is ${state} โ€” nothing to file"; skip=yes + fi + case "$assoc" in + OWNER|MEMBER|COLLABORATOR) ;; + *) echo "issue #${ISSUE} author is ${assoc} โ€” not acting"; skip=yes ;; + esac + label="${{ github.event.label.name }}" + if [ -z "$label" ]; then + case ",$labels," in + *,queue-intake,*) label=queue-intake ;; + *,queue-cite,*) label=queue-cite ;; + esac + fi + if [ -z "$label" ]; then + case "$title" in + "queue intake:"*) label=queue-intake ;; + "queue cite:"*) label=queue-cite ;; + esac + if [ -n "$label" ]; then + gh issue edit "$ISSUE" --add-label "$label" || true + echo "issue #${ISSUE} carried no label โ€” read '${label}' from its title" + fi + fi + if [ -z "$label" ]; then + echo "issue #${ISSUE} is not a filing request (title: ${title}) โ€” nothing to do" + skip=yes + fi + printf '%s' "$label" > /tmp/issue_label.txt + echo "label=${label}" >> "$GITHUB_OUTPUT" + echo "skip=${skip}" >> "$GITHUB_OUTPUT" + + # Idempotence for re-taps and re-dispatches: a filing that already + # reached a branch is answered with where it is, not filed again (a + # second push to the same branch would not fast-forward anyway, and a + # second branch for the same paper is a merge conflict waiting to + # happen). The sweep treats the comment as "touched" and leaves it. + - name: Refuse a duplicate filing + id: dup + if: steps.resolve.outputs.skip != 'yes' run: | - printf '%s' "$ISSUE_BODY" > /tmp/issue_body.txt - printf '%s' "$LABEL" > /tmp/issue_label.txt + skip=no + compare="${GITHUB_SERVER_URL}/${GH_REPO}/compare/main...queue-filing/issue-" + existing=$(git ls-remote --heads origin 'refs/heads/queue-filing/*' \ + | sed -n 's#.*refs/heads/queue-filing/issue-\([0-9]*\)$#\1#p') + if grep -qx "$ISSUE" <<<"$existing"; then + gh issue comment "$ISSUE" --body \ + "This paper is already filed on branch \`queue-filing/issue-${ISSUE}\` โ€” nothing was filed again. Open its PR with one tap: ${compare}${ISSUE}?expand=1 โ€” merging it closes this issue." + skip=yes + else + title=$(cat /tmp/issue_title.txt) + paper="${title#queue intake: }"; paper="${paper#queue cite: }" + for n in $existing; do + other=$(gh api "repos/${GH_REPO}/issues/${n}" --jq '.title' 2>/dev/null || true) + o_paper="${other#queue intake: }"; o_paper="${o_paper#queue cite: }" + if [ -n "$paper" ] && [ "$paper" = "$o_paper" ]; then + gh issue comment "$ISSUE" --body \ + "This paper is already filed on branch \`queue-filing/issue-${n}\` (from #${n}) โ€” nothing was filed again. Open that PR with one tap: ${compare}${n}?expand=1 โ€” then close this issue." + skip=yes + break + fi + done + fi + echo "skip=${skip}" >> "$GITHUB_OUTPUT" - name: File the paper with Claude + if: steps.resolve.outputs.skip != 'yes' && steps.dup.outputs.skip != 'yes' uses: anthropics/claude-code-action@v1 with: # Claude subscription OAuth token โ€” NOT an API key (the org pattern @@ -156,16 +254,28 @@ jobs: delete the line from whichever of those two files `file:` names. A suggestion line is not reading history, so deleting it is correct โ€” the DONE line it becomes is the history. + - The line may have MOVED since the request was written: an + earlier โž• tap moves a suggestion into reading-queue.md + unread, and the nightly sweep lapses old inbox lines. So if + `file:` names a suggestion tier and the line is not there, + look for the same paper (by arXiv ref first, then title) in + reading-queue.md: an unread line there is DONE-marked in + place, exactly as the `reading-queue.md` case above. If it + is already DONE-marked anywhere in reading-queue.md, the + paper has been filed before โ€” stop and report that, without + editing anything. 5. Run `make validate` and `python -m pytest tests/ -q`; fix your own filing mistakes until both are green. Touch ONLY bibliography/, wiki/, reading-queue.md, arxiv-inbox.md and arxiv-interests.md. Do not commit โ€” the workflow handles git. If the paper cannot be - identified confidently, write the reason to /tmp/filing_error.txt - and stop without editing anything. + identified confidently, or its line cannot be found in any of + the three files, write the reason to /tmp/filing_error.txt and + stop without editing anything. - name: Abort cleanly if Claude could not identify the paper + if: steps.resolve.outputs.skip != 'yes' && steps.dup.outputs.skip != 'yes' run: | if [ -f /tmp/filing_error.txt ]; then gh issue comment "$ISSUE" --body-file /tmp/filing_error.txt @@ -174,11 +284,13 @@ jobs: fi - name: Gate the filing (validate + tests) + if: steps.resolve.outputs.skip != 'yes' && steps.dup.outputs.skip != 'yes' run: | make validate python -m pytest tests/ -q - name: Branch, commit, push, open the PR (explicit paths only) + if: steps.resolve.outputs.skip != 'yes' && steps.dup.outputs.skip != 'yes' run: | git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" @@ -210,8 +322,10 @@ jobs: else # Repo/org policy can forbid Actions-created PRs. The branch is # pushed and gated โ€” degrade to a one-tap PR link, not a failure. + # The board lists the branch under "Filings awaiting merge" until + # it lands or is deleted. gh issue comment "$ISSUE" --body \ - "Filing branch pushed and gated: \`${branch}\` โ€” but this repo's policy blocks Actions from opening PRs. Open it with one tap: ${GITHUB_SERVER_URL}/${GH_REPO}/compare/main...${branch}?expand=1 โ€” or enable 'Allow GitHub Actions to create and approve pull requests' (Settings โ†’ Actions โ†’ General) to make this automatic." + "Filing branch pushed and gated: \`${branch}\` โ€” but this repo's policy blocks Actions from opening PRs. Open it with one tap: ${GITHUB_SERVER_URL}/${GH_REPO}/compare/main...${branch}?expand=1 โ€” or enable 'Allow GitHub Actions to create and approve pull requests' (Settings โ†’ Actions โ†’ General) to make this automatic. Until it is merged, the paper is on that branch and not in memory; the dashboard lists it under *Filings awaiting merge*." fi - name: Report failure on the issue diff --git a/.github/workflows/queue_sweep.yml b/.github/workflows/queue_sweep.yml new file mode 100644 index 0000000..614837c --- /dev/null +++ b/.github/workflows/queue_sweep.yml @@ -0,0 +1,89 @@ +name: Queue sweep + +# The nightly broom behind the dashboard's per-paper buttons. Every button is +# an issue, and an issue is acted on by an *event* โ€” `opened` for +# queue_actions.yml, `labeled` for queue_filing.yml. Two things lose the +# event: an issue form that drops the label on the way in (a `queue add: โ€ฆ` +# title with no label โ€” it happens on phone submissions; five of the twelve +# open issues on 2026-09-10), and a run that GitHub cancels before it starts +# (a shared concurrency group did that to a sibling tap). Either way the +# issue sat open, un-commented, indistinguishable from "never swept". +# +# This job lists every open issue whose title is one of the board's request +# prefixes and re-dispatches queue_actions.yml on each that nothing has +# touched yet โ€” no comment from the bot, no filing branch. An issue the bot +# has already answered is a human's turn (a failed filing, a paper that has +# moved), and one whose `queue-filing/issue-` branch exists is awaiting +# merge, which the board shows. It is idempotent: the action scripts answer +# `already-there` / `already-gone` / `already-done` for work that landed +# by another tap, and queue_actions closes the issue on those too. +# +# Read-only here beyond the dispatch itself: the sweep mutates nothing, the +# dispatched runs do. Runs after both overnight digests have landed. + +on: + schedule: + - cron: "5 8 * * *" + workflow_dispatch: + +# issues: read โ†’ the listing; actions: write โ†’ the per-issue dispatch; +# contents: read โ†’ the checkout that ls-remote runs from. +permissions: + contents: read + issues: read + actions: write + +concurrency: + group: queue-sweep + cancel-in-progress: false + +jobs: + sweep: + runs-on: ubuntu-latest + env: + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + steps: + - uses: actions/checkout@v4 + + - name: Re-dispatch every untouched request + run: | + filed=$(git ls-remote --heads origin 'refs/heads/queue-filing/*' \ + | sed -n 's#.*refs/heads/queue-filing/issue-\([0-9]*\)$#\1#p') + { + echo "## Queue sweep" + echo + gh api --paginate "repos/${GH_REPO}/issues?state=open&per_page=100" \ + --jq '.[] | select(.pull_request == null) + | [.number, .author_association, .comments, .title] | @tsv' \ + | while IFS=$'\t' read -r num assoc comments title; do + case "$title" in + "queue read:"*|"queue add:"*|"queue dismiss:"*|\ + "queue intake:"*|"queue cite:"*|"interests clear:"*) ;; + *) continue ;; + esac + # Same gate as the workflows it feeds: a stranger's issue + # never drives the queue. + case "$assoc" in + OWNER|MEMBER|COLLABORATOR) ;; + *) continue ;; + esac + if grep -qx "$num" <<<"$filed"; then + echo "- #${num}: filed on \`queue-filing/issue-${num}\` โ€” awaiting merge (on the board)" + continue + fi + if [ "${comments:-0}" -gt 0 ]; then + bot=$(gh api "repos/${GH_REPO}/issues/${num}/comments" \ + --jq '[.[] | select(.user.login == "github-actions[bot]")] | length') + if [ "${bot:-0}" -gt 0 ]; then + echo "- #${num}: already answered by the bot โ€” a human's turn" + continue + fi + fi + if gh workflow run queue_actions.yml --ref main -f "issue=${num}"; then + echo "- #${num}: re-dispatched โ€” ${title}" + else + echo "- #${num}: dispatch FAILED โ€” ${title}" + fi + done + } | tee -a "$GITHUB_STEP_SUMMARY" diff --git a/README.md b/README.md index 23ba504..901dbe1 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,17 @@ paper links to its arXiv abstract page and carries a ๐Ÿ“„ button onto the PDF itself, so a phone can collect a stack of papers to read offline in one tap each. +The per-paper buttons file GitHub issues that two workflows act on: +`queue_actions.yml` makes the mechanical moves (โž• โœ… โœ–๏ธ ๐Ÿงน) and closes the +issue; `queue_filing.yml` has Claude file a ๐Ÿ“ฅ/๐Ÿ“‘ paper onto a +`queue-filing/issue-` branch, gated, for a human to merge โ€” and it opens +that PR itself only once *Allow GitHub Actions to create and approve pull +requests* is on (Settings โ†’ Actions โ†’ General). A tap whose label the issue +form dropped is read from its title; whatever nothing acted on is +re-dispatched by the nightly `queue_sweep.yml`; and a filing that reached its +branch but not `main` sits at the top of the board under **Filings awaiting +merge** until its PR lands โ€” the paper is not in memory before then. + ## Current contents diff --git a/scripts/board.py b/scripts/board.py index 0da891e..6047954 100644 --- a/scripts/board.py +++ b/scripts/board.py @@ -165,6 +165,40 @@ def _owner_repo() -> tuple[str, str]: return (m.group(1), m.group(2)) if m else ("", "") +# queue_filing.yml pushes each filing to `queue-filing/issue-`; the PR +# that merges it is a human's (and needs a repo setting to be opened by the +# workflow at all โ€” see the workflow's header). +FILING_BRANCH_RE = re.compile(r"refs/heads/queue-filing/issue-(\d+)$") + + +def _pending_filings() -> list[dict]: + """Filing branches on the remote that nobody has merged yet. + + A ๐Ÿ“ฅ/๐Ÿ“‘ tap ends on a branch, not on main: until its PR is merged the + paper is *not* in memory, and nothing on the page said so โ€” three such + branches sat unnoticed for a week (2026-09-01 โ†’ 09-10) while their + papers were re-tapped into the reading queue instead. One `ls-remote` + against origin; any failure (no remote, offline, no git) is an empty + list, never an error, so a spawned template or a laptop on a train + still renders. + """ + try: + out = subprocess.run( + ["git", "-C", str(MEMORY_HOME), "ls-remote", "--heads", "origin", + "refs/heads/queue-filing/*"], + capture_output=True, text=True, timeout=30, + ).stdout + except (OSError, subprocess.SubprocessError): + return [] + found = [] + for line in out.splitlines(): + m = FILING_BRANCH_RE.search(line.strip()) + if m: + n = int(m.group(1)) + found.append({"issue": n, "branch": f"queue-filing/issue-{n}"}) + return sorted(found, key=lambda f: f["issue"]) + + # --- collect (local files only) ---------------------------------------------- def collect(root: Path | None = None) -> dict: root = root or MEMORY_HOME @@ -187,6 +221,9 @@ def collect(root: Path | None = None) -> dict: "interests_backlog": 0, "interests_last_digest": None, "links": {"total": 0, "unique": 0, "wanted": 0}, + # Filings that reached a `queue-filing/issue-` branch but not + # main โ€” the one thing on the page that is a human's merge to make. + "filings": _pending_filings() if root == MEMORY_HOME else [], } stems: set[str] = set() @@ -674,6 +711,22 @@ def _render_md(snapshot: dict) -> str: lines.append(f"| {w['name']} | {w['pages']} | {s.get('stub', 0)} | " f"{s.get('drafted', 0)} | {s.get('reviewed', 0)} | " f"{w.get('todo', 0)} |") + filings = snapshot.get("filings") or [] + if filings: + repo_url = _repo_url(snapshot) + lines += ["", "## Filings awaiting merge", "", + f"_{len(filings)} filing{'s' if len(filings) != 1 else ''} " + "reached a branch but not `main` โ€” a paper is not in memory " + "until its PR is merged._", ""] + for f in filings: + if repo_url: + lines.append( + f"- #{f['issue']} โ€” [issue]({repo_url}/issues/{f['issue']}) ยท " + f"[open the PR โ†’]({repo_url}/compare/main...{f['branch']}" + f"?expand=1)") + else: + lines.append(f"- #{f['issue']} โ€” `{f['branch']}`") + lines.append("") lines += ["", "## arXiv inbox", ""] inbox = snapshot.get("inbox") or [] fresh = _inbox_freshness(snapshot) @@ -1170,6 +1223,30 @@ def _fresh_span(fresh: dict, now_text: str, stale_text: str, f"{snapshot.get('interests_backlog') or 0} papers" if n_batches else "") + filings = snapshot.get("filings") or [] + filing_items = [] + for f in filings: + if repo_url: + issue_link = (f"" + f"#{f['issue']}") + pr = (f"" + f"\U0001f500 open PR") + else: + issue_link, pr = f"#{f['issue']}", "" + filing_items.append( + f"
  • {issue_link} " + f"{_html.escape(f['branch'])}{pr}
    • ") + # Above everything else when present, absent otherwise: it is the one + # section that is a human's merge, and an empty "nothing waiting" line + # would only teach the eye to skip it. + filings_block = ( + f"

    Filings awaiting merge ({len(filing_items)} " + f"on a branch, not yet in memory โ€” merge the PR to finish the " + f"intake)

      {''.join(filing_items)}
    " + if filing_items else "") + hero = t_.hero(BOARD_KEY, "Dashboard", _LEDE) stats = t_.stats((t["pages"], "Pages"), (f"{pct}%", "Cited"), (t["todo"], "To cite"), (t["queued"], "Queued")) @@ -1188,6 +1265,7 @@ def _fresh_span(fresh: dict, now_text: str, stale_text: str, {hero} {stats}

    markdown version{github_link}

    +{filings_block}

    arXiv inbox (suggested overnight โ€” un-acted papers lapse after {inbox_actions.INBOX_WINDOW_DAYS} days)

    {inbox_block} diff --git a/tests/test_board.py b/tests/test_board.py index 4eeaa22..8790265 100644 --- a/tests/test_board.py +++ b/tests/test_board.py @@ -665,3 +665,56 @@ def test_the_two_tiers_carry_their_own_stamps(tmp_path): assert "data-last-digest='2026-08-27'" in html assert "data-last-digest='2026-08-20'" in html + + +# --- filings awaiting merge --------------------------------------------------- +def test_a_synthetic_tree_has_no_pending_filings(tmp_path): + """Only the live checkout asks the remote; a synthetic root never does.""" + assert board.collect(_tree(tmp_path))["filings"] == [] + + +def test_pending_filings_are_parsed_from_ls_remote(monkeypatch): + import types + out = ("ab3b\trefs/heads/queue-filing/issue-91\n" + "ad7d\trefs/heads/queue-filing/issue-72\n" + "ffff\trefs/heads/feature/queue-filing-not-a-filing\n") + monkeypatch.setattr(board.subprocess, "run", + lambda *a, **k: types.SimpleNamespace(stdout=out)) + assert board._pending_filings() == [ + {"issue": 72, "branch": "queue-filing/issue-72"}, + {"issue": 91, "branch": "queue-filing/issue-91"}, + ] + + +def test_a_failed_ls_remote_is_an_empty_list_not_an_error(monkeypatch): + def boom(*a, **k): + raise OSError("no git here") + monkeypatch.setattr(board.subprocess, "run", boom) + assert board._pending_filings() == [] + + +def test_pending_filings_render_above_the_inbox_with_a_pr_link(tmp_path): + """A filing on a branch is not in memory: the board says so, first, and + hands out the one tap that finishes it.""" + snap = _snap_with_remote(tmp_path) + snap["filings"] = [{"issue": 91, "branch": "queue-filing/issue-91"}] + for text in (board._render_md(snap), board._render_html(snap)): + assert "Filings awaiting merge" in text + assert "compare/main...queue-filing/issue-91?expand=1" in text + assert "/issues/91" in text + assert text.index("Filings awaiting merge") < text.index("arXiv inbox") + + +def test_no_filings_section_when_nothing_waits(tmp_path): + snap = _snap_with_remote(tmp_path) + assert snap["filings"] == [] + for text in (board._render_md(snap), board._render_html(snap)): + assert "Filings awaiting merge" not in text + + +def test_pending_filings_without_a_remote_still_name_the_branch(tmp_path): + snap = board.collect(_tree(tmp_path)) + snap["filings"] = [{"issue": 7, "branch": "queue-filing/issue-7"}] + for text in (board._render_md(snap), board._render_html(snap)): + assert "queue-filing/issue-7" in text + assert "compare/main" not in text