Skip to content

memory: /catch_up — harvest what was missed since the last ingest - #108

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/memory-catch-up
Sep 30, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/memory-catch-up

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

After time away the suggestion tiers have lapsed, the queue has grown, and the nightly digest skipped 09-17→22 and 09-24→28. This adds one door for catching up.

  • scripts/catch_up.py (pure + CLI, stdlib): --scope lensing|interests|all [--topic T] [--since D] [--json] [--offline].
    • Cutoff = newest DONE <date> in the scope's queue sections, or the newest commit that adds an @type{key, bib entry and touches wiki/<domain>/sources/ (merges diffed vs first parent; shallow clones ignored). The 2026-09-17 seed split, which touched sources but added no entry, does not count.
    • Harvests: swept inbox/interests lines from git log -p, open reading-queue lines (via board._parse_paper), and an arXiv gap-fill that re-runs the Mind digest's QUERY (imported from PyAutoMind/.github/scripts, resolved via Brain _repo_paths.py) over submittedDate:[cutoff−3d, today]. Id-only lookups fill in missing abstracts. Interests is capped at 300 results, ranked by the digest's scorer, and reported as truncated.
    • Dedup by arXiv id, falling back to normalised title, against the bib and DONE lines. A network or Mind failure shows up as a warning and never crashes the run.
  • skills/catch_up/: the first organ-owned Memory skill. Harvest → curate into intake / cite / queue / skip → one human gate → Opus subagents file with the _read_prompt recipe → make validate → wiki/<domain>/log.md → one PR. skills is added to ALLOWED_TOP_DIRS.
  • Dashboard: once CATCH_UP_DAYS = 7 pass without an ingest, a banner appears above the hero and at the top of the markdown: "…so… I've been lost for a while", "N days since papers were last ingested into memory", and a 📋 /catch_up lensing button. state.json gains days_since_ingest. knowledge_board.yml now checks out with fetch-depth: 0 so the banner can date the ingest.

Live run (lensing, 2026-09-30)

The cutoff is 2026-09-11: the last DONE and the last bib+sources ingest (7e716dd) agree. That makes 19 days lost and 87 candidates: queue 50, history 8, arXiv gap-fill 29. None of them are already in memory.

Test plan

  • make test: 243 passed (new tests/test_catch_up.py covers a fixture git repo for the cutoff, dedup, history parsing, and arXiv stubbed; banner tests are in test_board.py)
  • make validate green
  • python scripts/board.py --md / --html show the 19-day banner above the hero

Follow-ups, outside this repo: the ~/.claude/commands/catch_up.md symlink and a Brain COMMANDS.md line. Mind's spawn.py MEMORY_RULES has no rule for skills/*, so a spawn dry-run will report it UNMATCHED until a partition rule is added.

🤖 Generated with Claude Code

scripts/catch_up.py: per scope (lensing | interests [--topic] | all),
the cutoff is the newest DONE line in the scope's queue sections or the
newest commit that adds a bib entry AND touches wiki/<domain>/sources/
(merges diffed against first parent; shallow clones ignored). Harvests
swept suggestion lines from git history, open queue lines, and an arXiv
gap-fill re-running the Mind digest's query over [cutoff-3d, today],
deduped by arXiv id / normalised title against the bib and DONE lines.
Network is best-effort: failures are warnings, never crashes.

skills/catch_up/: the first organ-owned Memory skill — harvest, curate
into intake/cite/queue/skip, one human gate, file via Opus subagents, one PR.
skills/ added to the structure allowlist.

board.py: "…so… I've been lost for a while" banner above the hero (and
atop the markdown) once CATCH_UP_DAYS=7 pass without an ingest, with a
copy button for /catch_up lensing; state.json gains days_since_ingest.
knowledge_board.yml checks out full history so the banner can date it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Jammy2211
Jammy2211 merged commit 9bd1677 into main Sep 30, 2026
1 check passed
@Jammy2211
Jammy2211 deleted the feature/memory-catch-up branch September 30, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant