Nested git repos never reindex on branch switch: watcher only tracks session_project, index_status reports false freshness

Summary

In a multi-repo monorepo workspace (parent git root + many nested independent git checkouts), background auto-watch / reindex only tracks the MCP session project (cwd root). Branch switches and HEAD moves in already-indexed nested repositories are ignored.

Worse: index_status / list_projects surface live branch / head_sha from disk git, while the knowledge graph remains on the previously indexed tree. Agents therefore treat a stale graph as fresh and answer structural questions incorrectly with no error signal.

This is a high-severity reliability issue for agent workflows: silent wrong answers, not a hard failure.

Environment

Item	Value
Version	`codebase-memory-mcp 0.9.0` (darwin-arm64 release binary)
OS	macOS (arm64)
Config	`auto_index=true`, `auto_watch=true`, `auto_index_limit=50000`
Client	Agent MCP session with cwd = monorepo parent
Cache	default `~/.cache/codebase-memory-mcp`

Setup shape (common agent monorepo)

/path/to/workspace/                 ← git root; MCP session_root
  group/nested-service/             ← nested independent git repo (already indexed)
  other-nested-repos/               ← also indexed as separate projects

Session project name: derived from parent path (e.g. Users-...-workspace)
Nested project name: e.g. nested-service (indexed earlier via index_repository)

The nested repo has mutually exclusive branch trees (different modules/files per branch). Example: branch A has module service-variant-a/, branch B has service-variant-b/ instead.

Reproduction

Start MCP from the parent monorepo root. Confirm autoindex/watcher only registers the session project:

level=info msg=autoindex.start project=<session-project> path=/path/to/workspace
level=info msg=watcher.watch project=<session-project> path=/path/to/workspace
level=info msg=watcher.baseline project=<session-project> strategy=git files=yes

No watcher.watch line appears for the nested project.

Ensure nested repo is indexed and on branch A:

codebase-memory-mcp cli index_status --project nested-service
# nodes ~N, graph contains symbols under service-variant-a/

Clean switch nested repo to a very different branch B:

cd group/nested-service
git status --porcelain   # empty
git switch branch-b
# service-variant-a gone from disk; service-variant-b present

Wait well beyond poll interval (minutes). Re-check:

codebase-memory-mcp cli index_status --project nested-service

Actual

Surface	After branch switch
Live files on disk	Branch B tree
`index_status.git.branch` / `head_sha`	Branch B (live)
Graph `nodes` / `edges`	Unchanged from branch A
`search_graph` for branch-A-only symbols	Still hits (e.g. paths under `service-variant-a/`)
`search_graph` for branch-B paths	Missing / empty
`search_code` (disk grep)	Correctly reflects branch B
stderr `watcher.reindex` for nested project	Never

Expected

Either:

Nested indexed projects under (or beside) the session are watched and reindexed when their HEAD moves; or
Query surfaces refuse / flag staleness when graph snapshot ≠ current HEAD; and
index_status exposes an indexed HEAD (or stale: true), not only live git metadata that always looks current.

Control: manual reindex works

codebase-memory-mcp cli index_repository \
  --repo-path /path/to/workspace/group/nested-service \
  --name nested-service \
  --mode fast

Branch B → fewer/different nodes; service-variant-b/* present, variant A gone from file_hashes
Switch back + reindex → graph matches branch A again

So the pipeline can rebuild correctly; automatic path never targets the nested project.

Note: omitting --name derived a second project name from absolute path, leaving the original project stale. Separate naming footgun, but not the core bug.

Root cause (source read on main / v0.9.0-era tree)

1. Watcher registration is session-scoped only

register_watcher_if_enabled only watches srv->session_project / srv->session_root:

/* Register the session project with the background watcher for ongoing
 * change detection — unless auto_watch is disabled. */
static void register_watcher_if_enabled(cbm_mcp_server_t *srv) {
    if (!srv->watcher || srv->session_project[0] == '\0' || srv->session_root[0] == '\0') {
        return;
    }
    ...
    cbm_watcher_watch(srv->watcher, srv->session_project, srv->session_root);
}

Already-indexed nested projects are never registered unless the MCP session cwd is that nested root.

2. Watcher can detect checkout when registered

watcher.c tracks git rev-parse HEAD and treats HEAD movement (commit / checkout / pull) as a change. Nested repos never reach this path.

3. Parent git HEAD does not move when a nested repo switches branches

Even continuous reindex of the session/parent project does not rebuild the nested project's DB. Nested .git trees are separate indexes.

4. False freshness compounds the failure

index_status / project list git fields read live git state (also discussed in #1213). After a nested branch switch, agents see matching branch/sha and assume the graph is current — while search_graph still returns symbols deleted on disk.

Related open issues (distinct but compounding):

Watcher re-index leaves MCP server serving a stale cached store (POSIX, silent) #1191 / MCP session serves a stale graph indefinitely after the background watcher re-indexes #1296 — watcher reindex may leave MCP serving a stale cached store (POSIX)
Branch.head_sha stale after re-indexing an existing project (freezes at creation-time HEAD) #1213 — Branch / head_sha freshness signals unreliable
Feature request: worktree overlay indexing for branch-aware agent workflows #573 — branch/worktree overlay (longer-term product direction)

This report is specifically: unwatched nested projects + live git metadata on stale graphs.

Impact

Agent monorepos with many nested git checkouts are a common CBM install shape.
Silent structural wrong answers on branch-divergent code (different modules / layouts per branch).
Worse than “index missing”: tools return confident, high-quality looking wrong results.
No reliable client-side freshness check today without external bookkeeping.

Severity: high for agent reliability (silent incorrect graph).

Suggested fixes (shortest path first)

Watch all indexed projects (or all projects whose root_path is under session root / configured roots), not only session_project.
Persist indexed_head_sha (and branch) at successful index time; expose in index_status with stale = (indexed_head_sha != live_head_sha).
On stale, either auto-queue reindex or return an explicit staleness flag on graph tools so agents do not trust results blindly.
Keep Watcher re-index leaves MCP server serving a stale cached store (POSIX, silent) #1191/MCP session serves a stale graph indefinitely after the background watcher re-indexes #1296 store invalidation so successful reindex is visible to long-lived MCP sessions.
Docs: state clearly that auto_watch only covers the session project today, and nested multi-repo setups need per-repo reindex or separate sessions until (1) lands.

Workaround (current)

After every git switch / checkout in a nested indexed repo:

codebase-memory-mcp cli index_repository \
  --repo-path <nested-repo> \
  --name <existing-project-name> \
  --mode fast

Or keep one clone per long-lived branch as separate project roots (avoids in-place branch switches).

Offer

Happy to refine the repro into a small fixture (parent repo + nested repo with two branches of divergent file trees) if useful for a regression test.

I reproduced the same false-freshness outcome on Windows with
codebase-memory-mcp 0.10.8, but without requiring a nested repository.

Observed sequence:

The project graph was indexed while the working tree contained symbols from
a later feature branch.
The watcher/session was unregistered or restarted.
The repository was cleanly switched to another branch/HEAD.
On re-registration, the watcher logged a new Git baseline but did not
reindex.
index_status.git.head_sha reflected the live checkout, while
search_graph continued returning symbols absent from that checkout.

The project database last-write timestamp also predated the current checkout
state, so this was not only a stale in-process reader: the persisted graph
itself had not been rebuilt.

Source inspection appears to explain the lifecycle gap:

index_status resolves Git context live from root_path.
watcher last_head is process-local.
init_baseline() adopts the current HEAD without comparing it against a
persisted indexed HEAD.
therefore a clean checkout change that happens while unwatched cannot be
distinguished from a fresh graph after re-registration.

This means the graph can remain stale indefinitely after the new HEAD is
adopted as the baseline; waiting for another poll is insufficient.

Persisting indexed_head_sha at successful publication, comparing it during
watcher initialization, and exposing indexed-vs-live Git state (or stale)
would cover both this lifecycle case and the false-freshness surface described
here.

@bosschen0429

Reproducing from macOS (Apple Silicon) on 0.11.0 with two additions that may help
shape the fix: a per-file freshness oracle already exists and works, and an
mtime-only change latches metadata_changed irrecoverably.

1. False freshness reproduces without a branch switch or a nested repo

A project simply left un-reindexed while commits accumulate behaves identically.

Users-kanyun-it-services-its-ms-attendance (~/it-services/its-ms-attendance):

Signal	Value
`indexed_at`	`2026-08-06T08:01:34Z`
commits since index	20
last commit	`2026-09-18 17:13:35 +0800`
`index_status.git.head_sha`	matches current `HEAD` exactly
`status`	`ready`
`parse_partial` / `skipped`	empty

Both surfaced fields look healthy; the graph is six weeks behind. Verified by content,
not inference — of 12 functions added in that window, all 12 are absent, and the
file carrying most of them has zero nodes:

$ for f in DefaultDirectoryMappingConfig LoadDirectoryMappingConfig validPositionGroup \
           mergeDirectoryUnmatchedPool positionGroupForRegion directoryPoolID ...; do
    sqlite3 "$DB" "SELECT COUNT(*) FROM nodes WHERE name='$f';"; done
# 12 checked -> 12 missing
$ sqlite3 "$DB" "SELECT COUNT(*) FROM nodes WHERE file_path LIKE '%directory_unmatched%';"
0

No snapshot exists to compare against: projects holds only name / indexed_at /
root_path, and store_meta is empty. This is consistent with @bosschen0429's
lifecycle analysis — indexed_head_sha is genuinely absent from the store.

2. `check_index_coverage` already carries the per-file staleness signal

This is the stale signal point 3 asks for, per path, and it works. Clean A/B on a
current-format project, nothing touched but mtime:

# before
lib/model-scope.ts  no_recorded_issue  metadata_match    use_graph_with_best_effort_caveat
# after `touch lib/model-scope.ts` (content byte-identical)
lib/model-scope.ts  no_recorded_issue  metadata_changed  read_source_and_reindex

So the oracle exists and is reachable today. Surfacing it at project level would
reuse machinery that is already there rather than adding a second notion of staleness.

3. An mtime-only change latches `metadata_changed` permanently

metadata_changed cannot be cleared by re-indexing when the content is unchanged —
neither the default mode nor --mode full refreshes the stored mtime_ns:

# baseline
freshness=metadata_match    rec=1789788292866885000  disk=1789788292866885000
# touch lib/paths.ts  (content byte-identical)
freshness=metadata_changed  rec=1789788292866885000  disk=1789788490508198000
# index_repository (default)
freshness=metadata_changed  rec=1789788292866885000  disk=1789788490508198000
# index_repository --mode full
freshness=metadata_changed  rec=1789788292866885000  disk=1789788490508198000

A content change does clear it (rec and disk converge, metadata_match
returns), because the incremental path compares hashes, correctly concludes the file
changed, and then writes fresh metadata. An mtime-only change is judged "unchanged"
and skipped entirely — including the metadata write.

Consequence: any tool that rewrites mtimes without changing content pins the project
in metadata_changed indefinitely. A git checkout in CI, a git stash/restore, an
editor that re-saves identical bytes, or a tar/rsync restore all do this. I confirmed
recovery requires a real content edit; re-indexing alone never does it.

4. `get_code_snippet` serves shifted source under this state (observed on 0.11.0)

I could not test PR #1788's source_drift path directly — strings finds no
source_drift symbol in the 0.11.0 binary, so that fix is not in this release. On
0.11.0 the behaviour is the pre-#1788 one, and it is worth restating because it is
silent:

Inserting two lines above a function (indexed start_line: 46; the function is now at
line 48):

$ codebase-memory-mcp cli get_code_snippet --project ... --qualified-name ...toNativePath
start_line: 46
end_line: 49
source: |
   * Only pass paths — a branch name like `feature/x` would become `feature\x`.
   */
  export function toNativePath(p: string): string {
    if (!p || process.platform !== "win32") return p;

The reported range is unchanged, the slice is shifted, and the first line is the tail
of the preceding JSDoc block — presented as the function source. No error, no flag.

Suggestions

Points 1–4 in the issue are the right shape. Three additions:

Reuse the existing oracle at project level. Report stale when any recorded
file's metadata no longer matches, using the same reader check_index_coverage
already uses, instead of a new signal that could disagree with it.
Refresh mtime_ns on the incremental path even when the content hash matches.
This removes the one-way door in §3 with a one-line metadata write.
Scope the fix beyond branch switches. §1 needed no nested repo and no checkout;
§3 is independent of git entirely. A fix framed only as "watcher should track
nested HEADs" would leave both in place.

Happy to retest against a candidate build.

Thank you for the v0.11.0 evidence. The watcher still gates work on project registration, so the earlier dirty-to-clean fix does not settle this nested-project report. The stale generation, persistent metadata mismatch and shifted snippet need separate checks. This remains open; the earlier watcher fix is not being treated as proof that these cases are resolved.

@xnng

Thank you, @xnng, @bosschen0429 and @zhangbiyao1129! Between you, you separated the lifecycle cause from the reporting surface. There are now two linked tracks:

feat(mcp): report truthful index freshness from checkout evidence #1561 persists the checkout a graph was built from and reports a freshness verdict with reasons. That addresses the false freshness.
fix(watcher): show watch state in index_status, correct auto-sync docs (#2167) #2348 shows in index_status whether a project is actually being watched.

Neither of them makes nested repositories watched: the watcher is still registered only for the session project. That part remains open here.

Confirming this on 0.11.0 / Windows, in a host that serves many session workspaces (the MCP client's workspace here is a non-git parent directory containing several independent git checkouts), and adding the mechanism plus a concrete ask.

Mechanism (measured from ~/.cache/codebase-memory-mcp/logs/cbm-daemon.log): the watch is registered by the client that creates the session-managed daemon, keyed to that client's cwd; clients that merely join an already-active daemon register nothing.

daemon.start 580 / daemon.lifetime_end 579 (reason=runtime_exited) — the daemon, and its watch, live only as long as the creating client.
Every daemon.start is followed by exactly one watcher.watch naming the creator's cwd: …-work (the session workspace) 598 times, …-dsh-codebase-memory 118 times, and the nested repo I was actually editing (…-ruankao-ai) 0 times — despite many cli index_repository --repo-path …/ruankao-ai runs (a transient CLI is not the creator, so it acquires no watch).
Two probes where I spawned my own long-lived MCP client with cwd = <nested repo> while a daemon was already held by the host's client: watcher.watch / baseline / changed / unwatch = 0 / 0 / 0 / 0 for both. Joining does not add a watch, even for a fully interactive client.
For the non-git parent, watcher.baseline … strategy=none and changed=0 throughout — no polling at all, matching the source comment.

Quantifying the silent-freshness half (same repo): after the index had drifted, I sampled 59 symbols and 20 (34%) returned a line range that does not contain the named symbol — e.g. adminGate claimed 27-34 while the file has it at 47; registerRoutes claimed 36-382 vs actual 56. index_status reports ready throughout, so an agent reads a neighbour function under the correct name, exactly as you describe.

Ask: an explicit registration surface — e.g. daemon watch <path>, an MCP watch_project tool, or a config list of watch roots — so a host serving many workspaces can register the projects it cares about. Worth noting the per-project-MCP-server workaround does not help either: with one shared daemon, the extra clients join it and register nothing.

Cross-referencing #1660 (session-scoped watch); the data above refines its wording: it is not "an interactive client connects", but specifically the daemon-creating client that registers.

Thank you for the Windows measurements distinguishing daemon creation from joining an existing daemon. They clarify why a separate interactive client did not add the expected watch. The freshness/reporting tracks named above do not settle registration itself; I have no new fix to report and the explicit registration surface remains a maintainer decision.

Thank you for the joining-client measurements. Watch registration itself remains open beyond the freshness/reporting tracks, and the explicit registration interface needs a maintainer decision. Sorry for the delay; this is waiting on maintainer follow-up, and I’ll follow up soon.

I have a question related to this bug, I think. I had the problem that using github copilot inside of rider (a jetbrains IDE) didn't watch the projects, because it was using the root path of something else instead of the project. The question is: why does the path of the MCP session has any relevance in all of this? If an agent opens a session and queries about the project X, I think that it should simply watch the project's path (the one we select when we create an index), not trying to infer it from some MCP session working dir. Probably it should force a reindex when a session is opened, so we can solve #2743 too (or rather only if the index is stale).

Navigation Menu

Repository navigation

Nested git repos never reindex on branch switch: watcher only tracks session_project, index_status reports false freshness #1429

Summary

Environment

Setup shape (common agent monorepo)

Reproduction

Actual

Expected

Control: manual reindex works

Root cause (source read on main / v0.9.0-era tree)

1. Watcher registration is session-scoped only

2. Watcher can detect checkout when registered

3. Parent git HEAD does not move when a nested repo switches branches

4. False freshness compounds the failure

Impact

Suggested fixes (shortest path first)

Workaround (current)

Offer

1. False freshness reproduces without a branch switch or a nested repo

2. `check_index_coverage` already carries the per-file staleness signal

3. An mtime-only change latches `metadata_changed` permanently

4. `get_code_snippet` serves shifted source under this state (observed on 0.11.0)

Suggestions

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Nested git repos never reindex on branch switch: watcher only tracks session_project, index_status reports false freshness #1429

Description

Summary

Environment

Setup shape (common agent monorepo)

Reproduction

Actual

Expected

Control: manual reindex works

Root cause (source read on main / v0.9.0-era tree)

1. Watcher registration is session-scoped only

2. Watcher can detect checkout when registered

3. Parent git HEAD does not move when a nested repo switches branches

4. False freshness compounds the failure

Impact

Suggested fixes (shortest path first)

Workaround (current)

Offer

Activity

bosschen0429 commented on Aug 27, 2026

zhangbiyao1129 commented on Sep 19, 2026

1. False freshness reproduces without a branch switch or a nested repo

2. check_index_coverage already carries the per-file staleness signal

3. An mtime-only change latches metadata_changed permanently

4. get_code_snippet serves shifted source under this state (observed on 0.11.0)

Suggestions

DeusData commented on Sep 19, 2026

DeusData commented on Sep 26, 2026

njuptlzf commented on Sep 29, 2026

DeusData commented on Oct 19, 2026

DeusData commented on Oct 25, 2026

juantxorena commented on Oct 29, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

2. `check_index_coverage` already carries the per-file staleness signal

3. An mtime-only change latches `metadata_changed` permanently

4. `get_code_snippet` serves shifted source under this state (observed on 0.11.0)