Stale rendezvous directory (/tmp/cbm-daemon-<uid>) permanently blocks install/update with "active CBM sessions ... could not be stopped safely" after hard-killed CBM processes #1760

Activity

1. hansinator85

   added

   bugSomething isn't workingSomething isn't working

   on Aug 20, 2026
2. github-actions

   added

   editor/integrationEditor compatibility and CLI integrationEditor compatibility and CLI integration

   stability/performanceServer crashes, OOM, hangs, high CPU/memoryServer crashes, OOM, hangs, high CPU/memory

   windowsWindows-specific issuesWindows-specific issues

   on Aug 20, 2026
3. LinRds commented on Aug 27, 2026

   

   LinRds

   on Aug 27, 2026

   More actions

   Hit this on 0.10.8, macOS 15 (Apple Silicon) running install.sh from ~/.local/bin. Same failure text, and the event log shows the refusal happens with zero active clients, which may be the useful part.

   What the installer printed

   Download and checksum both succeeded; the candidate verified as the same version already installed. Then:

   Stopping active CBM sessions and operations for install...
   error: active CBM sessions and operations could not be stopped safely; no activation was committed.
   error: something is still using CBM. If an editor or agent is running an MCP server, close it and retry;
          'codebase-memory-mcp daemon status' lists the holders when a cbm binary is still installed.
   

   Exit code was 0 despite no activation was committed, so a scripted/CI update would record this as success while silently keeping the old binary.

   activation-events.ndjson

   Today's failed run (fields trimmed):

   {"phase":"requested","action":"install","daemon_active_clients":0,"daemon_active_connections":0,...}
   {"phase":"failed","action":"install","daemon_active_clients":0,"daemon_active_connections":0,
    "detail":"cohort drain or coordination failed"}

   For contrast, the successful install of the same version on this machine 8 days earlier:

   {"phase":"daemon_stopped","action":"install","detail":"no active cohort"}
   {"phase":"completed","action":"install","detail":"activation mutation completed; ..."}

   So the drain fails while the installer itself observes daemon_active_clients: 0 and daemon_active_connections: 0.

   Contradictory user-facing signals

   The error message points at daemon status to list the holders, but at that moment:

   $ codebase-memory-mcp daemon status
   daemon: not running
   hint: `codebase-memory-mcp daemon start` keeps a daemon warm so CLI commands and hooks skip the per-command startup cost.
   

   No processes existed either (ps showed zero codebase-memory-mcp). Following the error's own advice therefore dead-ends — there is no way to discover that /tmp/cbm-daemon-<uid> is the blocker.

   Same root cause as the daemon-side failure

   On this machine the stale rendezvous directory was simultaneously blocking daemon start with CBM daemon could not start within 30000 ms (details in #1757 — markerless hard-linked .sock/.anc pair, no .identity/.pending). One rm -rf /tmp/cbm-daemon-501 cleared both failures at once: the daemon started on the next attempt, and install/update work again.

   Suggestions, in rough order of usefulness:

   1. Name the blocking path in the error text — /tmp/cbm-daemon-<uid> — rather than pointing at daemon status, which reports not running in exactly this state.
   2. Return non-zero when no activation was committed.
   3. Recover the rendezvous directory automatically when no process holds the socket and connect returns ECONNREFUSED (this is what PR fix(cli): recover stale rendezvous publication before declaring BUSY (#1760) #1790 appears to be reaching for).

   👀React with 👀1Reacted by Shay Erlichmen
4. mvanhorn

   added a commit that references this issue on Aug 31, 2026

   fix: recover markerless stale rendezvous sockets during activation

   ...

   This issue will close once commit 8e70590 is merged into the 'main' branch.

   8e70590
5. gitokato commented on Aug 31, 2026

   

   gitokato

   on Aug 31, 2026

   More actions

   Environment: codebase-memory-mcp 0.10.8 (build 2412e017…), macOS (Apple Silicon), install.sh,
   binary at ~/.local/bin, ui variant. Daemon was watching 4 projects with auto-index/watch on.

   Different symptom: the runtime guard was completely blocked, not just install/update.

   The same stale rendezvous state made every runtime entry point fail with the message:

   codebase-memory-mcp: CBM CLI could not start because a pre-coordination or unverified CBM generation is active; close all CBM sessions and commands, then retry
   

   This blocked all of: the MCP server connection (editor saw CONNECTION_CLOSED),
   codebase-memory-mcp cli <tool>, plain codebase-memory-mcp config list, and the daemon's own
   background index workers. Same as here, every process/lock check was clean (pgrep = 0 CBM
   processes; a fresh single-shot config list with no daemon alive still failed).

   The stuck artifacts were the cbm-version-cohort-*-v1.lock family in /tmp/cbm-daemon-<uid>/
   (…lifetime-v1.lock held 214 bytes of state), consistent with a stuck pre-coordination /
   unverified generation claim — which may be a different artifact than the .sock/.anc pair that
   PR #1894 targets. Worth checking whether #1894's activation-path recovery also clears a stuck
   version-cohort / unverified-generation claim that blocks the runtime guard, or only the install/update
   path.

   Concrete trigger (captured in cbm-daemon.log)

   The daemon log shows the exact pivot into the wedged state. Workers reap cleanly until one is
   killed mid-generation, and it never recovers after that:

   level=info  msg=index.supervisor.reap outcome=clean exit_code=0 signal=0        # healthy
   ...
   level=warn  msg=index.supervisor.worker_cancelled outcome=killed  log=.../.worker-log-XXXXXX   # <-- PIVOT (worker log 0 bytes)
   ...
   level=info  msg=index.supervisor.reap outcome=exit_nonzero exit_code=1 signal=0
   level=warn  msg=index.supervisor.worker_failed outcome=exit_nonzero exit_code=1  log=.../.worker-log-XXXXXX
   level=warn  msg=watcher.index.err project=<PROJECT-A>
   

   So: a worker killed mid-index left the generation reservation unreleased, and the pre-coordination
   guard then refused every subsequent start. Each failing worker log is identical:

   {"level":"control","event":"index.worker.start","version":"0.10.8","build":"<hash>","pid":"<PID>","repo_path":"<PROJECT-A>","args":"..."}
   level=info msg=version_cohort.claimed_unheld build=<hash>
   CBM index worker could not start: a pre-coordination or unverified CBM generation is active
   

   Amplification: infinite auto-index retry loop

   With auto-index/watch enabled, the file watcher re-fired the index on every watcher.changed for
   a dirty git tree, so the failing worker spawned continuously — ~230 identical failing worker logs
   accumulated and one process sat pinned in R state at high CPU for days. A stuck reservation plus
   auto-index turns a one-time crash into a hot loop.

   Happy to grab a CBM_DIAGNOSTICS=1 trajectory or the activation-events.ndjson if useful — this was
   on a live wedge that's since been cleared, but it's reproducible by killing a worker mid-generation.
6. DeusData

   added

   priority/highNeeds near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker.

   on Sep 5, 2026
7. Tomauskasz commented on Sep 7, 2026

   

   Tomauskasz

   on Sep 7, 2026

   More actions

   Hit this independently today (2026-09-07) on 0.10.8, same symptoms and same root cause.

   One more data point: with the rendezvous dir in this stuck state, codebase-memory-mcp daemon start also fails outright — it times out with CBM daemon could not start within 30000 ms after only printing version_cohort.claimed_unheld to stderr. It never got far enough to write anything to ~/.cache/codebase-memory-mcp/logs/cbm-daemon.log, even with zero live clients and zero daemon processes running. So this isn't limited to install/update/cli — it blocks daemon startup entirely too.

   Confirmed the same fix: rm -rf /tmp/cbm-daemon-$(id -u) resolved it immediately — daemon start and config get/set all worked right after, no reinstall needed.
8. DeusData

   closed this as completedin #1894on Sep 7, 2026
9. DeusData commented on Sep 7, 2026

   

   DeusData

   on Sep 7, 2026

   Owner

   More actions

   Fixed on main by 760c08f (PR #1894, thank you @mvanhorn, and thanks to everyone who confirmed the shape here): a hard-killed daemon or worker no longer leaves a markerless .sock/.anc pair plus an unheld cohort lock that wedges install, update and every CLI or MCP entry point behind the 30 s "active or starting" timeout; activation now runs the serialized stale-generation cleanup before classifying. Ships in v0.10.9. If you still hit it on that release, please say so here with daemon status and the contents of /tmp/cbm-daemon-.
10. daidaiJ

    added a commit that references this issue on Sep 29, 2026

    fix: recover markerless stale rendezvous sockets during activation

    ...

    This issue will close once commit 35913fb is merged into the 'main' branch.

    35913fb