Skip to content

fix(cli): recover stale rendezvous publication before declaring BUSY (#1760) - #1790

Closed
umi008 wants to merge 1 commit into
DeusData:mainfrom
umi008:fix/issue-1760-stale-rendezvous
Closed

umi008 wants to merge 1 commit into
DeusData:mainfrom
umi008:fix/issue-1760-stale-rendezvous

Conversation

@umi008

@umi008 umi008 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

After CBM processes are hard-killed (pkill/kill -9), the runtime dir keeps the published socket/anchor/identity files while every file lock is kernel-released. The CLI activation guard probed generation and treated the ECONNREFUSED socket as an active daemon — a deliberate fail-closed choice that protects saturated BSD listeners — so install/update failed forever with "active CBM sessions ... could not be stopped safely" even though nothing was running. Only daemon-start paths ran the stale cleanup, which the CLI could never reach: deadlock by residue.

cli_activation_production_reserve now recognizes the provably-stale combination — generation present, lifetime reservation absent, startup lock already held — and runs the existing cbm_daemon_ipc_stale_generation_cleanup repair, then re-probes generation before deciding. The cleanup re-checks both conditions and refuses on a genuinely held reservation, so a live daemon (including a saturated BSD listener) is never disturbed and the guard falls back to the original BUSY verdict. POSIX-only; Windows keeps its rendezvous-record semantics unchanged.

Verified with an end-to-end regression test: a child listens and exits without unlink (hard-kill residue), then the real activation guard installs successfully, removes the residue, and records the completed phase.

Checklist

  • Every commit is signed off (git commit -s) — required, CI rejects unsigned commits (DCO, see CONTRIBUTING.md)
  • Tests pass locally (make -f Makefile.cbm test)
  • Lint passes (make -f Makefile.cbm lint-ci)
  • New behavior is covered by a test (reproduce-first for bug fixes)

After a hard-killed daemon, the runtime dir keeps the socket/anchor/
identity publication while every lock is kernel-released. The CLI
activation guard probed generation and treated the ECONNREFUSED socket
as an active daemon (fail-closed, intentional for saturated listeners),
so install/update failed forever with 'active CBM sessions ... could not
be stopped safely' even though nothing was running — a deadlock by
residue, since only daemon-start paths ran the stale cleanup.

cli_activation_production_reserve now recognizes the provably-stale
combination — generation present with no lifetime reservation, under the
startup lock it already holds — and runs the existing
cbm_daemon_ipc_stale_generation_cleanup repair, then re-probes before
deciding. Stale cleanup re-checks both conditions and refuses on a
genuinely held reservation, so a live daemon (including a saturated BSD
listener) is never disturbed: the guard falls back to the original BUSY
verdict. POSIX-only; Windows semantics are unchanged.

Fixes DeusData#1760

Signed-off-by: Ulises Millan Guerrero <ulises.millanguerrero@gmail.com>
@umi008
umi008 requested a review from DeusData as a code owner August 21, 2026 20:21
@github-actions

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

@DeusData DeusData added bug Something isn't working stability/performance Server crashes, OOM, hangs, high CPU/memory priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. labels Aug 24, 2026
@DeusData

Copy link
Copy Markdown
Owner

Thank you for the detailed stale-residue analysis and for reusing the existing guarded cleanup path. I checked current main: cli_activation_production_reserve() probes the published generation under the startup lock and can return the existing BUSY result without attempting stale-generation cleanup. That grounds the activation deadlock described here.

There is one factual contribution-policy blocker before code review. This focused fix also adds 131 lines under five new openspec/ files, including openspec/config.yaml; current main has no openspec/ tree, and CONTRIBUTING.md requires prior approval for project configuration. Please remove the entire openspec/ portion from this PR and leave the focused C change plus regression test. That preserves the useful work while keeping one independently reviewable claim. I have labeled the underlying issue as a high-priority stability bug. Splitting enables review but does not pre-decide the merge outcome. Thank you for the focused production repair.

@DeusData

DeusData commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Not a review yet — this one is parked on our side for an unrelated reason, and I did not want you to find that out only after doing more work.

Please drop the openspec/ files here too. As on #1788 and #1789: openspec/ does not exist on main, no other contributor uses it, and adopting a spec framework is a project-level decision rather than something a bug fix should carry in. We have decided not to adopt it, so the ask is to keep src/ and tests/ and strip the rest.

That request stands independently of the review, so you can act on it now across all three without waiting on me.

I will come back to the change itself. Thank you for the batch — #1787 in particular is a good catch and I have approved it.

@DeusData

DeusData commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Thank you for this — you were the first to put a fix in front of us for #1760, and your diagnosis (generation present, lifetime reservation absent, startup lock held, so the cleanup that already exists should run before declaring BUSY) was exactly right.

We went with #1894 (now on main as 760c08f), which does the same recovery one layer lower, inside the daemon's IPC stale-generation cleanup, so it also covers the markerless .sock/.anc pair from #1757 and the worker hot-loop in #2015; it also carries the device/inode and two-link revalidation the unlink needs. Since it subsumes this change, I am closing this PR rather than merging both; the credit for getting there first is yours, and the fix ships in v0.10.9. If you see any residue of the wedge on that release, please open a fresh report and ping me. Thanks again.

@DeusData DeusData closed this Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority/high Needs near-term maintainer attention; high-impact bug, regression, safety issue, or release blocker. stability/performance Server crashes, OOM, hangs, high CPU/memory

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants