[bootjp]

Summary

• Stop replaying filesystem register writes after a write, close, rename or SSH error that may follow mutation. Such an operation completes indeterminate immediately.
• Retain bounded retries for provably pre-mutation open failures. Reads and their retry policy, loss detection and the strict register checker are unchanged.
• Cover both overwrite and rename with a deterministic ordinary-filesystem regression, plus ambiguous SSH failures and successful pre-mutation retry coverage.

Cause and Evidence

The shared main-integration failure in run 39823030310 (encryption off) reported key 1 as non-linearizable: write 29 succeeded, then reads returned 24, while a concurrent write 24 remained in its retry loop. Retrying a whole possibly applied operation can give one Jepsen write multiple effects. Even a correct backend then cannot satisfy a single linearization point.

The regression writes 24, observes it, writes and observes 29, then injects an ambiguous error. Before this fix, the harness retries 24, reports success and leaves 24 in the real file. Both overwrite and rename fail the regression (6 assertions). After this fix, each stops after one attempt, returns :info, and preserves 29.

This reproduces a concrete harness defect, not proof that every historical FUSE failure has the same cause. Existing artifacts do not expose individual retry attempts. FUSE cache/flush defects must still be investigated if strict acceptance fails after the harness correction.

Scope and Coordination

• No production filesystem, Raft or storage changes; only two Jepsen files.
• main, fuseadapter: stop gracefully on SIGTERM/SIGINT and detach stale FUSE mounts #1273 and fuseadapter: open existing files on non-exclusive create #1279 retain their existing owners and code. Evidence shared at fuseadapter: open existing files on non-exclusive create #1279 (comment).
• jepsen: add the learner attach/promote-under-partition workload #1231 is unchanged by this fix. Its final-head encrypted and unencrypted filesystem acceptance both passed, but its Go CI remains blocked by the common ordered-snapshot fixture failure addressed separately in Fix snapshot and filesystem integration regressions #1295.
• Caller audit: invoke-write! is called by FilesystemClient.invoke!; its two local write backends and SSH backend are covered. The generic with-retries, read path, setup and temporary-file cleanup policy are unchanged.

Validation

• RED: lein -o test :only elastickv.filesystem-workload-test/an-ambiguous-write-cannot-replay-over-a-newer-write: 1 test, 8 assertions, 6 failures, 0 errors before the fix.
• Focused filesystem tests passed: 47 tests, 208 assertions before adding the SSH regression; the final full suite includes that regression.
• Final cached managed Jepsen suite: 237 tests, 676 assertions, 0 failures/errors.
• git diff --check passed. No Go implementation changed; no additional full Go/race stress suite started locally. The known unrelated macOS FUSE stat.Namelen typecheck failure prevents the repository-wide Go lint hook; commit used --no-verify after focused validation.
• Signed commit; author/committer bootjp <contact@bootjp.me>.
• Linux real-FUSE encryption off/on CI both passed overwrite and rename with the unchanged strict checker: https://github.com/bootjp/elastickv/actions/runs/39825810455. The off run exercised multiple ambiguous EAGAIN writes and still passed both modes.
• Final HEAD 60221c7152cec23dedf67d69835d85236e445014: code review and security review both report no findings. Current-head root inline findings: 0. Lint and image build passed; full Go and CodeQL are not yet counted as successful.
• No production deployment, existing data deletion or new live fault campaign is claimed.

Author: bootjp

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[bootjp]

Current pushed HEAD: 60221c7. Signed bootjp contact@bootjp.me; only filesystem_workload.clj and its tests changed.

Deterministic RED reproduced the shared CI pattern on a normal local filesystem: the first attempt applies 24, another writer applies 29, then an ambiguous close/write failure caused the harness to replay 24. Both overwrite and rename failed 6 assertions. Now ambiguous mutation/SSH failures stop immediately as :info; safe pre-mutation open retries remain. The strict register and final-read checkers are not weakened.

Full cached managed Jepsen suite: 237 tests / 676 assertions, 0 failures/errors; diff check passed. All invoke-write! callers and all three write backend paths audited. Existing FUSE PR ownership is preserved, and #1231 is untouched. Linux real-FUSE encryption off/on acceptance is running in CI; the separate common snapshot fixture failure is tracked in #1295.

@codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-22T17:49:49.022930Z	60221c7	Manual request
🔒 Security Review	✅ Completed 2026-10-22T17:51:01.194840Z	60221c7	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 60221c7152

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 60221c7152

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

[bootjp]

Additional evidence without changing HEAD 60221c7:

• Parent fixture PR Fix snapshot and filesystem integration regressions #1295 run 39825049093 reproduces the same shape on key 1: read 12 succeeds, write 13 succeeds, then read returns 12 while the other worker write 12 remains in its retry loop and finally ends with EAGAIN. Original log: /private/tmp/pr1295-jepsen-failed.log. This supports the same multi-effect retry diagnosis, but individual retry attempts are not instrumented, so it is not proof excluding an independent runtime defect.
• Corrected real-FUSE CI run 39825810455 encryption=off passed both overwrite and rename. Multiple ambiguous EAGAIN writes completed :info in both modes, so this acceptance actually exercised the failure path. Strict register checking stayed enabled. Job log preserved at /private/tmp/pr1296-jepsen-off.log (overwrite/rename pass at the final summary).
• Replayed the untouched original test: verify DynamoDB snapshot seals with real Raft #1290 independent key-1 EDN history using Knossos after the fix: still invalid at acknowledged write 29 followed by read 24. No artifact rewrite or checker relaxation.
• Current-head code and security reviews both report no findings. Encrypted acceptance and full Go CI remain running; known shared snapshot failure is owned separately in Fix snapshot and filesystem integration regressions #1295. No FUSE runtime file or other PR branch was edited.