Skip to content

LocalFileIdentifiableStore: add per-object write locking to prevent silent data loss under concurrent writes #554

Open

Description

@s-heppner

LocalFileIdentifiableStore has no concurrency control across processes. Under a multi-worker WSGI deployment, two workers handling concurrent requests that modify the same object will each call commit(), write to separate temp files, and then race on os.replace(). The last writer wins silently (no exception, no conflict signal, no log entry). The earlier write is permanently lost.

CouchDB avoids this via _rev/409: the caller is forced to detect and resolve the conflict. The local file path has no equivalent mechanism.

I suggest to take an advisory lock on a stable sibling lockfile (<hash>.lock) for the duration of the write in _write_atomic(). A sibling lockfile is preferred over locking the JSON file itself because os.replace() swaps the inode, so a lock held on the old file does not carry over to the replacement.

fcntl.flock is sufficient for single-host deployments (the local file backend implies a single host by design) and is safe under uWSGI multi-worker mode. It should be wrapped in a try/finally to guarantee the lock is released even on exception or KeyboardInterrupt.

This issue has been found by @zrgt during review of #553.

Activity

  1. added
    enhancementEnhancement of an existing feature
    sdkSomething to do with the `sdk` package
    on May 31, 2026
  2. zrgt commented on Oct 22, 2026

    @zrgt
    Contributor

    Side note on "CouchDB avoids this via _rev/409": that protection
    has a gap when the object is cached. A fetch refreshes _rev but
    returns the older cached object, so a later commit() overwrites
    concurrent changes without a 409, see #667. Worth keeping in mind
    if the local file store ever gets version checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementEnhancement of an existing featuresdkSomething to do with the `sdk` package

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions