Skip to content

Release v0.7.8: queue SQLite writes, upgrade libsql to 0.18 - #117

Merged
SylonZero merged 4 commits into
masterfrom
fix/sqlite-concurrent-writes
Oct 26, 2026
Merged

SylonZero merged 4 commits into
masterfrom
fix/sqlite-concurrent-writes

Conversation

@SylonZero

@SylonZero SylonZero commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Summary

Parallel CodePlans MCP writes against a SQLite database failed with SQLITE_BUSY. This was seen on the multi-tenant MindStaq instance, where each workspace has its own SQLite file. Tracked as the CodePlans work item "SQLite mode: concurrent MCP writes interleave on the shared libsql connection and silently lose rows". Releases as v0.7.8: app and CLI.

Root cause

The work item blamed statements interleaving on one shared connection. That isn't what @libsql/client 0.17.3 does:

  • client.transaction() handed its connection to the transaction and lazily opened a new connection for later statements.
  • That new connection had a busy timeout of 0, because PRAGMA busy_timeout=5000 only ever applied to the first one. So after the first transaction in a process, any write that overlapped an open transaction failed immediately.
  • The transaction's connection was never closed. In a measurement, 200 transactions left about 400 file handles open until garbage collection.

Change

  • Write queue (lib/db/sqlite-writes.ts, new): one WriteQueue per local SQLite file, so one per workspace database in multi-tenant deployments.
    • db.transaction() and every statement that isn't a SELECT run one at a time.
    • Reads aren't queued; WAL lets them run beside a writer.
    • A write through db inside db.transaction() now throws NestedWriteError instead of deadlocking. None of the 10 existing transaction sites do this.
  • @libsql/client 0.17.3 → 0.18.0.
    • It pools connections. A transaction borrows one and returns it on commit or rollback, and a connection left mid-transaction is rolled back or closed rather than kept.
    • Its new timeout option sets the busy timeout on every pooled connection, so the PRAGMA workaround is gone.
    • Released 2026-09-02, so it passes the deploy's minimum-release-age rule.
  • lib/db/index.ts: creates the queue for file: and :memory: URLs and sends db.transaction() through it from the existing proxy. Postgres and hosted libsql (Turso) are unchanged.
  • Version 0.7.8 in package.json and cli/package.json.

Why the queue is still needed with 0.18

libsql runs statements synchronously. With a pool, a write that overlaps a transaction runs on another connection and busy-waits on the lock, blocking the event loop the transaction needs in order to commit. With the queue disabled on 0.18, the parallel test took 41.7 s and still failed with SQLITE_BUSY.

Alternatives considered

Switching to better-sqlite3 was set aside. Drizzle's better-sqlite3 driver requires synchronous transaction callbacks, and all of ours await. It would also add a native build to deploys.

Testing

  • New tests/lib/db/concurrent-writes.test.ts covers:
    • the busy timeout surviving a transaction
    • 100 transactions not leaving connections open
    • a parallel batch of 8 work items, a task, a spec and 2 spec links all saving and showing up in search
    • a write arriving during an open transaction waiting its turn instead of failing
    • reads not being blocked by an open transaction
    • the nested-write error
    • ordering and failure handling in WriteQueue
  • These tests fail with SQLITE_BUSY against the old lib/db/index.ts.
  • Full suite: 637/637. tsc --noEmit is clean (apart from stale .next/ output).
  • Clean pnpm@10.33.0 install --frozen-lockfile: the only strict-build warning is sharp, the same as on master.

After merge

Tag v0.7.8 and publish the GitHub release; the Publish Release workflow also publishes the CLI.

Not addressed

The work item's "returned an id, then the row vanished" symptom never reproduced.

🤖 Generated with Claude Code

Sai Prakash and others added 4 commits October 25, 2026 20:10
libsql's local client hands its connection to each transaction and opens a
new one for later statements without a busy timeout, so any write that
overlapped an open transaction failed at once with SQLITE_BUSY. Parallel
MCP writes hit this.

Writes against a local SQLite file now run one at a time through a
per-handle queue (transactions and non-SELECT statements; reads stay
concurrent under WAL), and busy_timeout is re-applied to each replacement
connection. A write through db inside db.transaction() now throws
NestedWriteError instead of deadlocking.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
App and CLI both at 0.7.8. Ships the SQLite write queue: writes to each
local database file (per workspace, in multi-tenant deployments) run one
at a time, and busy_timeout is kept on the connections libsql opens after
a transaction, so parallel MCP writes no longer fail with SQLITE_BUSY.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ions

0.17.3 opened a new connection for every transaction and never closed it,
leaving about two file handles open per transaction until garbage
collection, and the new connections had no busy timeout. 0.18.0 pools
connections: a transaction borrows one and returns it on commit or
rollback (rolling back or closing one left mid-transaction), and the new
`timeout` client option sets the busy timeout on every pooled connection.

The busy timeout moves to that option and the PRAGMA re-apply workaround
goes. The write queue stays: without it, a write overlapping a transaction
busy-waits synchronously on another pooled connection, blocking the event
loop the transaction needs (the parallel test took 41.7s and still failed).

Adds a test that 100 transactions don't leave connections open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SylonZero SylonZero changed the title Queue SQLite writes and keep busy_timeout on new connections Release v0.7.8: queue SQLite writes, upgrade libsql to 0.18 Oct 26, 2026
@SylonZero
SylonZero merged commit 6ccaa52 into master Oct 26, 2026
1 check passed
@SylonZero
SylonZero deleted the fix/sqlite-concurrent-writes branch October 26, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant