Skip to content

refactor(index): close IndexStore connections deterministically - #69

Open
Hotragn wants to merge 1 commit into
AlmanacCode:mainfrom
Hotragn:fix/index-store-connection-leak
Open

refactor(index): close IndexStore connections deterministically#69
Hotragn wants to merge 1 commit into
AlmanacCode:mainfrom
Hotragn:fix/index-store-connection-leak

Conversation

@Hotragn

@Hotragn Hotragn commented Aug 19, 2026

Copy link
Copy Markdown

Summary

IndexStore is the only store in the codebase that does not close its SQLite connections. All eight of its methods rely on sqlite3.Connection.__exit__, which scopes a transaction and returns without closing.

Adds open_index, which nests the existing transaction context inside a try/finally that closes. Behaviour is otherwise identical.

Why

Every other store routes through codealmanac.database.open_local_database, which closes in a finally and carries a comment explaining exactly why it exists:

sqlite3.Connection's own context manager scopes a transaction and never closes; long agent runs record hundreds of events and leaked descriptors until sqlite3.connect failed with "unable to open database file".

IndexStore bypasses that helper, so it never received the fix:

with connect_index(db_path) as connection:   # x8 — commits, does not close

Confirmed by grep that repositories/store.py, runs/store.py, runs/events.py, runs/worker_locks.py and telemetry/store.py all use open_local_database, and services/index/store.py is the sole holdout.

It matters more here than elsewhere because every query command reindexes implicitly, so this is the hot path rather than a rare one.

Scope note — what I am not claiming

I could not reproduce descriptor exhaustion from this path, and psutil's open-file reporting on Windows was not reliable enough to measure it (200 deliberately-retained connections reported as 3 handles, unchanged after gc.collect()). So please read this as deterministic resource hygiene and consistency with the pattern the repo already chose everywhere else, not as a fix for an observed crash. I would rather say that up front than have you find it in review.

What is verifiable is the contract: after this change the connection is closed when the block exits, and the test proves it.

Verification

# Windows 11, Python 3.13 (uv-managed), uv 0.12.0
uv run pytest tests/test_database.py tests/test_read_model.py \
              tests/test_architecture.py -q
# 72 passed

uv run pytest -q          # 13 failed, 551 passed, 1 skipped
# baseline on main:         13 failed, 550 passed, 1 skipped   (+1 = the new test)

uv run ruff check .       # All checks passed!
git diff --check          # clean

The new test fails on the previous code:

FAILED tests/test_database.py::test_index_store_connections_close_after_every_use

Docs and wiki

  • Not applicable — internal resource handling; no user-facing behaviour or output change.

Notes for reviewers

  • Nesting with connection: rather than replacing it is the load-bearing detail. None of the eight callers commits explicitly — IndexStore has zero commit() calls and replace_documents does not commit either. So they depend entirely on __exit__ for durability, and swapping it for a plain closing context manager would silently stop persisting the index. Nesting preserves commit-on-success and rollback-on-error exactly; the only change is that the handle is released.
  • connect_index is deliberately kept, both because open_index builds on it and because tests/test_architecture.py:187 pins "def connect_index" in schema. I did not want to quietly weaken an architecture test to make a refactor fit.
  • store.py line count is unchanged at 131, so the <= 160 budget in test_architecture.py still holds with room to spare.
  • The test mirrors the existing one for RepositoryStore (test_store_connections_close_after_every_use) rather than inventing a new style — same tracking-connect monkeypatch, same closed-connection assertion.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

`IndexStore` is the only store that does not close its SQLite connections. All
eight of its methods use `with connect_index(path) as connection:`, and
`sqlite3.Connection.__exit__` scopes a *transaction* — it commits or rolls back
and returns, leaving the connection open for the interpreter to reclaim.

Every other store routes through `codealmanac.database.open_local_database`,
which closes in a `finally` and carries a comment explaining why it exists:

    sqlite3.Connection's own context manager scopes a transaction and never
    closes; long agent runs record hundreds of events and leaked descriptors
    until sqlite3.connect failed with "unable to open database file".

`IndexStore` bypasses that helper, so it never got the fix. This matters more
here than elsewhere because every query command reindexes implicitly, making
this the hot path rather than a rare one.

Adds `open_index`, which nests `with connection:` inside a `try/finally` that
closes. Nesting rather than replacing is deliberate: none of the eight callers
commits explicitly, so they depend entirely on `__exit__` for durability.
Dropping it for a plain close would silently stop persisting the index.
Commit-on-success and rollback-on-error are therefore unchanged; the only
difference is that the handle is released when the block exits.

`connect_index` is kept as-is, both because `open_index` builds on it and
because `tests/test_architecture.py` pins its presence in this module.

The regression test mirrors the existing `test_store_connections_close_after_
every_use` for `RepositoryStore`, and fails on the previous code.

Scope note: I could not reproduce descriptor exhaustion from this path, and
psutil's open-file reporting on Windows was not reliable enough to measure it, so
this is deterministic resource hygiene and consistency with the pattern the repo
already chose everywhere else — not a fix for an observed crash.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant