refactor(index): close IndexStore connections deterministically - #69
Open
Hotragn wants to merge 1 commit into
Open
refactor(index): close IndexStore connections deterministically#69Hotragn wants to merge 1 commit into
Hotragn wants to merge 1 commit into
Conversation
`IndexStore` is the only store that does not close its SQLite connections. All
eight of its methods use `with connect_index(path) as connection:`, and
`sqlite3.Connection.__exit__` scopes a *transaction* — it commits or rolls back
and returns, leaving the connection open for the interpreter to reclaim.
Every other store routes through `codealmanac.database.open_local_database`,
which closes in a `finally` and carries a comment explaining why it exists:
sqlite3.Connection's own context manager scopes a transaction and never
closes; long agent runs record hundreds of events and leaked descriptors
until sqlite3.connect failed with "unable to open database file".
`IndexStore` bypasses that helper, so it never got the fix. This matters more
here than elsewhere because every query command reindexes implicitly, making
this the hot path rather than a rare one.
Adds `open_index`, which nests `with connection:` inside a `try/finally` that
closes. Nesting rather than replacing is deliberate: none of the eight callers
commits explicitly, so they depend entirely on `__exit__` for durability.
Dropping it for a plain close would silently stop persisting the index.
Commit-on-success and rollback-on-error are therefore unchanged; the only
difference is that the handle is released when the block exits.
`connect_index` is kept as-is, both because `open_index` builds on it and
because `tests/test_architecture.py` pins its presence in this module.
The regression test mirrors the existing `test_store_connections_close_after_
every_use` for `RepositoryStore`, and fails on the previous code.
Scope note: I could not reproduce descriptor exhaustion from this path, and
psutil's open-file reporting on Windows was not reliable enough to measure it, so
this is deterministic resource hygiene and consistency with the pattern the repo
already chose everywhere else — not a fix for an observed crash.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
IndexStoreis the only store in the codebase that does not close its SQLite connections. All eight of its methods rely onsqlite3.Connection.__exit__, which scopes a transaction and returns without closing.Adds
open_index, which nests the existing transaction context inside atry/finallythat closes. Behaviour is otherwise identical.Why
Every other store routes through
codealmanac.database.open_local_database, which closes in afinallyand carries a comment explaining exactly why it exists:IndexStorebypasses that helper, so it never received the fix:Confirmed by grep that
repositories/store.py,runs/store.py,runs/events.py,runs/worker_locks.pyandtelemetry/store.pyall useopen_local_database, andservices/index/store.pyis the sole holdout.It matters more here than elsewhere because every query command reindexes implicitly, so this is the hot path rather than a rare one.
Scope note — what I am not claiming
I could not reproduce descriptor exhaustion from this path, and psutil's open-file reporting on Windows was not reliable enough to measure it (200 deliberately-retained connections reported as 3 handles, unchanged after
gc.collect()). So please read this as deterministic resource hygiene and consistency with the pattern the repo already chose everywhere else, not as a fix for an observed crash. I would rather say that up front than have you find it in review.What is verifiable is the contract: after this change the connection is closed when the block exits, and the test proves it.
Verification
The new test fails on the previous code:
Docs and wiki
Notes for reviewers
with connection:rather than replacing it is the load-bearing detail. None of the eight callers commits explicitly —IndexStorehas zerocommit()calls andreplace_documentsdoes not commit either. So they depend entirely on__exit__for durability, and swapping it for a plain closing context manager would silently stop persisting the index. Nesting preserves commit-on-success and rollback-on-error exactly; the only change is that the handle is released.connect_indexis deliberately kept, both becauseopen_indexbuilds on it and becausetests/test_architecture.py:187pins"def connect_index" in schema. I did not want to quietly weaken an architecture test to make a refactor fit.store.pyline count is unchanged at 131, so the<= 160budget intest_architecture.pystill holds with room to spare.RepositoryStore(test_store_connections_close_after_every_use) rather than inventing a new style — same tracking-connect monkeypatch, same closed-connection assertion.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.