Skip to content

fix(spanner): reset transaction flag when session checkout fails - #19014

Open
founderinaction wants to merge 1 commit into
googleapis:mainfrom
founderinaction:fix/spanner-transaction-flag-on-session-error
Open

founderinaction wants to merge 1 commit into
googleapis:mainfrom
founderinaction:fix/spanner-transaction-flag-on-session-error

Conversation

@founderinaction

Copy link
Copy Markdown
  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #18375 🦕

Database.run_in_transaction sets _local.transaction_running = True and then calls get_session() outside the try/finally that resets the flag. If checkout raises, for example ServiceUnavailable while Spanner is down, the flag is never cleared. Every later call on that thread then fails with RuntimeError: Spanner does not support nested transactions, even after Spanner recovers.

The change moves session checkout inside the try, and only calls put_session when a session was actually obtained, so None is never returned to the sessions manager. The async implementation in _async/database.py had the same shape. I fixed it there and made the identical change to the generated sync module, and checked that the CrossSync conversion of the async method matches the sync one.

Tests, sync and async:

  • test_run_in_transaction_session_error: the first checkout raises ServiceUnavailable, and a second call succeeds. The failed attempt doesn't call put_session, the flag is reset, and the session from the second call is returned.
  • test_run_in_transaction_callback_error: when the callback raises, the flag is reset and the session is still returned. This pins the existing behaviour.

Without the fix, both session_error tests fail with the nested-transaction RuntimeError. With it, tests/unit passes, and ruff and flake8 are clean.

@mukeshr06 reproduced this in September and mentioned preparing a fix. I haven't seen a PR since, so here is one. Happy to close this in favour of theirs if it's close.

`Database.run_in_transaction` set `_local.transaction_running` before
checking out a session, outside the `try`/`finally` that resets it. If
`get_session()` raised (for example `ServiceUnavailable` while Spanner
is down), the flag stayed set and every later call on that thread failed
with "Spanner does not support nested transactions".

Check out the session inside the `try`, and only return it to the
sessions manager when one was obtained. The async implementation had the
same shape and is fixed the same way; the sync module is regenerated
output of it.

Fixes googleapis#18375
@founderinaction
founderinaction requested a review from a team as a code owner November 6, 2026 18:54

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the run_in_transaction method in both the synchronous and asynchronous database implementations to initialize the session variable to None and safely retrieve it within the try block. In the finally block, the session is only returned to the session manager if it was successfully created (i.e., is not None). This prevents secondary errors when session retrieval fails. Corresponding unit tests have been added to verify error handling during session retrieval and callback execution. There are no review comments, so I have no feedback to provide.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A client called run_in_transaction be stuck if Spanner is down

1 participant