Skip to content

feat(spanner): add DataBoost and auto_partition_mode support to DBAPI driver - #18161

Merged
sakthivelmanii merged 3 commits into
mainfrom
feat-spanner-dbapi-databoost
Aug 21, 2026
Merged

feat(spanner): add DataBoost and auto_partition_mode support to DBAPI driver#18161
sakthivelmanii merged 3 commits into
mainfrom
feat-spanner-dbapi-databoost

Conversation

@sakthivelmanii

@sakthivelmanii sakthivelmanii commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Description

Add support for Cloud Spanner DataBoost and auto_partition_mode in the Python DBAPI (PEP 249) driver (google.cloud.spanner_dbapi).

Commit Structure

  1. Commit 1 (cff1650f719): feat(spanner): add DataBoost support to DBAPI driver and client-side statements

    • Adds data_boost_enabled: bool = False to connect() and Connection.
    • Adds SET DATA_BOOST_ENABLED and SHOW VARIABLE DATA_BOOST_ENABLED (returning DATA_BOOST_ENABLED column with BOOL type).
    • Forwards data_boost_enabled in Connection.partition_query() and Connection.run_partitioned_query().
    • Confines DataBoost execution to partitioned queries (per Spanner RPC requirement of partition_token).
  2. Commit 2 (9c53e3f9a81): feat(spanner): add auto_partition_mode support to DBAPI driver and client-side statements

    • Adds auto_partition_mode: bool = False to connect() and Connection.
    • Adds SET AUTO_PARTITION_MODE and SHOW VARIABLE AUTO_PARTITION_MODE (returning AUTO_PARTITION_MODE column with BOOL type).
    • Automatically partitions and parallelizes standard queries (cursor.execute() / pandas.read_sql()) when auto_partition_mode=True.
    • Enables seamless DataBoost execution for standard queries when both auto_partition_mode=True and data_boost_enabled=True.
    • Introduces reusable _parse_bool helper for client-side boolean parameter extraction.

Testing

  • All 234 unit and gRPC mock server tests passing.
  • Verified flake8 and ruff formatting/import sorting compliance.

@sakthivelmanii
sakthivelmanii requested a review from a team as a code owner August 19, 2026 13:26

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for configuring and querying the DATA_BOOST_ENABLED setting on Spanner DB-API connections, including client-side statement parsing and execution for SET DATA_BOOST_ENABLED and SHOW VARIABLE DATA_BOOST_ENABLED. The reviewer feedback suggests improving the robustness of parsing by stripping quotes from the parameter value and initializing column_values directly as a list for consistency and safety.

@olavloite olavloite left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding this part in the PR description:

enabling analytical and bulk read workloads (e.g. pandas.read_sql) to execute on serverless compute

Note that this feature in its current implementation can only be used with RUN PARTITIONED QUERY .... Other Spanner drivers, like JDBC, additionally also support an autoPartitionMode=true|false variable to turn all queries into partitioned queries, which makes it easier to use DataBoost for all queries.

Comment thread packages/google-cloud-spanner/google/cloud/spanner_dbapi/cursor.py Outdated
Comment thread packages/google-cloud-spanner/tests/mockserver_tests/test_dbapi_databoost.py Outdated
Comment thread packages/google-cloud-spanner/tests/mockserver_tests/test_dbapi_databoost.py Outdated
…statements

Add support for Cloud Spanner DataBoost in the Python DBAPI (PEP 249) driver (`google.cloud.spanner_dbapi`) for partitioned queries.

Key updates:
* Add `data_boost_enabled: bool = False` argument to `spanner_dbapi.connect()` and `Connection.__init__()`.
* Add `@property def data_boost_enabled(self)` getter and setter on `Connection`.
* Add client-side statement parsing and execution for:
  - `SET DATA_BOOST_ENABLED = TRUE|FALSE`
  - `SHOW VARIABLE DATA_BOOST_ENABLED` (returns column `DATA_BOOST_ENABLED` with `BOOL` type)
* Forward `data_boost_enabled` in `Connection.partition_query()` and `Connection.run_partitioned_query()`.
* Add unit and gRPC mock server test coverage across DBAPI connection, cursor, parser, statement executor, and mock server test suites.
@sakthivelmanii
sakthivelmanii force-pushed the feat-spanner-dbapi-databoost branch from dd78a1b to cff1650 Compare August 19, 2026 15:05
@parthea parthea assigned olavloite and unassigned sakthivelmanii Aug 19, 2026
@sakthivelmanii
sakthivelmanii force-pushed the feat-spanner-dbapi-databoost branch from cff1650 to 63b2f31 Compare August 19, 2026 15:41
@sakthivelmanii sakthivelmanii changed the title feat(spanner): add DataBoost support to DBAPI driver and client-side statements feat(spanner): add DataBoost and auto_partition_mode support to DBAPI driver Aug 19, 2026
@sakthivelmanii
sakthivelmanii force-pushed the feat-spanner-dbapi-databoost branch 2 times, most recently from 70243b1 to 8b24d43 Compare August 19, 2026 15:52
…ient-side statements

Add support for `auto_partition_mode` in the Python DBAPI driver (`google.cloud.spanner_dbapi`).

Key updates:
* Add `auto_partition_mode: bool = False` parameter to `spanner_dbapi.connect()` and `Connection.__init__()`.
* Add `@property def auto_partition_mode(self)` getter and setter on `Connection`.
* Add client-side statement parsing and execution for:
  - `SET AUTO_PARTITION_MODE = TRUE|FALSE`
  - `SHOW VARIABLE AUTO_PARTITION_MODE` (returns column `AUTO_PARTITION_MODE` with `BOOL` type)
* Automatically route queries in `Cursor._execute()` to `run_partitioned_query` when `auto_partition_mode=True`.
* Add unit and gRPC mock server test coverage for automatic query partitioning.
@sakthivelmanii
sakthivelmanii force-pushed the feat-spanner-dbapi-databoost branch from 8b24d43 to 9c53e3f Compare August 19, 2026 15:57
:param database: The database to which the connection is linked.

:type read_only: bool
:param read_only:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: add documentation here for our new input arguments data_boost_enabled and auto_partition_mode

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Added documentation for both data_boost_enabled and auto_partition_mode arguments to the Connection class docstring.

_BOOL_MAP = {"true": True, "false": False}


def _parse_bool(raw_val: str, var_name: str) -> bool:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This function does not work if the application added a semicolon to the end of the statement. That is:

SET AUTO_PARTITION_MODE = TRUE;

Will produce an error. We could easily fix this by adjusting the regex parsing a bit:

def _parse_bool(raw_val: str, var_name: str) -> bool:
    cleaned = raw_val.strip().rstrip(";").strip().strip("'\"").lower()
    if cleaned not in _BOOL_MAP:
        raise ProgrammingError(
            f"Invalid value for {var_name}: '{raw_val}'. Expected TRUE or FALSE."
        )
    return _BOOL_MAP[cleaned]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Updated _parse_bool to strip trailing semicolons, and updated the SHOW VARIABLE parser patterns to accept optional trailing semicolons as well.

…g semicolons in client-side statements

Address code review comments:
- Document `data_boost_enabled` and `auto_partition_mode` arguments in the `Connection` class docstring.
- Support trailing semicolons and quotes in `_parse_bool` when executing `SET DATA_BOOST_ENABLED` and `SET AUTO_PARTITION_MODE`.
- Allow optional trailing semicolons in `SHOW VARIABLE DATA_BOOST_ENABLED` and `SHOW VARIABLE AUTO_PARTITION_MODE` regex parser patterns.
- Add test cases for statements ending with semicolons.
@sakthivelmanii
sakthivelmanii merged commit 5bc3899 into main Aug 21, 2026
44 checks passed
@sakthivelmanii
sakthivelmanii deleted the feat-spanner-dbapi-databoost branch August 21, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants