feat(spanner): add DataBoost and auto_partition_mode support to DBAPI driver - #18161
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds support for configuring and querying the DATA_BOOST_ENABLED setting on Spanner DB-API connections, including client-side statement parsing and execution for SET DATA_BOOST_ENABLED and SHOW VARIABLE DATA_BOOST_ENABLED. The reviewer feedback suggests improving the robustness of parsing by stripping quotes from the parameter value and initializing column_values directly as a list for consistency and safety.
olavloite
left a comment
There was a problem hiding this comment.
Regarding this part in the PR description:
enabling analytical and bulk read workloads (e.g. pandas.read_sql) to execute on serverless compute
Note that this feature in its current implementation can only be used with RUN PARTITIONED QUERY .... Other Spanner drivers, like JDBC, additionally also support an autoPartitionMode=true|false variable to turn all queries into partitioned queries, which makes it easier to use DataBoost for all queries.
…statements Add support for Cloud Spanner DataBoost in the Python DBAPI (PEP 249) driver (`google.cloud.spanner_dbapi`) for partitioned queries. Key updates: * Add `data_boost_enabled: bool = False` argument to `spanner_dbapi.connect()` and `Connection.__init__()`. * Add `@property def data_boost_enabled(self)` getter and setter on `Connection`. * Add client-side statement parsing and execution for: - `SET DATA_BOOST_ENABLED = TRUE|FALSE` - `SHOW VARIABLE DATA_BOOST_ENABLED` (returns column `DATA_BOOST_ENABLED` with `BOOL` type) * Forward `data_boost_enabled` in `Connection.partition_query()` and `Connection.run_partitioned_query()`. * Add unit and gRPC mock server test coverage across DBAPI connection, cursor, parser, statement executor, and mock server test suites.
dd78a1b to
cff1650
Compare
cff1650 to
63b2f31
Compare
70243b1 to
8b24d43
Compare
…ient-side statements Add support for `auto_partition_mode` in the Python DBAPI driver (`google.cloud.spanner_dbapi`). Key updates: * Add `auto_partition_mode: bool = False` parameter to `spanner_dbapi.connect()` and `Connection.__init__()`. * Add `@property def auto_partition_mode(self)` getter and setter on `Connection`. * Add client-side statement parsing and execution for: - `SET AUTO_PARTITION_MODE = TRUE|FALSE` - `SHOW VARIABLE AUTO_PARTITION_MODE` (returns column `AUTO_PARTITION_MODE` with `BOOL` type) * Automatically route queries in `Cursor._execute()` to `run_partitioned_query` when `auto_partition_mode=True`. * Add unit and gRPC mock server test coverage for automatic query partitioning.
8b24d43 to
9c53e3f
Compare
| :param database: The database to which the connection is linked. | ||
|
|
||
| :type read_only: bool | ||
| :param read_only: |
There was a problem hiding this comment.
nit: add documentation here for our new input arguments data_boost_enabled and auto_partition_mode
There was a problem hiding this comment.
Done! Added documentation for both data_boost_enabled and auto_partition_mode arguments to the Connection class docstring.
| _BOOL_MAP = {"true": True, "false": False} | ||
|
|
||
|
|
||
| def _parse_bool(raw_val: str, var_name: str) -> bool: |
There was a problem hiding this comment.
nit: This function does not work if the application added a semicolon to the end of the statement. That is:
SET AUTO_PARTITION_MODE = TRUE;Will produce an error. We could easily fix this by adjusting the regex parsing a bit:
def _parse_bool(raw_val: str, var_name: str) -> bool:
cleaned = raw_val.strip().rstrip(";").strip().strip("'\"").lower()
if cleaned not in _BOOL_MAP:
raise ProgrammingError(
f"Invalid value for {var_name}: '{raw_val}'. Expected TRUE or FALSE."
)
return _BOOL_MAP[cleaned]There was a problem hiding this comment.
Done! Updated _parse_bool to strip trailing semicolons, and updated the SHOW VARIABLE parser patterns to accept optional trailing semicolons as well.
…g semicolons in client-side statements Address code review comments: - Document `data_boost_enabled` and `auto_partition_mode` arguments in the `Connection` class docstring. - Support trailing semicolons and quotes in `_parse_bool` when executing `SET DATA_BOOST_ENABLED` and `SET AUTO_PARTITION_MODE`. - Allow optional trailing semicolons in `SHOW VARIABLE DATA_BOOST_ENABLED` and `SHOW VARIABLE AUTO_PARTITION_MODE` regex parser patterns. - Add test cases for statements ending with semicolons.
Description
Add support for Cloud Spanner DataBoost and
auto_partition_modein the Python DBAPI (PEP 249) driver (google.cloud.spanner_dbapi).Commit Structure
Commit 1 (
cff1650f719):feat(spanner): add DataBoost support to DBAPI driver and client-side statementsdata_boost_enabled: bool = Falsetoconnect()andConnection.SET DATA_BOOST_ENABLEDandSHOW VARIABLE DATA_BOOST_ENABLED(returningDATA_BOOST_ENABLEDcolumn withBOOLtype).data_boost_enabledinConnection.partition_query()andConnection.run_partitioned_query().partition_token).Commit 2 (
9c53e3f9a81):feat(spanner): add auto_partition_mode support to DBAPI driver and client-side statementsauto_partition_mode: bool = Falsetoconnect()andConnection.SET AUTO_PARTITION_MODEandSHOW VARIABLE AUTO_PARTITION_MODE(returningAUTO_PARTITION_MODEcolumn withBOOLtype).cursor.execute()/pandas.read_sql()) whenauto_partition_mode=True.auto_partition_mode=Trueanddata_boost_enabled=True._parse_boolhelper for client-side boolean parameter extraction.Testing