Skip to content

Commit 2ebc5e1

Browse files
docs: fix two incorrect Kernel claims in CONNECTION_PARAMETERS.md (#930)
* docs: fix two incorrect Kernel claims in CONNECTION_PARAMETERS.md Two rows disagreed with the code: - `_tls_client_cert_key_password`: documented as honored on Kernel (✅), but the kernel path raises `NotSupportedError` when it is set (`_kernel_tls_kwargs`, src/databricks/sql/backend/kernel/client.py) — the kernel has no surface for an encrypted client key. Marked ❌ and noted the workaround (unencrypted key, or Thrift). Also caveated the section callout, which had claimed all TLS options are honored on both backends. - `username` / `password`: the "raises `ValueError`" note is Thrift-only. That error is raised inside `get_python_sql_connector_auth_provider` (src/databricks/sql/auth/auth.py), which the kernel path never calls, so on Kernel the params are silently ignored rather than rejected. Doc-only change. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com> * docs: correct _disable_pandas Kernel support in CONNECTION_PARAMETERS.md _disable_pandas is a Python-side Arrow->row conversion toggle, not a wire option. KernelResultSet inherits the shared _convert_arrow_table, which reads connection.disable_pandas, so the flag is honored on the kernel path too (the kernel returns Arrow that flows through the same conversion). Mark the row Kernel-supported and drop it from the "ignored on Kernel" summary list. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com> * docs: clarify decimal/timestamp arrow-native flags (Thrift), verified live Both `_use_arrow_native_decimals` and `_use_arrow_native_timestamps` look like twin knobs, but behave oppositely when off — confirmed against a live warehouse: - `_use_arrow_native_decimals=False`: no value-level effect. The wire encoding becomes an Arrow string, but the connector unconditionally re-casts it back to `decimal128` (`convert_decimals_in_arrow_table`), so fetches always yield `Decimal`. - `_use_arrow_native_timestamps=False`: genuinely returns Python `str` (Arrow `string`) — there is no re-cast on the Arrow path — while `cursor.description` still reports `'timestamp'`. The `timestampAsArrow=False` flag also wins over the always-sent `timestampAsString=false` conf. Doc-only change. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com> * ai: apply changes for #930 (1 review thread) Addresses: - #3847633323 at CONNECTION_PARAMETERS.md:131 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> --------- Signed-off-by: eric-wang-1990 <e.wang@databricks.com> Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> Co-authored-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
1 parent 4ed862d commit 2ebc5e1

1 file changed

Lines changed: 9 additions & 8 deletions

File tree

CONNECTION_PARAMETERS.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ to change without notice.
8686
| `azure_client_id` / `azure_client_secret` / `azure_tenant_id` | `str` ||| `None` | Azure service-principal (Entra ID M2M), selected by `auth_type="azure-sp-m2m"`. On the kernel path the connector forwards these to the kernel, which owns Azure resolution (Entra v2.0 token endpoint + the Databricks-resource `.default` scope) (#919). **`azure_tenant_id` is optional on the kernel path too** — like Thrift, the kernel auto-discovers it from the workspace's `/aad/auth` redirect when omitted. |
8787
| `azure_workspace_resource_id` | `str` ||| `None` | For `azure-sp-m2m`. When set, the SP **management token** (`X-Databricks-Azure-SP-Management-Token`) + `X-Databricks-Azure-Workspace-Resource-Id` header are sent, to authorize an SP that has an Azure RBAC role but is not a workspace member. Omit it for a workspace-member SP (the data token authenticates alone; no management token is fetched). Works on both the kernel and Thrift paths. |
8888
| `_use_cert_as_auth` (+ `_tls_client_cert_file`) | `bool` ||| `False` | Authenticate with a TLS client certificate instead of a token. Thrift-only. |
89-
| `username` / `password` | `str` ||| `None` | **Removed.** Basic auth is no longer supported; passing either raises `ValueError`. |
89+
| `username` / `password` | `str` ||| `None` | **Removed.** Basic auth is no longer supported. On **Thrift**, passing either raises `ValueError`; on the **kernel** path it is silently ignored (the Thrift auth provider that raises is never built). |
9090

9191
## HTTP client, proxy, retries
9292

@@ -118,8 +118,9 @@ to change without notice.
118118

119119
> TLS options are assembled into a single `SSLOptions` object in `session.py`
120120
> and passed to **every** backend, so they are honored on both Thrift and
121-
> Kernel. Verification is **on by default**; you must pass `_tls_no_verify=True`
122-
> to disable it.
121+
> Kernel — with one exception: `_tls_client_cert_key_password` is **not**
122+
> supported on the kernel path (see below). Verification is **on by default**;
123+
> you must pass `_tls_no_verify=True` to disable it.
123124
124125
| Option | Type | Thrift | Kernel | Default Value | Note |
125126
| ------------------------------- | ----- | :----: | :----: | ------------- | -------------------------------------------------------------------------- |
@@ -128,7 +129,7 @@ to change without notice.
128129
| `_tls_trusted_ca_file` | `str` ||| `None` | Path to a CA bundle. Defaults to the system trust store. |
129130
| `_tls_client_cert_file` | `str` ||| `None` | Client certificate for mutual TLS. |
130131
| `_tls_client_cert_key_file` | `str` ||| `None` | Private key for the client certificate. |
131-
| `_tls_client_cert_key_password` | `str` || | `None` | Password for an encrypted client-key file. |
132+
| `_tls_client_cert_key_password` | `str` || | `None` | Password for an encrypted client-key file. On the kernel path this is rejected with `NotSupportedError` **only when mTLS is configured** (i.e. `_tls_client_cert_file` is also set); without mTLS it is ignored. The kernel has no surface for an encrypted client key today — pass an unencrypted PEM key, or use the Thrift backend. |
132133

133134
## Results & type rendering
134135

@@ -137,10 +138,10 @@ to change without notice.
137138
| `use_cloud_fetch` | `bool` ||| `True` | Download large result sets in parallel from cloud storage. The kernel manages result transport internally. |
138139
| `max_download_threads` | `int` ||| `10` | Worker threads for cloud-fetch downloads. Not forwarded to the kernel. |
139140
| `enable_query_result_lz4_compression` | `bool` ||| `True` | LZ4-compress result payloads. Not forwarded; the kernel handles compression internally. |
140-
| `_disable_pandas` | `bool` || | `False` | Skip the pandas-based Arrow deserialization path. Not forwarded to the kernel. |
141+
| `_disable_pandas` | `bool` || | `False` | Skip the pandas-based Arrow→row deserialization and materialize rows directly with PyArrow. This is a **Python-side** result-conversion toggle, not a wire option: the kernel returns results as Arrow (`RecordBatch`es) and the connector runs the *same* `_convert_arrow_table` for both backends, so the flag is honored on the kernel path too. Affects only row fetches (`fetchone`/`fetchmany`/`fetchall`); the `fetch*_arrow` methods return the Arrow table unchanged regardless of this flag. |
141142
| `_use_arrow_native_complex_types` | `bool` ||| `True` | Return `ARRAY`/`MAP`/`STRUCT` as native Arrow types instead of JSON strings. Forwarded to the kernel. |
142-
| `_use_arrow_native_decimals` | `bool` ||| `True` | Return `DECIMAL` as a native Arrow type instead of a string. Thrift-only. |
143-
| `_use_arrow_native_timestamps` | `bool` ||| `True` | Return `TIMESTAMP` as a native Arrow type instead of a string. Thrift-only. |
143+
| `_use_arrow_native_decimals` | `bool` ||| `True` | Thrift wire encoding for `DECIMAL`: `True` native Arrow `decimal128`, `False` → Arrow string. **No value-level effect**, though: the connector unconditionally re-casts the column back to `decimal128` (`convert_decimals_in_arrow_table`, `thrift_backend.py`), so both `fetchall()` and `fetchall_arrow()` yield `Decimal` / `decimal128(p,s)` either way (verified live). Not forwarded to the kernel, which always returns native Arrow decimals. |
144+
| `_use_arrow_native_timestamps` | `bool` ||| `True` | Thrift wire encoding for `TIMESTAMP`: `True` native Arrow timestamp (→ Python `datetime`), `False` → Arrow string (→ Python **`str`**). **Unlike decimals there is no re-cast**, so `False` genuinely surfaces strings — and `cursor.description` still reports the type code as `'timestamp'`, a mismatch to watch for (verified live). Note the connector always also sends the `spark.thriftserver.arrowBasedRowSet.timestampAsString=false` conf, but the `timestampAsArrow=False` flag wins. Not forwarded to the kernel, which always returns native Arrow timestamps. |
144145

145146
## Session defaults & transactions
146147

@@ -186,7 +187,7 @@ regardless of `use_kernel`.
186187
4. TLS-client-cert *authentication* (`_use_cert_as_auth`) — note the TLS
187188
*transport* options (`_tls_*`) themselves **are** honored on both backends.
188189
5. Result-transport tuning: `use_cloud_fetch`, `max_download_threads`,
189-
`enable_query_result_lz4_compression`, `_disable_pandas`.
190+
`enable_query_result_lz4_compression`.
190191
6. Arrow-native rendering for `_use_arrow_native_decimals` /
191192
`_use_arrow_native_timestamps` (complex types **are** forwarded).
192193
7. `staging_allowed_local_path` (Volume `PUT`/`GET`).

0 commit comments

Comments
 (0)