Repository navigation
Stop rejecting unknown grant privileges in validate and PyDABs - #7315
Merged
janniklasrose merged 4 commits intoNov 3, 2026
Merged
janniklasrose merged 4 commits into
janniklasrose merged 4 commits into
Conversation
The platform accepts grant privileges that the SDK's catalog.Privilege enum does not yet list (e.g. CREATE_SECRET, see #6770). Both consumers of that enum rejected them: - `bundle validate` warned, and `--strict` errored, because the enum validation allowlist is reflected from the SDK's Values(). - PyDABs generated a strict `Privilege(Enum)` and `_transform` raised ValueError on unknown strings. Exclude catalog.Privilege from the validation generator so no `grants[*].privileges[*]` pattern is checked, and generate Privilege as an open enum in PyDABs: a `_missing_` hook turns unknown strings into cached pseudo-members and `PrivilegeParam` gains `| str`. The JSON schema keeps the enum, so completion is unchanged in both. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Move the schema grant in the enum acceptance test below the jobs so output.txt no longer differs from main by line numbers only. Drop the "platform adds privileges faster" wording from the changelog fragment and the issue link from the PyDABs codegen comment. Co-authored-by: Isaac <no-reply@databricks.com>
…nt-privilege-enum Resolved bundle/internal/validation/generated/enum_fields.go by taking main and re-running ./task generate-validation. Re-ran ./task pydabs-codegen; it produced no further changes. Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 2 tests: regression=2 unreleased=0 coverage=0 failing=0
TestAccept/bundle/validate/enum/DMS=: branch ✅ | main (c0e1859) ❌ | latest (v1.23.0) ❌main (c0e1859): latest (v1.23.0): TestAccept/bundle/validate/enum/DMS=true: branch ✅ | main (c0e1859) ❌ | latest (v1.23.0) ❌main (c0e1859): latest (v1.23.0): |
Collaborator
Integration test reportCommit: 270275e
Top 6 slowest tests (at least 2 minutes):
|
pietern
approved these changes
Nov 3, 2026
janniklasrose
deleted the
janniklasrose/stop-warning-and-erroring-on-grant-privilege-enum
branch
November 3, 2026 15:43
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Nov 4, 2026
## Release v1.24.0 ### Notable Changes * A variable used as the value of a secret is now sensitive: referencing it from any other field, such as a job parameter or a secret's `comment`, is an error because the value would be used in plain text. Mark other secret-holding variables with `sensitive: true` to get the same protection. ([#7288](#7288)) ### CLI * Fix `databricks auth login` and `databricks configure` overwriting the parent profile's host and token when saving a profile with a dotted name such as `team.prod` next to an existing `[team]`. ([#7266](#7266)) * `databricks auth login` and `databricks auth token` now reject the name of an existing profile after "Create a new profile" instead of logging in to that profile. ([#7196](#7196)) * `databricks auth login` and `databricks auth token` now reject a host that is not a URL with a hostname, such as `/`, `?o=123` or `https://`, instead of falling back to `DATABRICKS_HOST` or failing later. An empty host after "Enter a host URL manually" in the login profile picker no longer starts discovery login. ([#7195](#7195)) * Prompts that validate their input, such as the `databricks configure` host prompt, no longer open with a red cross before anything is typed. ([#7314](#7314)) * `databricks ssh connect` now documents `--environment-version` for choosing the serverless environment version, starts a separate default-named server for each version, and rejects the flag together with `--cluster`. Default connection names also include the environment version in their hash, so the default name when `--environment-version` is not specified changes with this release. Configuration associated with the old connection name (secret scope, SSH keys and known hosts in your local `.databricks` folder) will no longer be used and can be safely removed. ([#7310](#7310)) ### AI Runtime * Show workspace-supported accelerator types in `databricks air run -h config.compute.accelerator_type`. ([#7275](#7275)) * `air list` now reports `No active runs found.` for an empty active-only result. ([#7276](#7276)) * Stop printing a retry notice on every `air run --watch`; retries are still reported when they happen. ([#7270](#7270)) ### Bundles * Fix `bundle plan` reporting an update to an app's `config.env` values on every run, which also redeployed the app on every `bundle deploy`. The Apps API returns environment variable names only, so `value` and `value_from` are no longer compared against the remote deployment. ([#7326](#7326)) * Add the `data_quality_monitors` resource for data quality monitors on Unity Catalog schemas and tables (direct engine only). ([#7094](#7094)) * `bundle deploy` now asks for confirmation before recreating or deleting a Feature View, and `bundle deploy` and `bundle destroy` warn that FeatureSpecs using a Feature View are not checked. ([#7164](#7164)) * Accept grant `privileges` values that the SDK enum does not list yet: `bundle validate` no longer warns (or errors under `--strict`) on them, and the PyDABs `Privilege` enum accepts unknown strings instead of raising. The JSON schema and PyDABs still advertise the known values for completion. ([#7315](#7315)) * Add the `materialization_sets` resource, which materializes Feature Store Feature Views to offline and online stores. ([#7164](#7164)) * Detect notebooks with the same rule as the workspace: a `.py`, `.sql`, `.scala` or `.r` file is a notebook when its first line contains `Databricks notebook source`, for example after a byte order mark. Previously such a file was tracked as a plain file, so removing it locally left the remote notebook behind. ([#7290](#7290)) * Add `sensitive: true` for bundle variables. A variable used as the value of a secret is sensitive by default. `bundle validate -o json` redacts only sensitive variables instead of every string variable, and a sensitive variable can only be referenced from a sensitive resource field (such as a secret's `value`) or from another sensitive variable. ([#7288](#7288)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#7305](#7305)) * Bump dependencies with known vulnerabilities. ([#7327](#7327)) * Update the Go SDK to [v0.191.1](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.191.1). ([#7277](#7277)) * Update the Go SDK to [v0.193.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.193.0). ([#7303](#7303))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
The platform accepts grant privileges that the SDK's
catalog.Privilegeenum does not yet list (e.g.CREATE_SECRET, see #6770). The enum values flow from the OpenAPI spec into the SDK and from there into two CLI consumers, both of which rejected platform-valid values:bundle validatewarned, and--stricterrored, because the enum validation allowlist (bundle/internal/validation/generated/enum_fields.go) is reflected from the SDK'sValues().class Privilege(Enum)and_transformraisedValueErroron unknown strings.This PR relaxes both while keeping the enum (and completion) in the JSON schema and PyDABs:
bundle/internal/validation/enum.go: excludecatalog.Privilegefrom the generator, so noresources.*.grants[*].privileges[*]pattern is checked. Regenerated with./task generate-validation; the diff is exactly the 10 privilege entries.python/codegen/codegen/generated_enum.py: newOPEN_ENUMS = {"catalog.Privilege"}. Open enums are generated with a_missing_hook that turns unknown strings into pseudo-members cached in_value2member_map_(the stdlibFlagpattern, so identity/equality andfrom_dict→as_dictround-trips hold), andPrivilegeParamgains| strso the dict form type-checks for unknown values.main.pyfails if anOPEN_ENUMSkey is not a known schema. Regenerated with./task pydabs-codegen; the diff is exactly the 10*/_models/privilege.pymodules.Deploy was never affected:
catalog.Privilegeistype Privilege stringin Go, so unknown values already passed through both engines.Why not wait for an SDK bump?
The allowlist is only as current as the SDK, which lags the platform by design. Privileges are an area where the platform adds values frequently, so an SDK bump fixes today's gap and reopens tomorrow's. Dropping the client-side check for this one field removes the recurring breakage; the general enum validation stays in place for everything else.
Tests
acceptance/bundle/validate/enum: added a schema grant withINVALID_PRIVILEGE; the regenerated output shows no warning for it while the other enum warnings still fire.python/codegen/codegen_tests/test_generated_enum.py: open vs closed enum code generation.python/databricks_tests/core/test_open_enum.py: known value, unknown value accepted, cached identity, non-string rejected, dict round-trip.pyrightclean, Sphinx docs build with-W.This pull request and its description were written by Isaac.