Skip to content

Add a new boolean field 'sensitive' for variables - #7288

Merged
andrewnester merged 5 commits into
mainfrom
fix/sensitive-redaction
Nov 4, 2026
Merged

andrewnester merged 5 commits into
mainfrom
fix/sensitive-redaction

Conversation

@andrewnester

Copy link
Copy Markdown
Contributor

Changes

Add a new boolean field 'sensitive' for variables

Why

This allows to mark certain variables senstive and as a result all the other variable values won't be redacted in validate json output

Fixes #7099

Tests

Added accptance tests

@andrewnester
andrewnester requested review from a team as code owners November 2, 2026 13:18
@github-actions github-actions Bot added the DABs DABs related issues label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Commit: c29c7fee6 | Workflow run

Regression test report

Selected 4 changed tests (limit=20, 0 not selected)

8 tests: regression=8 unreleased=0 coverage=0 failing=0
test branch main (5bda5d4) latest (v1.23.0)
TestAccept/bundle/python/secrets-support/DMS=/PYDAB_VERSION=current ✅ ❌ ❌
TestAccept/bundle/python/secrets-support/DMS=true/PYDAB_VERSION=current ✅ ❌ ❌
TestAccept/bundle/refschema/DMS= ✅ ❌ ❌
TestAccept/bundle/refschema/DMS=true ✅ ❌ ❌
TestAccept/bundle/resources/secrets/basic/DMS= ✅ ❌ ❌
TestAccept/bundle/resources/secrets/basic/DMS=true ✅ ❌ ❌
TestAccept/bundle/variables/sensitive/DMS= ✅ ❌ ❌
TestAccept/bundle/variables/sensitive/DMS=true ✅ ❌ ❌
TestAccept/bundle/python/secrets-support/DMS=/PYDAB_VERSION=current: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/python/secrets-support/output.txt
        +++ /tmp/TestAcceptbundlepythonsecrets-supportDMS=PYDAB_VERSION=curr763748875/002/output.txt
        @@ -32,12 +32,10 @@
           "variables": {
             "python_secret_value": {
               "description": "The value of the secret added from Python",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "secret_value": {
               "description": "The value of the secret",
        -      "sensitive": true,
               "value": "[redacted]"
             }
           }

latest (v1.23.0): same output as main (5bda5d4)

TestAccept/bundle/python/secrets-support/DMS=true/PYDAB_VERSION=current: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/python/secrets-support/output.txt
        +++ /tmp/TestAcceptbundlepythonsecrets-supportDMS=truePYDAB_VERSION=2368570779/002/output.txt
        @@ -32,12 +32,10 @@
           "variables": {
             "python_secret_value": {
               "description": "The value of the secret added from Python",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "secret_value": {
               "description": "The value of the secret",
        -      "sensitive": true,
               "value": "[redacted]"
             }
           }
        
FAIL

latest (v1.23.0):

acceptance_test.go:1342: Diff:
        --- bundle/python/secrets-support/output.txt
        +++ /tmp/TestAcceptbundlepythonsecrets-supportDMS=truePYDAB_VERSION=2314896161/002/output.txt
        @@ -32,12 +32,10 @@
           "variables": {
             "python_secret_value": {
               "description": "The value of the secret added from Python",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "secret_value": {
               "description": "The value of the secret",
        -      "sensitive": true,
               "value": "[redacted]"
             }
           }
TestAccept/bundle/refschema/DMS=: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/refschema/output/root.txt
        +++ /tmp/TestAcceptbundlerefschemaDMS=1898199574/001/output/root.txt
        @@ -148,7 +148,6 @@
         environments.*.variables.*.lookup.query	string	INPUT
         environments.*.variables.*.lookup.service_principal	string	INPUT
         environments.*.variables.*.lookup.warehouse	string	INPUT
        -environments.*.variables.*.sensitive	bool	INPUT
         environments.*.variables.*.type	variable.VariableType	INPUT
         environments.*.variables.*.value	any	INPUT
         environments.*.workspace	*config.Workspace	INPUT
        @@ -365,7 +364,6 @@
         targets.*.variables.*.lookup.query	string	INPUT
         targets.*.variables.*.lookup.service_principal	string	INPUT
         targets.*.variables.*.lookup.warehouse	string	INPUT
        -targets.*.variables.*.sensitive	bool	INPUT
         targets.*.variables.*.type	variable.VariableType	INPUT
         targets.*.variables.*.value	any	INPUT
         targets.*.workspace	*config.Workspace	INPUT
        @@ -449,7 +447,6 @@
         variables.*.lookup.query	string	INPUT
         variables.*.lookup.service_principal	string	INPUT
         variables.*.lookup.warehouse	string	INPUT
        -variables.*.sensitive	bool	INPUT
         variables.*.type	variable.VariableType	INPUT
         variables.*.value	any	INPUT
         workspace	config.Workspace	INPUT

latest (v1.23.0):

acceptance_test.go:1342: Diff:
        --- bundle/refschema/output/root.txt
        +++ /tmp/TestAcceptbundlerefschemaDMS=3405472420/001/output/root.txt
        @@ -148,7 +148,6 @@
         environments.*.variables.*.lookup.query	string	INPUT
         environments.*.variables.*.lookup.service_principal	string	INPUT
         environments.*.variables.*.lookup.warehouse	string	INPUT
        -environments.*.variables.*.sensitive	bool	INPUT
         environments.*.variables.*.type	variable.VariableType	INPUT
         environments.*.variables.*.value	any	INPUT
         environments.*.workspace	*config.Workspace	INPUT
        @@ -365,7 +364,6 @@
         targets.*.variables.*.lookup.query	string	INPUT
         targets.*.variables.*.lookup.service_principal	string	INPUT
         targets.*.variables.*.lookup.warehouse	string	INPUT
        -targets.*.variables.*.sensitive	bool	INPUT
         targets.*.variables.*.type	variable.VariableType	INPUT
         targets.*.variables.*.value	any	INPUT
         targets.*.workspace	*config.Workspace	INPUT
        @@ -449,7 +447,6 @@
         variables.*.lookup.query	string	INPUT
         variables.*.lookup.service_principal	string	INPUT
         variables.*.lookup.warehouse	string	INPUT
        -variables.*.sensitive	bool	INPUT
         variables.*.type	variable.VariableType	INPUT
         variables.*.value	any	INPUT
         workspace	config.Workspace	INPUT
        
    acceptance_test.go:1313: Missing output file: output/data_quality_monitors.txt
    acceptance_test.go:1342: Diff:
        --- bundle/refschema/output/jobs.txt
        +++ /tmp/TestAcceptbundlerefschemaDMS=3405472420/001/output/jobs.txt
        @@ -193,8 +193,6 @@
         resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients	compute.ClientsTypes	ALL
         resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients.jobs	bool	ALL
         resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients.notebooks	bool	ALL
        -resources.jobs.*.job_clusters[*].serverless_compute	*jobs.JobClusterServerlessCompute	ALL
        -resources.jobs.*.job_clusters[*].serverless_compute.policy_id	string	ALL
         resources.jobs.*.job_clusters[*].serverless_compute_id	string	ALL
         resources.jobs.*.job_id	int64	REMOTE
         resources.jobs.*.lifecycle	resources.Lifecycle	INPUT
        @@ -233,42 +231,6 @@
         resources.jobs.*.tags.*	string	ALL
         resources.jobs.*.tasks	[]jobs.Task	ALL
         resources.jobs.*.tasks[*]	jobs.Task	ALL
        -resources.jobs.*.tasks[*].ai_function_task	*jobs.AiFunctionTask	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify	*jobs.AiClassifyFunction	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.enable_rationales	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels	*jobs.AiClassifyFunctionInlineLabels	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels	[]jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels[*]	jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels[*].description	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels[*].name	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.instructions	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.multilabel	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.version	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract	*jobs.AiExtractFunction	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.enable_citations	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.instructions	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.mode	jobs.AiExtractFunctionMode	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.schema	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.version	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document	*jobs.AiParseDocumentFunction	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.description_element_types	[]jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.description_element_types[*]	jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.image_output_path	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.page_range	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.version	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table	*jobs.DeltaTableSource	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.input_column	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.merge_key_columns	[]string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.merge_key_columns[*]	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.table_name	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output	jobs.QualifiedName	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output.catalog	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output.schema	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output.table	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.volume	*jobs.VolumeSource	ALL
        -resources.jobs.*.tasks[*].ai_function_task.volume.volume_path	string	ALL
         resources.jobs.*.tasks[*].ai_runtime_task	*jobs.AiRuntimeTask	ALL
         resources.jobs.*.tasks[*].ai_runtime_task.code_source_path	string	ALL
         resources.jobs.*.tasks[*].ai_runtime_task.deployments	[]jobs.DeploymentSpec	ALL
        @@ -377,42 +339,6 @@
         resources.jobs.*.tasks[*].for_each_task.concurrency	int	ALL
         resources.jobs.*.tasks[*].for_each_task.inputs	string	ALL
         resources.jobs.*.tasks[*].for_each_task.task	jobs.Task	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task	*jobs.AiFunctionTask	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify	*jobs.AiClassifyFunction	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.enable_rationales	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels	*jobs.AiClassifyFunctionInlineLabels	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels	[]jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels[*]	jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels[*].description	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels[*].name	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.instructions	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.multilabel	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.version	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract	*jobs.AiExtractFunction	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.enable_citations	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.instructions	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.mode	jobs.AiExtractFunctionMode	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.schema	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.version	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document	*jobs.AiParseDocumentFunction	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.description_element_types	[]jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.description_element_types[*]	jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.image_output_path	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.page_range	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.version	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table	*jobs.DeltaTableSource	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.input_column	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.merge_key_columns	[]string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.merge_key_columns[*]	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.table_name	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output	jobs.QualifiedName	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output.catalog	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output.schema	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output.table	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.volume	*jobs.VolumeSource	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.volume.volume_path	string	ALL
         resources.jobs.*.tasks[*].for_each_task.task.ai_runtime_task	*jobs.AiRuntimeTask	ALL
         resources.jobs.*.tasks[*].for_each_task.task.ai_runtime_task.code_source_path	string	ALL
         resources.jobs.*.tasks[*].for_each_task.task.ai_runtime_task.deployments	[]jobs.DeploymentSpec	ALL
        @@ -1117,12 +1043,6 @@
         resources.jobs.*.trigger.file_arrival.min_time_between_triggers_seconds	int	ALL
         resources.jobs.*.trigger.file_arrival.url	string	ALL
         resources.jobs.*.trigger.file_arrival.wait_after_last_change_seconds	int	ALL
        -resources.jobs.*.trigger.job_completion	*jobs.JobCompletionTriggerConfiguration	ALL
        -resources.jobs.*.trigger.job_completion.if_status	jobs.JobCompletionTriggerConfigurationIfStatus	ALL
        -resources.jobs.*.trigger.job_completion.jobs	[]jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.trigger.job_completion.jobs[*]	jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.trigger.job_completion.jobs[*].job_id	string	ALL
        -resources.jobs.*.trigger.job_completion.jobs[*].workspace_id	string	ALL
         resources.jobs.*.trigger.model	*jobs.ModelTriggerConfiguration	ALL
         resources.jobs.*.trigger.model.aliases	[]string	ALL
         resources.jobs.*.trigger.model.aliases[*]	string	ALL
        @@ -1166,17 +1086,6 @@
         resources.jobs.*.trigger_details[*].state.continuous.next_attempt_ms	int64	REMOTE
         resources.jobs.*.trigger_details[*].state.file_arrival	*jobs.FileArrivalTriggerState	REMOTE
         resources.jobs.*.trigger_details[*].state.file_arrival.using_file_events	bool	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion	*jobs.JobCompletionTriggerState	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states	[]jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*]	jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].has_seen_completion	bool	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].is_deleted	bool	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].job_id	string	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].last_seen_completion_time	*time.Time	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].last_seen_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].last_seen_run_id	string	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].latest_non_matching_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].workspace_id	string	REMOTE
         resources.jobs.*.trigger_details[*].state.model	*jobs.ModelTriggerState	REMOTE
         resources.jobs.*.trigger_details[*].state.pause_status	jobs.PauseStatus	REMOTE
         resources.jobs.*.trigger_details[*].state.periodic	*jobs.PeriodicTriggerState	REMOTE
        @@ -1195,17 +1104,6 @@
         resources.jobs.*.trigger_state	*jobs.TriggerStateProto	REMOTE
         resources.jobs.*.trigger_state.file_arrival	*jobs.FileArrivalTriggerState	REMOTE
         resources.jobs.*.trigger_state.file_arrival.using_file_events	bool	REMOTE
        -resources.jobs.*.trigger_state.job_completion	*jobs.JobCompletionTriggerState	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states	[]jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*]	jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].has_seen_completion	bool	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].is_deleted	bool	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].job_id	string	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].last_seen_completion_time	*time.Time	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].last_seen_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].last_seen_run_id	string	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].latest_non_matching_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].workspace_id	string	REMOTE
         resources.jobs.*.trigger_state.pause_status	jobs.PauseStatus	REMOTE
         resources.jobs.*.trigger_state.sql_condition	*jobs.SqlConditionState	REMOTE
         resources.jobs.*.trigger_state.sql_condition.latest_condition_evaluation_satisfied	bool	REMOTE
        @@ -1229,12 +1127,6 @@
         resources.jobs.*.triggers[*].file_arrival.min_time_between_triggers_seconds	int	ALL
         resources.jobs.*.triggers[*].file_arrival.url	string	ALL
         resources.jobs.*.triggers[*].file_arrival.wait_after_last_change_seconds	int	ALL
        -resources.jobs.*.triggers[*].job_completion	*jobs.JobCompletionTriggerConfiguration	ALL
        -resources.jobs.*.triggers[*].job_completion.if_status	jobs.JobCompletionTriggerConfigurationIfStatus	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs	[]jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs[*]	jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs[*].job_id	string	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs[*].workspace_id	string	ALL
         resources.jobs.*.triggers[*].model	*jobs.ModelTriggerConfiguration	ALL
         resources.jobs.*.triggers[*].model.aliases	[]string	ALL
         resources.jobs.*.triggers[*].model.aliases[*]	string	ALL
TestAccept/bundle/refschema/DMS=true: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/refschema/output/root.txt
        +++ /tmp/TestAcceptbundlerefschemaDMS=true1460306633/001/output/root.txt
        @@ -148,7 +148,6 @@
         environments.*.variables.*.lookup.query	string	INPUT
         environments.*.variables.*.lookup.service_principal	string	INPUT
         environments.*.variables.*.lookup.warehouse	string	INPUT
        -environments.*.variables.*.sensitive	bool	INPUT
         environments.*.variables.*.type	variable.VariableType	INPUT
         environments.*.variables.*.value	any	INPUT
         environments.*.workspace	*config.Workspace	INPUT
        @@ -365,7 +364,6 @@
         targets.*.variables.*.lookup.query	string	INPUT
         targets.*.variables.*.lookup.service_principal	string	INPUT
         targets.*.variables.*.lookup.warehouse	string	INPUT
        -targets.*.variables.*.sensitive	bool	INPUT
         targets.*.variables.*.type	variable.VariableType	INPUT
         targets.*.variables.*.value	any	INPUT
         targets.*.workspace	*config.Workspace	INPUT
        @@ -449,7 +447,6 @@
         variables.*.lookup.query	string	INPUT
         variables.*.lookup.service_principal	string	INPUT
         variables.*.lookup.warehouse	string	INPUT
        -variables.*.sensitive	bool	INPUT
         variables.*.type	variable.VariableType	INPUT
         variables.*.value	any	INPUT
         workspace	config.Workspace	INPUT

latest (v1.23.0):

acceptance_test.go:1342: Diff:
        --- bundle/refschema/output/jobs.txt
        +++ /tmp/TestAcceptbundlerefschemaDMS=true3483033345/001/output/jobs.txt
        @@ -193,8 +193,6 @@
         resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients	compute.ClientsTypes	ALL
         resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients.jobs	bool	ALL
         resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients.notebooks	bool	ALL
        -resources.jobs.*.job_clusters[*].serverless_compute	*jobs.JobClusterServerlessCompute	ALL
        -resources.jobs.*.job_clusters[*].serverless_compute.policy_id	string	ALL
         resources.jobs.*.job_clusters[*].serverless_compute_id	string	ALL
         resources.jobs.*.job_id	int64	REMOTE
         resources.jobs.*.lifecycle	resources.Lifecycle	INPUT
        @@ -233,42 +231,6 @@
         resources.jobs.*.tags.*	string	ALL
         resources.jobs.*.tasks	[]jobs.Task	ALL
         resources.jobs.*.tasks[*]	jobs.Task	ALL
        -resources.jobs.*.tasks[*].ai_function_task	*jobs.AiFunctionTask	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify	*jobs.AiClassifyFunction	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.enable_rationales	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels	*jobs.AiClassifyFunctionInlineLabels	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels	[]jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels[*]	jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels[*].description	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.inline_labels.labels[*].name	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.instructions	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.multilabel	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_classify.version	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract	*jobs.AiExtractFunction	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.enable_citations	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.instructions	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.mode	jobs.AiExtractFunctionMode	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.schema	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_extract.version	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document	*jobs.AiParseDocumentFunction	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.description_element_types	[]jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.description_element_types[*]	jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.image_output_path	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.page_range	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.ai_parse_document.version	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table	*jobs.DeltaTableSource	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.input_column	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.merge_key_columns	[]string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.merge_key_columns[*]	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.delta_table.table_name	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output	jobs.QualifiedName	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output.catalog	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output.schema	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.output.table	string	ALL
        -resources.jobs.*.tasks[*].ai_function_task.volume	*jobs.VolumeSource	ALL
        -resources.jobs.*.tasks[*].ai_function_task.volume.volume_path	string	ALL
         resources.jobs.*.tasks[*].ai_runtime_task	*jobs.AiRuntimeTask	ALL
         resources.jobs.*.tasks[*].ai_runtime_task.code_source_path	string	ALL
         resources.jobs.*.tasks[*].ai_runtime_task.deployments	[]jobs.DeploymentSpec	ALL
        @@ -377,42 +339,6 @@
         resources.jobs.*.tasks[*].for_each_task.concurrency	int	ALL
         resources.jobs.*.tasks[*].for_each_task.inputs	string	ALL
         resources.jobs.*.tasks[*].for_each_task.task	jobs.Task	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task	*jobs.AiFunctionTask	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify	*jobs.AiClassifyFunction	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.enable_rationales	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels	*jobs.AiClassifyFunctionInlineLabels	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels	[]jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels[*]	jobs.AiClassifyFunctionLabel	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels[*].description	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.inline_labels.labels[*].name	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.instructions	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.multilabel	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_classify.version	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract	*jobs.AiExtractFunction	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.enable_citations	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.enable_confidence_scores	bool	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.instructions	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.mode	jobs.AiExtractFunctionMode	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.schema	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_extract.version	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document	*jobs.AiParseDocumentFunction	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.description_element_types	[]jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.description_element_types[*]	jobs.AiParseDocumentFunctionDescriptionElementType	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.image_output_path	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.page_range	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.ai_parse_document.version	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table	*jobs.DeltaTableSource	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.input_column	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.merge_key_columns	[]string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.merge_key_columns[*]	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.delta_table.table_name	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output	jobs.QualifiedName	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output.catalog	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output.schema	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.output.table	string	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.volume	*jobs.VolumeSource	ALL
        -resources.jobs.*.tasks[*].for_each_task.task.ai_function_task.volume.volume_path	string	ALL
         resources.jobs.*.tasks[*].for_each_task.task.ai_runtime_task	*jobs.AiRuntimeTask	ALL
         resources.jobs.*.tasks[*].for_each_task.task.ai_runtime_task.code_source_path	string	ALL
         resources.jobs.*.tasks[*].for_each_task.task.ai_runtime_task.deployments	[]jobs.DeploymentSpec	ALL
        @@ -1117,12 +1043,6 @@
         resources.jobs.*.trigger.file_arrival.min_time_between_triggers_seconds	int	ALL
         resources.jobs.*.trigger.file_arrival.url	string	ALL
         resources.jobs.*.trigger.file_arrival.wait_after_last_change_seconds	int	ALL
        -resources.jobs.*.trigger.job_completion	*jobs.JobCompletionTriggerConfiguration	ALL
        -resources.jobs.*.trigger.job_completion.if_status	jobs.JobCompletionTriggerConfigurationIfStatus	ALL
        -resources.jobs.*.trigger.job_completion.jobs	[]jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.trigger.job_completion.jobs[*]	jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.trigger.job_completion.jobs[*].job_id	string	ALL
        -resources.jobs.*.trigger.job_completion.jobs[*].workspace_id	string	ALL
         resources.jobs.*.trigger.model	*jobs.ModelTriggerConfiguration	ALL
         resources.jobs.*.trigger.model.aliases	[]string	ALL
         resources.jobs.*.trigger.model.aliases[*]	string	ALL
        @@ -1166,17 +1086,6 @@
         resources.jobs.*.trigger_details[*].state.continuous.next_attempt_ms	int64	REMOTE
         resources.jobs.*.trigger_details[*].state.file_arrival	*jobs.FileArrivalTriggerState	REMOTE
         resources.jobs.*.trigger_details[*].state.file_arrival.using_file_events	bool	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion	*jobs.JobCompletionTriggerState	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states	[]jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*]	jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].has_seen_completion	bool	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].is_deleted	bool	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].job_id	string	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].last_seen_completion_time	*time.Time	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].last_seen_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].last_seen_run_id	string	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].latest_non_matching_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_details[*].state.job_completion.monitored_job_states[*].workspace_id	string	REMOTE
         resources.jobs.*.trigger_details[*].state.model	*jobs.ModelTriggerState	REMOTE
         resources.jobs.*.trigger_details[*].state.pause_status	jobs.PauseStatus	REMOTE
         resources.jobs.*.trigger_details[*].state.periodic	*jobs.PeriodicTriggerState	REMOTE
        @@ -1195,17 +1104,6 @@
         resources.jobs.*.trigger_state	*jobs.TriggerStateProto	REMOTE
         resources.jobs.*.trigger_state.file_arrival	*jobs.FileArrivalTriggerState	REMOTE
         resources.jobs.*.trigger_state.file_arrival.using_file_events	bool	REMOTE
        -resources.jobs.*.trigger_state.job_completion	*jobs.JobCompletionTriggerState	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states	[]jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*]	jobs.JobCompletionTriggerStateMonitoredJobState	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].has_seen_completion	bool	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].is_deleted	bool	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].job_id	string	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].last_seen_completion_time	*time.Time	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].last_seen_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].last_seen_run_id	string	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].latest_non_matching_result	jobs.TerminationCodeCode	REMOTE
        -resources.jobs.*.trigger_state.job_completion.monitored_job_states[*].workspace_id	string	REMOTE
         resources.jobs.*.trigger_state.pause_status	jobs.PauseStatus	REMOTE
         resources.jobs.*.trigger_state.sql_condition	*jobs.SqlConditionState	REMOTE
         resources.jobs.*.trigger_state.sql_condition.latest_condition_evaluation_satisfied	bool	REMOTE
        @@ -1229,12 +1127,6 @@
         resources.jobs.*.triggers[*].file_arrival.min_time_between_triggers_seconds	int	ALL
         resources.jobs.*.triggers[*].file_arrival.url	string	ALL
         resources.jobs.*.triggers[*].file_arrival.wait_after_last_change_seconds	int	ALL
        -resources.jobs.*.triggers[*].job_completion	*jobs.JobCompletionTriggerConfiguration	ALL
        -resources.jobs.*.triggers[*].job_completion.if_status	jobs.JobCompletionTriggerConfigurationIfStatus	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs	[]jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs[*]	jobs.JobCompletionTriggerConfigurationMonitoredJob	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs[*].job_id	string	ALL
        -resources.jobs.*.triggers[*].job_completion.jobs[*].workspace_id	string	ALL
         resources.jobs.*.triggers[*].model	*jobs.ModelTriggerConfiguration	ALL
         resources.jobs.*.triggers[*].model.aliases	[]string	ALL
         resources.jobs.*.triggers[*].model.aliases[*]	string	ALL
        
    acceptance_test.go:1342: Diff:
        --- bundle/refschema/output/root.txt
        +++ /tmp/TestAcceptbundlerefschemaDMS=true3483033345/001/output/root.txt
        @@ -148,7 +148,6 @@
         environments.*.variables.*.lookup.query	string	INPUT
         environments.*.variables.*.lookup.service_principal	string	INPUT
         environments.*.variables.*.lookup.warehouse	string	INPUT
        -environments.*.variables.*.sensitive	bool	INPUT
         environments.*.variables.*.type	variable.VariableType	INPUT
         environments.*.variables.*.value	any	INPUT
         environments.*.workspace	*config.Workspace	INPUT
        @@ -365,7 +364,6 @@
         targets.*.variables.*.lookup.query	string	INPUT
         targets.*.variables.*.lookup.service_principal	string	INPUT
         targets.*.variables.*.lookup.warehouse	string	INPUT
        -targets.*.variables.*.sensitive	bool	INPUT
         targets.*.variables.*.type	variable.VariableType	INPUT
         targets.*.variables.*.value	any	INPUT
         targets.*.workspace	*config.Workspace	INPUT
        @@ -449,7 +447,6 @@
         variables.*.lookup.query	string	INPUT
         variables.*.lookup.service_principal	string	INPUT
         variables.*.lookup.warehouse	string	INPUT
        -variables.*.sensitive	bool	INPUT
         variables.*.type	variable.VariableType	INPUT
         variables.*.value	any	INPUT
         workspace	config.Workspace	INPUT
        
    acceptance_test.go:1313: Missing output file: output/data_quality_monitors.txt
FAIL
TestAccept/bundle/resources/secrets/basic/DMS=: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/resources/secrets/basic/output.txt
        +++ /tmp/TestAcceptbundleresourcessecretsbasicDMS=3548133725/002/output.txt
        @@ -16,7 +16,6 @@
         {
           "secret_value": {
             "description": "Test secret value",
        -    "sensitive": true,
             "value": "[redacted]"
           }
         }

latest (v1.23.0): same output as main (5bda5d4)

TestAccept/bundle/resources/secrets/basic/DMS=true: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/resources/secrets/basic/output.txt
        +++ /tmp/TestAcceptbundleresourcessecretsbasicDMS=true344878849/002/output.txt
        @@ -16,7 +16,6 @@
         {
           "secret_value": {
             "description": "Test secret value",
        -    "sensitive": true,
             "value": "[redacted]"
           }
         }

latest (v1.23.0): same output as main (5bda5d4)

TestAccept/bundle/variables/sensitive/DMS=: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/variables/sensitive/output.txt
        +++ /tmp/TestAcceptbundlevariablessensitiveDMS=2805917281/002/output.txt
        @@ -2,26 +2,29 @@
         === A variable used as the value of a secret is sensitive and redacted in validate -o json
         
         >>> [CLI] bundle validate -o json
        +Warning: unknown field: sensitive
        +  at variables.api_header
        +  in databricks.yml:13:5
        +
        +contains error: 'dummy-credential' was not expected: '"default": "Bearer dummy-credential",'
         {
           "variables": {
             "api_credential": {
               "description": "Credential for the external API, passed via BUNDLE_VAR_api_credential. Sensitive because it is used as the value of a secret.",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "api_header": {
        -      "default": "[redacted]",
        +      "default": "Bearer dummy-credential",
               "description": "A sensitive variable may reference another sensitive variable",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "target_catalog": {
               "default": "example_catalog",
        -      "value": "example_catalog"
        +      "value": "[redacted]"
             },
             "target_schema": {
               "default": "example_schema",
        -      "value": "example_schema"
        +      "value": "[redacted]"
             }
           },
           "resources": {
        @@ -36,62 +24,4 @@
           }
         }
         
        -=== Sensitive variable used in non-sensitive resource fields
        -
        ->>> [CLI] bundle validate
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at resources.jobs.example_job.tasks[0].notebook_task.base_parameters.token
        -  in databricks.yml:25:22
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at resources.secrets.example_secret.comment
        -  in databricks.yml:15:16
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Name: sensitive-variables
        -Target: default
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/sensitive-variables/default
        -
        -Found 2 errors
        -
        -=== Sensitive variable used in the default of a non-sensitive variable
        -
        ->>> [CLI] bundle validate
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at variables.api_header.default
        -  in databricks.yml:9:14
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Name: sensitive-variables
        -Target: default
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/sensitive-variables/default
        -
        -Found 1 error
        -
        -=== Sensitive variable passed as the value of a non-sensitive variable
        -
        ->>> [CLI] bundle validate --var target_schema=${var.api_credential}
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at variables.target_schema.value
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Name: sensitive-variables
        -Target: default
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/sensitive-variables/default
        -
        -Found 1 error
        +Exit code: 1

latest (v1.23.0): same output as main (5bda5d4)

TestAccept/bundle/variables/sensitive/DMS=true: branch ✅ | main (5bda5d4) ❌ | latest (v1.23.0) ❌

main (5bda5d4):

acceptance_test.go:1342: Diff:
        --- bundle/variables/sensitive/output.txt
        +++ /tmp/TestAcceptbundlevariablessensitiveDMS=true193032660/002/output.txt
        @@ -2,26 +2,29 @@
         === A variable used as the value of a secret is sensitive and redacted in validate -o json
         
         >>> [CLI] bundle validate -o json
        +Warning: unknown field: sensitive
        +  at variables.api_header
        +  in databricks.yml:13:5
        +
        +contains error: 'dummy-credential' was not expected: '"default": "Bearer dummy-credential",'
         {
           "variables": {
             "api_credential": {
               "description": "Credential for the external API, passed via BUNDLE_VAR_api_credential. Sensitive because it is used as the value of a secret.",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "api_header": {
        -      "default": "[redacted]",
        +      "default": "Bearer dummy-credential",
               "description": "A sensitive variable may reference another sensitive variable",
        -      "sensitive": true,
               "value": "[redacted]"
             },
             "target_catalog": {
               "default": "example_catalog",
        -      "value": "example_catalog"
        +      "value": "[redacted]"
             },
             "target_schema": {
               "default": "example_schema",
        -      "value": "example_schema"
        +      "value": "[redacted]"
             }
           },
           "resources": {
        @@ -36,62 +24,4 @@
           }
         }
         
        -=== Sensitive variable used in non-sensitive resource fields
        -
        ->>> [CLI] bundle validate
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at resources.jobs.example_job.tasks[0].notebook_task.base_parameters.token
        -  in databricks.yml:25:22
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at resources.secrets.example_secret.comment
        -  in databricks.yml:15:16
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Name: sensitive-variables
        -Target: default
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/sensitive-variables/default
        -
        -Found 2 errors
        -
        -=== Sensitive variable used in the default of a non-sensitive variable
        -
        ->>> [CLI] bundle validate
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at variables.api_header.default
        -  in databricks.yml:9:14
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Name: sensitive-variables
        -Target: default
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/sensitive-variables/default
        -
        -Found 1 error
        -
        -=== Sensitive variable passed as the value of a non-sensitive variable
        -
        ->>> [CLI] bundle validate --var target_schema=${var.api_credential}
        -Error: Sensitive variable "api_credential" is used in a non-sensitive field
        -  at variables.target_schema.value
        -
        -The variable "api_credential" is sensitive, so its value must not appear in plain text.
        -A variable is sensitive when it is marked with "sensitive: true" or used as the value of a sensitive resource field (such as resources.secrets.*.value). It can only be used in such a field or in the definition of another sensitive variable.
        -
        -Name: sensitive-variables
        -Target: default
        -Workspace:
        -  User: [USERNAME]
        -  Path: /Workspace/Users/[USERNAME]/.bundle/sensitive-variables/default
        -
        -Found 1 error
        +Exit code: 1

latest (v1.23.0): same output as main (5bda5d4)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: c29c7fe

Run: 41828041551

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 274 18 4:48
✅​ aws-windows-latest-4core-16gb 276 16 4:06
✅​ azure linux-2core-8gb 273 18 5:22
✅​ azure-windows-latest-4core-16gb 275 16 3:57
✅​ gcp linux-2core-8gb 274 18 5:33
✅​ gcp-windows-latest-4core-16gb 276 16 4:14
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:23 gcp-windows-latest-4core-16gb TestAccept
3:18 aws-windows-latest-4core-16gb TestAccept
3:17 azure-windows-latest-4core-16gb TestAccept

@github-actions github-actions Bot added the PyDABs label Nov 4, 2026
@andrewnester
andrewnester requested a review from pietern November 4, 2026 11:00
@andrewnester
andrewnester added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 7291cad Nov 4, 2026
31 checks passed
@andrewnester
andrewnester deleted the fix/sensitive-redaction branch November 4, 2026 12:46
deco-sdk-tagging Bot added a commit that referenced this pull request Nov 4, 2026
## Release v1.24.0

### Notable Changes

 * A variable used as the value of a secret is now sensitive: referencing it from any other field, such as a job parameter or a secret's `comment`, is an error because the value would be used in plain text. Mark other secret-holding variables with `sensitive: true` to get the same protection. ([#7288](#7288))

### CLI

 * Fix `databricks auth login` and `databricks configure` overwriting the parent profile's host and token when saving a profile with a dotted name such as `team.prod` next to an existing `[team]`. ([#7266](#7266))
 * `databricks auth login` and `databricks auth token` now reject the name of an existing profile after "Create a new profile" instead of logging in to that profile. ([#7196](#7196))
 * `databricks auth login` and `databricks auth token` now reject a host that is not a URL with a hostname, such as `/`, `?o=123` or `https://`, instead of falling back to `DATABRICKS_HOST` or failing later. An empty host after "Enter a host URL manually" in the login profile picker no longer starts discovery login. ([#7195](#7195))
 * Prompts that validate their input, such as the `databricks configure` host prompt, no longer open with a red cross before anything is typed. ([#7314](#7314))
 * `databricks ssh connect` now documents `--environment-version` for choosing the serverless environment version, starts a separate default-named server for each version, and rejects the flag together with `--cluster`. Default connection names also include the environment version in their hash, so the default name when `--environment-version` is not specified changes with this release. Configuration associated with the old connection name (secret scope, SSH keys and known hosts in your local `.databricks` folder) will no longer be used and can be safely removed. ([#7310](#7310))

### AI Runtime

 * Show workspace-supported accelerator types in `databricks air run -h config.compute.accelerator_type`. ([#7275](#7275))
 * `air list` now reports `No active runs found.` for an empty active-only result. ([#7276](#7276))
 * Stop printing a retry notice on every `air run --watch`; retries are still reported when they happen. ([#7270](#7270))

### Bundles

 * Fix `bundle plan` reporting an update to an app's `config.env` values on every run, which also redeployed the app on every `bundle deploy`. The Apps API returns environment variable names only, so `value` and `value_from` are no longer compared against the remote deployment. ([#7326](#7326))
 * Add the `data_quality_monitors` resource for data quality monitors on Unity Catalog schemas and tables (direct engine only). ([#7094](#7094))
 * `bundle deploy` now asks for confirmation before recreating or deleting a Feature View, and `bundle deploy` and `bundle destroy` warn that FeatureSpecs using a Feature View are not checked. ([#7164](#7164))
 * Accept grant `privileges` values that the SDK enum does not list yet: `bundle validate` no longer warns (or errors under `--strict`) on them, and the PyDABs `Privilege` enum accepts unknown strings instead of raising. The JSON schema and PyDABs still advertise the known values for completion. ([#7315](#7315))
 * Add the `materialization_sets` resource, which materializes Feature Store Feature Views to offline and online stores. ([#7164](#7164))
 * Detect notebooks with the same rule as the workspace: a `.py`, `.sql`, `.scala` or `.r` file is a notebook when its first line contains `Databricks notebook source`, for example after a byte order mark. Previously such a file was tracked as a plain file, so removing it locally left the remote notebook behind. ([#7290](#7290))
 * Add `sensitive: true` for bundle variables. A variable used as the value of a secret is sensitive by default. `bundle validate -o json` redacts only sensitive variables instead of every string variable, and a sensitive variable can only be referenced from a sensitive resource field (such as a secret's `value`) or from another sensitive variable. ([#7288](#7288))

### Dependency Updates

 * Bump dependencies with known vulnerabilities. ([#7305](#7305))
 * Bump dependencies with known vulnerabilities. ([#7327](#7327))
 * Update the Go SDK to [v0.191.1](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.191.1). ([#7277](#7277))
 * Update the Go SDK to [v0.193.0](https://github.com/databricks/databricks-sdk-go/releases/tag/v0.193.0). ([#7303](#7303))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues PyDABs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UC secrets: supported access to resolved non-secret variables and redaction compatibility

3 participants