Repository navigation
Conversation
radakam
marked this pull request as ready for review
August 21, 2026 15:59
Contributor
Approval status: pending
|
Collaborator
✅ Integration test reportCommit: 5d9dfa5
Top 4 slowest tests (at least 2 minutes):
|
radakam
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
August 24, 2026 09:36
0db1f4b to
e431088
Compare
radakam
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
3 times, most recently
from
August 25, 2026 10:12
d65e640 to
b0437a3
Compare
radakam
marked this pull request as draft
August 26, 2026 13:27
radakam
marked this pull request as ready for review
August 27, 2026 17:19
denik
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
6 times, most recently
from
September 9, 2026 14:12
69a8871 to
cfc94b7
Compare
denik
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
from
September 10, 2026 12:52
24f9f1f to
6391342
Compare
Base automatically changed from
deco-job-runs-lifecycle-on-file-change
to
main
September 11, 2026 08:23
denik
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
2 times, most recently
from
September 14, 2026 11:15
e303d92 to
aa5189f
Compare
denik
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
September 21, 2026 11:55
aa5189f to
fb7ee59
Compare
denik
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
October 5, 2026 09:37
fb7ee59 to
f529e5b
Compare
denik
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
October 5, 2026 09:55
f529e5b to
c682c05
Compare
Adds `lifecycle.triggers.on_value_change` for `resources.job_runs` (direct
engine): watch an interpolated expression (a variable, workspace field, or a
${resources.*} reference) and re-fire the run when its resolved value changes.
- Snapshot the expression before interpolation and use the trimmed form as the
stable state key (SnapshotJobRunValueTriggers), so a value change is a
modification of one entry rather than a change to the set of watches.
- Resolution reuses existing mechanisms: normal variable interpolation resolves
${var.*}/${workspace.*} into triggers_state, and the planner resolves
${resources.*} via the references PrepareInputConfig registers, so the deploy
graph depends on them. The resolved value is stored in state directly.
- Removing or adding a watch is classified in OverrideChangeDesc alongside
on_file_change: adding/changing recreates the run, removing keeps it.
Built on the on_file_change work (#6309): no PersistState, path.String()-based
change classification, prevent_destroy left to the generic checkForPreventDestroy.
Co-authored-by: Isaac <no-reply@databricks.com>
…y is allowed Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
denik
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
November 9, 2026 10:29
c682c05 to
94850c1
Compare
Contributor
|
Commit: Regression test reportSelected 6 changed tests (limit=20, 0 not selected) 14 tests: regression=14 unreleased=0 coverage=0 failing=0
TestAccept/bundle/python/job_runs-on_value_change/DMS=/PYDAB_VERSION=current: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/python/job_runs-on_value_change/DMS=true/PYDAB_VERSION=current: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/refschema/DMS=: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/refschema/DMS=true: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/resources/job_runs/on_value_change/DMS=: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/resources/job_runs/on_value_change/DMS=true: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): TestAccept/bundle/resources/job_runs/on_value_change_empty/DMS=: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/resources/job_runs/on_value_change_empty/DMS=true: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/resources/job_runs/on_value_change_errors/DMS=: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/resources/job_runs/on_value_change_errors/DMS=true: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) TestAccept/bundle/resources/job_runs/on_value_change_ref/DMS=/READPLAN=: branch ✅ | main (df79350) ❌ | latest (v1.25.0) ❌main (df79350): latest (v1.25.0): same output as main (df79350) 3 more failure details omitted, see the full report in the run. |
Record today's behavior, including the bugs: - python/job_runs-on_value_change: any bundle with a python: section and an on_value_change watch fails validate with "Unexpected field 'triggers_state' for class JobRunLifecycle", for YAML- and Python-defined job_runs alike. - on_value_change_empty: a watched variable whose value is "" fails with "lifecycle.triggers entry must set ..." on an entry that does set it. - on_value_change_errors: a complex variable gives two "expected string, found map" warnings (one at the internal triggers_state path, at :0:0) and the same misleading error. - on_value_change_ref: updating a referenced job in place does not re-fire the run (correct; locks it in). Co-authored-by: Isaac <no-reply@databricks.com>
SnapshotJobRunValueTriggers ran in ProcessStaticResources, before PythonMutator, so Python saw the internal lifecycle.triggers_state, which the PyDABs JobRunLifecycle model does not declare: any bundle with a python: section and an on_value_change watch failed with "Unexpected field 'triggers_state' for class JobRunLifecycle". on_value_change expressions are now never resolved in place (dynvar.ResolveExcept, keepUnresolved), so the expression survives until after PythonMutator, where it is copied into triggers_state and resolved there (ResolveJobRunValueTriggerReferences). This also lets a Python mutator add a watch, drops the "already snapshotted" guard, and fixes a watched value of "": the trigger entry keeps its expression, so DropEmptyStrings no longer empties it. A complex value now only warns; the next commit rejects it. Co-authored-by: Isaac <no-reply@databricks.com>
The value is stored as a string, so normalization dropped a complex value with a warning and the watch silently never fired. ValidateJobRunTriggers now reports it. The snapshot is written to the dynamic config with the trigger's location, so the normalization warning points at the trigger instead of ":0:0". The old snapshot unit test edited typed config between two applies, which the next apply overwrote from the dynamic config, so it never reached the guard it meant to test; replace it with one for the current behavior. Co-authored-by: Isaac <no-reply@databricks.com>
Removing a watch keeps its last value in state, the same as on_file_change, so re-adding it with an unchanged value does not re-fire the run. The field description said any added watch re-fires, and the test step that shows the opposite was titled "re-fires". Describe the rule as a comparison with the value recorded when the run last fired, and fix the title. Co-authored-by: Isaac <no-reply@databricks.com>
The planner's extractReferences already registers the ${resources.*}
references in lifecycle.triggers_state.on_value_change from the dynamic
config, so PrepareInputConfig added the same refs a second time; the README
reserves the hook for sub-resources. The maps.Clone in PrepareState
protected entries from fingerprinting that no longer exists.
Co-authored-by: Isaac <no-reply@databricks.com>
Iterating the job_runs map made the order of diagnostics from several job_runs random; ResolveJobRunFileTriggers sorts for the same reason. The [VALUE_HASH] replacement in on_value_change_ref was left over from when values were stored as hashes and matches nothing. Co-authored-by: Isaac <no-reply@databricks.com>
Replace the resolver change from the previous commits: on_value_change no longer stays unresolved (dynvar.ResolveExcept and keepUnresolved are gone), so the trigger shows its resolved value like any other field. SnapshotJobRunValueTriggers records the expressions on the Bundle before variable resolution (as RootPathIsNameTargetScoped does), and after PythonMutator ResolveJobRunValueTriggers stores the resolved trigger values in triggers_state under them. Python never sees triggers_state. Values are paired with expressions by trigger position, so a Python mutator that adds or removes triggers of a job_run with on_value_change gets an error. Co-authored-by: Isaac <no-reply@databricks.com>
Replace the expression-keyed state with the interpolated values in trigger order. ResolveJobRunValueTriggers copies them into triggers_state after PythonMutator; nothing is recorded before interpolation, so the snapshot mutator, the Bundle field and the Python mutator limitation are gone, and duplicate or blank expressions are no longer errors. The planner compares the values per position: a changed or added value re-fires the run, and a list that keeps the leading values of the old one (watches removed from the end) does not. Removing or reordering any other watch shifts the values after it, which re-fires. ValidateJobRunTriggers moves before DropEmptyStrings, which would drop a watched "" value and leave an entry that sets nothing. Co-authored-by: Isaac <no-reply@databricks.com>
DropEmptyStrings exists so "" on an omitempty field is not force-sent to the backend. lifecycle is never sent, and there an empty string is a value: job_runs lifecycle.triggers.on_value_change can watch a variable that is "". Dropping it left a trigger entry that sets nothing. With lifecycle left alone, ValidateJobRunTriggers goes back to its place after DropEmptyStrings. Co-authored-by: Isaac <no-reply@databricks.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Adds
lifecycle.triggers.on_value_changeforresources.job_runs(direct engine): re-fire the run when the interpolated value of a watched expression changes, e.g. a variable,${workspace.root_path}, or another resource's id.${resources.*}references in the expression become deploy dependencies.Resolved values are stored in state in trigger order and compared by position: changing or adding a watch re-fires the run, removing the last watches does not, and removing or reordering any other watch does.
Why
on_bundle_deployandon_file_changealready re-fire a run on deploy and on file changes.on_value_changecovers re-running when a config value the run depends on changes.Tests
Acceptance tests in
acceptance/bundle/resources/job_runs/on_value_change*(values, references, empty value, errors) andacceptance/bundle/python/job_runs-on_value_change(watches defined in YAML, in Python, and added by a Python mutator).This pull request and its description were written by Isaac.