Remove the empty .bundle/<name> directory on destroy - #6317
Merged
Conversation
`bundle destroy` deleted `~/.bundle/<name>/<target>` but never the `<name>` parent, so every destroy left an empty directory behind. Deployments that use a fresh bundle name each time never reuse those directories, so they accumulate and count against the workspace child-node limit; the CLI's own CI workspace reached ~27k of them per day. Delete the parent too, non-recursively so it survives while another target of the bundle is still deployed there. The delete is restricted to the `~/.bundle/<name>/<target>` layout the CLI generates, since `root_path` is user-configurable and the parent of an arbitrary path is not ours to remove. The fake workspace accepted a non-recursive delete of a non-empty directory, which the real API rejects with DIRECTORY_NOT_EMPTY. Model that, so the sibling-target case is covered by the local suite. Co-authored-by: Isaac
Co-authored-by: Isaac
Collaborator
Integration test reportCommit: f1f3a7c
Top 5 slowest tests (at least 2 minutes):
|
The first version keyed off the path shape (a parent directly under .bundle),
which both missed a root_path pointing elsewhere and could not tell a configured
path from the generated one.
Record instead, in DefineDefaultWorkspaceRoot, whether root_path ends in the
bundle name and target. That mutator runs before variable resolution, so
${bundle.name} and ${bundle.target} are still literal there and a path that
merely happens to end in the same two segments does not qualify. Applying the
default sets the flag too, since the default has exactly that shape.
Destroy removes the directory above the deployment root only when the flag is
set, so it works for any prefix and never touches a directory the configuration
did not scope to this bundle.
Co-authored-by: Isaac
Verified against a live workspace: a non-recursive delete of a directory holding a file or a subdirectory returns 400 DIRECTORY_NOT_EMPTY with "Folder (<path>) is not empty". The status and code already matched; align the message too. Co-authored-by: Isaac
Names and paths now carry $UNIQUE_NAME so concurrent legs do not collide, and each test cleans up the prefix it created. Also trims two comments to the 1-3 line cap. Co-authored-by: Isaac
shreyas-goenka
marked this pull request as ready for review
August 24, 2026 09:36
denik
approved these changes
Aug 24, 2026
| // bundle name and target references. | ||
| func endsWithNameAndTarget(rootPath string) bool { | ||
| rootPath = strings.TrimSuffix(rootPath, "/") | ||
| return path.Base(rootPath) == bundleTargetRef && path.Base(path.Dir(rootPath)) == bundleNameRef |
Contributor
There was a problem hiding this comment.
why not just HasSuffix(strings.TrimSuffix(rootPath, "/"), "${bundle.name}/${bundle.target}")?
given that that this is uninterpolated string, using path module on it might not work.
Also, I'd narrow it further HasSuffix(strings.TrimSuffix(rootPath, "/"), "/${bundle.target}"), the fact that it ends with target already tells us that parent is bundle-scoped.
Contributor
Author
There was a problem hiding this comment.
the fact that it ends with target already tells us that parent is bundle-scoped.
The directory we are deleting is ${bundle.name} directory though. If a user has something configurd:
workspace:
root_path: /Worksapce/my_personal_project/${bundle.target}`
Then we should avoid deleting an empty my_personal_project directory since that could be suprising / annoying to users, even though we know it was scoped to a single bundle.
Contributor
There was a problem hiding this comment.
but it's empty and when it was not empty it had children such as 'dev', 'prod' without any other context. So no harm in deleting it when last target is gone.
Contributor
Author
There was a problem hiding this comment.
I get that. I'd just be annoyed as a user if you started cleaning up my directories though. No strong feelings either ways for me though.
- Ignored release of the bundle directory logs at Info, the highest level that stays hidden, so it lines up with -q/-v once log levels are wired up. - The "directory is gone" assertions use musterr, so the test fails if get-status ever succeeds instead of merely tolerating the error. - endsWithNameAndTarget matches the suffix as a plain string. root_path is not interpolated yet, so path.Base/path.Dir on it are not dependable. Co-authored-by: Isaac
Collaborator
Integration test reportCommit: a387244
83 interesting tests: 46 FAIL, 25 MISS, 7 flaky, 2 KNOWN, 2 SKIP, 1 RECOVERED
Top 50 slowest tests (at least 2 minutes):
|
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Aug 26, 2026
## Release v1.14.0 ### Notable Changes * Bundles still on Terraform state are now migrated to the direct engine automatically, after a deploy whose dry-run conversion comes back clean. To opt out, set `engine: terraform` under `bundle` or `DATABRICKS_BUNDLE_ENGINE=terraform`. See https://docs.databricks.com/aws/en/dev-tools/bundles/direct ### CLI * Fixed idle `databricks ssh connect` sessions disconnecting after a few minutes, on dedicated clusters and on serverless. The tunnel now keeps itself warm: the SSH client and the SSH server on the compute exchange keepalives every 30 seconds, and the CLI's proxy pings the tunnel's websocket every 20 seconds. A session nobody is typing into stays connected, with no need to set `ServerAliveInterval` by hand. ### Bundles * `bundle plan` no longer reports a permanent, unconvergeable update on `securable_kind` for Unity Catalog `TABLE` securables declared under an app's `resources`. The field is computed by the backend (output-only), so it is now ignored during drift detection. Direct engine only ([#6342](#6342)). * Add support for the `cluster_policies` resource type in Declarative Automation Bundles. Cluster policies are only supported in direct deployment mode. * `bundle destroy` no longer counts resources that are already gone remotely in its `Destroy: N deleted` summary. Such deletes only clean up stale state and are not listed under the deletion prompt, so they are now excluded from the count as well, matching the terraform engine. * `bundle destroy` now also removes the directory named after the bundle, not just the target directory beneath it, when `workspace.root_path` ends in `${bundle.name}/${bundle.target}` — which includes the default root path. It is removed non-recursively, so it stays in place while another target of the same bundle is still deployed there. Previously every destroy left an empty directory behind ([#6317](#6317)). * `job_runs` now plans skip for a run that is still in progress and when `on_bundle_deploy` is removed, instead of treating either case as an update. ([#6357](#6357)) * Added `DATABRICKS_BUNDLE_RESOURCE_MAX_WAIT` (in seconds) to cap how long deploy and destroy wait for a resource. Direct engine only. * Fixed `bundle deploy` on the direct engine failing with `Nothing to update`, or reporting a change it never applied, when a schema, catalog, volume, registered model or external location field was removed from the configuration or set outside the bundle. The update now sends the fields the plan reports as cleared instead of dropping them. ### Dependency Updates * Bump `github.com/databricks/databricks-sdk-go` from v0.171.0 to v0.175.0 ([#6322](#6322)). * Bump Terraform provider from v1.127.0 to v1.128.0 ([#6323](#6323)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Performs additional cleanup on bundle destroy to reduce the number of treenodes we leave around.
Workspace team reported that they were seeing DB issues because of the load we were creating initially because of the number of treenodes our tests were leaving behind.