feat: store Terraform plans securely between pipeline stages - #146
Open
Git-PrinceNagar wants to merge 4 commits into
Open
feat: store Terraform plans securely between pipeline stages#146Git-PrinceNagar wants to merge 4 commits into
Git-PrinceNagar wants to merge 4 commits into
Conversation
…tifacts Ports the same fix already implemented and runtime-proven in terraform-azure-avm-ptn-alz-application-landing-zone-cicd-bootstrap-github into this repository, which is the actual entry point reached by Deploy-Accelerator (the AVM module is not invoked by Deploy-Accelerator in any mode). - modules/azure: add an optional plan storage container with container-scoped RBAC (Storage Blob Data Contributor for the plan and apply identities plus any additional principals) and a lifecycle management policy that deletes base blobs, snapshots, and versions after a configurable retention period (default 7 days). - alz/github: add use_storage_account_for_plan, show_plan_in_pipeline_logs, and plan_storage_retention_days inputs; compute a deterministic, length-bounded plan storage container name. - modules/github: publish the new flags and computed container name as GitHub Actions repository variables (USE_STORAGE_ACCOUNT_FOR_PLAN, SHOW_PLAN_IN_PIPELINE_LOGS, PLAN_STORAGE_CONTAINER_NAME), read at workflow runtime rather than threaded through templatefile() args. - cd-template.yaml: upload the plan to Blob Storage with bounded retry and content-length validation, download the exact run-scoped blob in the apply job (fail-closed, no listing/latest fallback), and delete it only after a successful apply. Falls back to the legacy artifact path when the new flag is off. Gate full plan-log printing behind SHOW_PLAN_IN_PIPELINE_LOGS, including on the failure path. - ci-template.yaml: gate full plan output behind the same flag. All new modules/azure inputs default to false/null so the untouched Azure DevOps caller (alz/azuredevops/main.tf) is unaffected. Validated: terraform fmt -check -recursive, terraform validate on both alz/github and alz/azuredevops, YAML syntax on both rewritten templates, and a full audit of \$ vs \$\$ templatefile() escaping (75/75 GitHub Actions expression sites correct). Fixed one real schema bug found by validate: azurerm_storage_management_policy's version block takes delete_after_days_since_creation, not delete_after_days_since_creation_greater_than. Not yet done: Azure DevOps pipeline templates (alz/azuredevops), and real Deploy-Accelerator runtime validation. Refs: Azure/Azure-Landing-Zones#4174 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…h steps The 'Show the Plan for Review' and 'Terraform Apply' steps were changed to use 'shell: pwsh' to support the new plan-storage conditional logic, but their terraform CLI invocations still used bash-style backslash line continuation, which PowerShell does not support. This broke terraform apply/show on self-hosted Linux runners (default shell is bash, but an explicit shell: pwsh switches parsing to PowerShell). Switched both to an array + call-operator invocation, matching the pattern already used in the 'Terraform Plan for Apply' step.
…d of pipeline artifacts
Ports the fix already implemented and live-proven on the GitHub side of this
repository across to the Azure DevOps bootstrap module, closing the same
plan-file exposure on the second supported version control system.
Three distinct leaks existed on the Azure DevOps side:
- terraform-plan.yaml wrote `-out=tfplan` into $(Build.SourcesDirectory), and
cd-template.yaml's CopyFiles@2 copied `**/*` from that directory into the
published pipeline artifact. Anyone with pipeline read access could download
the plan, which embeds resolved resource attributes.
- "Show the Plan for Review" ran `terraform show tfplan` unconditionally, so
the full resource diff was always printed to the pipeline log. No equivalent
of SHOW_PLAN_IN_PIPELINE_LOGS existed anywhere on the Azure DevOps side.
- ci-template.yaml's validation plan was ungated in the same way, exposing the
diff on pull request runs.
Changes:
- alz/azuredevops: add use_storage_account_for_plan (default true),
show_plan_in_pipeline_logs (default false) and plan_storage_retention_days
(default 7, positive-whole-number validated); compute a deterministic,
length-bounded plan storage container name identical to the GitHub side.
- modules/azure_devops: accept the two flags plus the computed container name
and publish them to the pipeline variable group as
USE_STORAGE_ACCOUNT_FOR_PLAN, SHOW_PLAN_IN_PIPELINE_LOGS and (only when the
hand-off is enabled) PLAN_STORAGE_CONTAINER_NAME, read at pipeline runtime
rather than threaded through templatefile() arguments.
- New pipeline helpers terraform-plan-workspace.yaml, terraform-plan-upload.yaml,
terraform-plan-download.yaml, terraform-plan-delete.yaml and
terraform-plan-cleanup.yaml. Upload retries three times with backoff and
verifies the remote content length against the local file; download is
fail-closed with no fallback to a stale or missing plan and rejects a
zero-byte blob; delete runs only after a successful apply and, on repeated
failure, warns and defers to the container lifecycle policy.
- terraform-plan.yaml writes the plan and its captured output into
$(Agent.TempDirectory), so the plan is structurally absent from the sources
directory the artifact is built from. On failure it prints only Terraform's
own boxed diagnostic blocks unless SHOW_PLAN_IN_PIPELINE_LOGS is exactly
'true'.
- cd-template.yaml gates "Show the Plan for Review", excludes tfplan and
tfplan.json from the artifact, uploads and downloads through Blob Storage,
deletes the blob only on apply success, and always cleans the temp directory.
- ci-template.yaml applies the same gating to the validation plan.
The blob key is runs/$(Build.BuildId)/tfplan. $(System.JobAttempt) appears only
in the local temp path and never in the remote key, so a retried job cannot pick
up residue from an earlier attempt.
Two real bugs found and fixed while porting:
- The legacy USE_STORAGE_ACCOUNT_FOR_PLAN=false path copied the plan to the
artifact root, but the apply stage reads `tfplan` relative to
root_module_folder_relative_path because of `terraform -chdir`. Those only
coincide when that path is ".". Both cd-template.yaml files now create the
root-module-relative folder under the staging directory and copy the plan
there, so the fallback works for any root module path.
- cd-template.yaml referenced $(Build.ArtifactsStagingDirectory), which is not
an Azure Pipelines predefined variable, so the macro was never expanded and
the artifact was staged into a literally-named folder. Corrected to
$(Build.ArtifactStagingDirectory), which the new plan-copy step also requires.
Every $(...) macro is confined to task `env:` blocks and task inputs and never
appears inside PowerShell source, so an undefined variable degrades to a
harmless literal string instead of being evaluated as a subexpression.
No change was needed for pipeline serialization. An earlier reading suggested
`lockBehavior: sequential` in cd.yaml was a no-op because environment.tf creates
no Exclusive Lock check, but modules/azure_devops/service_connections.tf already
declares azuredevops_check_exclusive_lock over var.environments targeting the
service endpoints, which every plan and apply stage authenticates through.
Validated: terraform fmt -recursive, terraform validate on both alz/azuredevops
and alz/github, templatefile render plus YAML parse of all 13 Azure DevOps and
4 GitHub pipeline templates, an audit confirming all 22 single-`${}` sites are
genuine templatefile substitutions and every runtime expression uses `$${{`, and
confirmation of Agent.TempDirectory, System.JobAttempt and
Build.ArtifactStagingDirectory against the Azure Pipelines predefined variable
reference.
Not yet done: live Deploy-Accelerator validation against a real Azure DevOps
organization.
Refs: Azure/Azure-Landing-Zones#4174
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Explain that retention must exceed the expected plan-to-apply approval wait and that expired plans should be regenerated. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
github-project-automation
Bot
moved this to To triage
in Azure Landing Zone Issues & PRs Triage
This was referenced Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Moves Terraform plan hand-off for both
alz/githubandalz/azuredevopsfrom downloadable CI/CD artifacts to a dedicated container in the private Azure Storage account used for Terraform state.Related to Azure/Azure-Landing-Zones#4174.
Companion production-module PRs:
Security and compatibility
use_storage_account_for_plan = true.runs/<run-id>/tfplanblob using Microsoft Entra ID/OIDC; no storage keys, blob listing, or mutablelatestalias are used.show_plan_in_pipeline_logs = false, including failed-plan output redaction.plan_storage_retention_daysdefaults to 7 and is configurable for longer approval windows.use_storage_account_for_plan = falsepreserves the legacy artifact hand-off.avm-res-storage-storageaccountdependency remains pinned at0.6.8.Implementation
Testing evidence
terraform init -backend=falseandterraform validatepass for bothalz/githubandalz/azuredevops.PR: Safe to test 🧪label and will run when a maintainer adds that label.Known limitations
mock_providerrejects its ephemeralazapi_resource_action(Test framework: add ephemeral resource support tomock_providerhashicorp/terraform#38608). The live matrices, contract suites, static checks, and Azure read-back provide compensating coverage.Type of Change
Checklist
The AVM module pre-commit command is not applicable to this repository. Native fmt, super-linter, title, and release checks pass; the gated E2E matrix awaits the maintainer-applied
PR: Safe to test 🧪label.