Terraform in Azure DevOps Pipelines
Run Terraform in Azure DevOps Pipelines with multi-environment stages, plan approvals, state management, and drift detection
Terraform in Azure DevOps Pipelines
Running Terraform locally works fine until your team grows past two people and someone accidentally applies a plan that overwrites someone else's changes. Azure DevOps Pipelines give you a structured, auditable workflow for Terraform that enforces plan reviews, manages state safely, and deploys infrastructure across environments with proper gates. This article walks through every piece of that pipeline, from backend configuration to drift detection.
Prerequisites
Before you start, make sure you have the following in place:
- An Azure DevOps organization and project
- An Azure subscription with Owner or Contributor access
- An Azure Storage Account for Terraform remote state
- Basic familiarity with Terraform HCL syntax
- Basic familiarity with YAML pipelines in Azure DevOps
- The Azure CLI installed locally (for initial setup)
- A Git repository in Azure Repos or GitHub connected to your project
Installing Terraform in Pipelines
Azure DevOps hosted agents do not ship with Terraform pre-installed. You have two options: use the Terraform extension from the Visual Studio Marketplace, or install the binary yourself.
Manual Installation
The simplest approach is to download and install the Terraform binary during your pipeline run. This gives you full control over versioning and avoids any dependency on third-party extensions.
steps:
- task: Bash@3
displayName: 'Install Terraform'
inputs:
targetType: 'inline'
script: |
TERRAFORM_VERSION="1.7.3"
curl -fsSL "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" -o terraform.zip
unzip -o terraform.zip -d /usr/local/bin/
terraform version
Pin the version. Never use latest. Terraform upgrades can introduce breaking changes to providers and state files, and you want every pipeline run to behave identically.
Using the Terraform Extension
The Microsoft DevLabs Terraform extension is the more popular route. Install it from the Visual Studio Marketplace. It provides three tasks:
TerraformInstaller— installs a specific Terraform versionTerraformTaskV4— runs Terraform commands with built-in service connection support- Automatic working directory handling
steps:
- task: TerraformInstaller@1
displayName: 'Install Terraform 1.7.3'
inputs:
terraformVersion: '1.7.3'
The extension handles path configuration and makes the terraform binary available to all subsequent steps. I prefer the extension for teams because it standardizes the installation and integrates cleanly with Azure service connections.
Backend Configuration for Azure
Terraform state must live in a shared, locked backend. For Azure, that means an Azure Storage Account with blob storage. Never store state in your pipeline workspace or commit it to version control.
Creating the Storage Account
Run this once from your local machine or a setup pipeline:
#!/bin/bash
RESOURCE_GROUP="rg-terraform-state"
STORAGE_ACCOUNT="stterraformstate$(openssl rand -hex 4)"
CONTAINER_NAME="tfstate"
LOCATION="eastus2"
az group create \
--name $RESOURCE_GROUP \
--location $LOCATION
az storage account create \
--name $STORAGE_ACCOUNT \
--resource-group $RESOURCE_GROUP \
--location $LOCATION \
--sku Standard_LRS \
--kind StorageV2 \
--min-tls-version TLS1_2 \
--allow-blob-public-access false
az storage container create \
--name $CONTAINER_NAME \
--account-name $STORAGE_ACCOUNT
# Enable versioning for state recovery
az storage account blob-service-properties update \
--account-name $STORAGE_ACCOUNT \
--resource-group $RESOURCE_GROUP \
--enable-versioning true
Enable versioning on the storage account. If someone corrupts the state file, you can roll back to a previous version without panic.
Terraform Backend Block
In your Terraform configuration, declare the backend:
terraform {
backend "azurerm" {
resource_group_name = "rg-terraform-state"
storage_account_name = "stterraformstateab12"
container_name = "tfstate"
key = "dev.terraform.tfstate"
}
}
The key parameter is your state file name. Use a naming convention that includes the environment: dev.terraform.tfstate, staging.terraform.tfstate, prod.terraform.tfstate. This keeps state files isolated per environment within the same container, or you can use separate containers per environment if you prefer harder boundaries.
Service Connection Setup
Terraform needs credentials to interact with Azure. The right way to provide these in Azure DevOps is through a service connection.
- Go to Project Settings > Service connections > New service connection
- Select Azure Resource Manager
- Choose Service principal (automatic) — this creates an app registration and assigns Contributor role
- Name it something meaningful like
azure-terraform-dev - Scope it to the subscription or resource group level
For the Terraform extension, the service connection is passed directly into the task. For manual Terraform runs, you export the credentials as environment variables:
steps:
- task: AzureCLI@2
displayName: 'Terraform Plan'
inputs:
azureSubscription: 'azure-terraform-dev'
scriptType: 'bash'
scriptLocation: 'inlineScript'
addSpnToEnvironment: true
inlineScript: |
export ARM_CLIENT_ID=$servicePrincipalId
export ARM_CLIENT_SECRET=$servicePrincipalKey
export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
export ARM_TENANT_ID=$tenantId
terraform plan -out=tfplan
The addSpnToEnvironment: true flag is critical. Without it, the service principal credentials are not exposed as variables and Terraform cannot authenticate.
Pipeline Stages: Init, Validate, Plan, Apply
A well-structured Terraform pipeline separates concerns into distinct stages. Each stage has a specific job and clear success criteria.
Init
Initialization downloads providers, configures the backend, and sets up the working directory:
- task: TerraformTaskV4@4
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
backendServiceArm: 'azure-terraform-dev'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'dev.terraform.tfstate'
Validate
Validation checks syntax and internal consistency without accessing any remote state or APIs:
- task: TerraformTaskV4@4
displayName: 'Terraform Validate'
inputs:
provider: 'azurerm'
command: 'validate'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
Run validate in your CI pipeline on every pull request. It catches typos and configuration errors before anyone reviews the code.
Plan
The plan stage is where Terraform calculates what changes it will make. Always output the plan to a file:
- task: TerraformTaskV4@4
displayName: 'Terraform Plan'
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
environmentServiceNameAzureRM: 'azure-terraform-dev'
commandOptions: '-out=$(Build.ArtifactStagingDirectory)/tfplan'
Apply
Apply executes the saved plan. Never run terraform apply without the -auto-approve flag in a pipeline, and never run it without a saved plan file. The combination ensures you apply exactly what was reviewed:
- task: TerraformTaskV4@4
displayName: 'Terraform Apply'
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
environmentServiceNameAzureRM: 'azure-terraform-dev'
commandOptions: '$(Pipeline.Workspace)/plan/tfplan'
Plan Artifacts and Approval Gates
The plan file is a binary artifact. Publishing it between stages ensures the apply stage runs the exact plan that was reviewed, not a new plan that might include changes someone pushed in between.
Publishing the Plan
- task: PublishPipelineArtifact@1
displayName: 'Publish Terraform Plan'
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/tfplan'
artifact: 'plan'
publishLocation: 'pipeline'
Consuming the Plan in Apply Stage
- task: DownloadPipelineArtifact@2
displayName: 'Download Terraform Plan'
inputs:
buildType: 'current'
artifactName: 'plan'
targetPath: '$(Pipeline.Workspace)/plan'
Approval Gates
Azure DevOps Environments provide approval gates. Create an environment for each deployment target:
- Go to Pipelines > Environments > New Environment
- Name it
terraform-prod - Under Approvals and checks, add an approval with the required reviewers
In your pipeline, reference the environment in the deployment job:
- stage: Apply_Prod
dependsOn: Plan_Prod
jobs:
- deployment: ApplyInfra
environment: 'terraform-prod'
strategy:
runOnce:
deploy:
steps:
- download: current
artifact: plan-prod
- task: TerraformInstaller@1
inputs:
terraformVersion: '1.7.3'
- task: TerraformTaskV4@4
displayName: 'Terraform Apply'
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
environmentServiceNameAzureRM: 'azure-terraform-prod'
commandOptions: '$(Pipeline.Workspace)/plan-prod/tfplan'
The pipeline will pause at the Apply stage and wait for the designated approvers to review and approve before proceeding.
Multi-Environment Deployment
Real projects deploy to dev, staging, and production. Use pipeline parameters and variable groups to drive environment-specific configuration.
Variable Groups
Create a variable group per environment in Azure DevOps Library:
terraform-vars-dev— containsenvironment=dev,resource_group_name=rg-app-devterraform-vars-staging— containsenvironment=staging,resource_group_name=rg-app-stagingterraform-vars-prod— containsenvironment=prod,resource_group_name=rg-app-prod
Template-Based Stages
Extract the Terraform stage into a template to avoid duplicating YAML:
# templates/terraform-stage.yml
parameters:
- name: environment
type: string
- name: serviceConnection
type: string
- name: stateKey
type: string
- name: variableGroup
type: string
- name: dependsOn
type: string
default: ''
stages:
- stage: Plan_${{ parameters.environment }}
dependsOn: ${{ parameters.dependsOn }}
variables:
- group: ${{ parameters.variableGroup }}
jobs:
- job: Plan
pool:
vmImage: 'ubuntu-latest'
steps:
- task: TerraformInstaller@1
inputs:
terraformVersion: '1.7.3'
- task: TerraformTaskV4@4
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
backendServiceArm: '${{ parameters.serviceConnection }}'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: '${{ parameters.stateKey }}'
- task: TerraformTaskV4@4
displayName: 'Terraform Plan'
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
commandOptions: '-var="environment=$(environment)" -out=$(Build.ArtifactStagingDirectory)/tfplan'
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/tfplan'
artifact: 'plan-${{ parameters.environment }}'
- stage: Apply_${{ parameters.environment }}
dependsOn: Plan_${{ parameters.environment }}
jobs:
- deployment: Apply
environment: 'terraform-${{ parameters.environment }}'
pool:
vmImage: 'ubuntu-latest'
strategy:
runOnce:
deploy:
steps:
- task: TerraformInstaller@1
inputs:
terraformVersion: '1.7.3'
- task: TerraformTaskV4@4
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
backendServiceArm: '${{ parameters.serviceConnection }}'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: '${{ parameters.stateKey }}'
- task: TerraformTaskV4@4
displayName: 'Terraform Apply'
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
commandOptions: '$(Pipeline.Workspace)/plan-${{ parameters.environment }}/tfplan'
Then your main pipeline becomes clean and concise:
# azure-pipelines.yml
trigger:
branches:
include:
- main
stages:
- template: templates/terraform-stage.yml
parameters:
environment: 'dev'
serviceConnection: 'azure-terraform-dev'
stateKey: 'dev.terraform.tfstate'
variableGroup: 'terraform-vars-dev'
- template: templates/terraform-stage.yml
parameters:
environment: 'staging'
serviceConnection: 'azure-terraform-staging'
stateKey: 'staging.terraform.tfstate'
variableGroup: 'terraform-vars-staging'
dependsOn: 'Apply_dev'
- template: templates/terraform-stage.yml
parameters:
environment: 'prod'
serviceConnection: 'azure-terraform-prod'
stateKey: 'prod.terraform.tfstate'
variableGroup: 'terraform-vars-prod'
dependsOn: 'Apply_staging'
State File Management
State is the single most important artifact in your Terraform workflow. Treat it accordingly.
Locking: Azure Storage supports blob leasing, which Terraform uses for state locking. This prevents two pipeline runs from modifying state simultaneously. The azurerm backend handles this automatically.
Encryption: Azure Storage encrypts blobs at rest by default. For additional security, enable customer-managed keys through Azure Key Vault.
Backup: Enable blob versioning on the storage account (shown in the setup script above). This gives you automatic point-in-time recovery without maintaining separate backup jobs.
Access control: Lock down the storage account. Only the service principals used by your pipelines should have access. Use Azure RBAC with the Storage Blob Data Contributor role instead of storage account keys.
# Assign RBAC to the service principal instead of using storage keys
az role assignment create \
--role "Storage Blob Data Contributor" \
--assignee <service-principal-object-id> \
--scope /subscriptions/<sub-id>/resourceGroups/rg-terraform-state/providers/Microsoft.Storage/storageAccounts/stterraformstateab12
Then configure the backend to use Azure AD authentication:
terraform {
backend "azurerm" {
resource_group_name = "rg-terraform-state"
storage_account_name = "stterraformstateab12"
container_name = "tfstate"
key = "dev.terraform.tfstate"
use_oidc = true
}
}
Secret Injection from Key Vault
Never hardcode secrets in Terraform variables or pipeline YAML. Use Azure Key Vault and the AzureKeyVault task:
steps:
- task: AzureKeyVault@2
displayName: 'Fetch Secrets from Key Vault'
inputs:
azureSubscription: 'azure-terraform-dev'
KeyVaultName: 'kv-terraform-secrets'
SecretsFilter: 'db-admin-password,api-key-external'
RunAsPreJob: false
- task: TerraformTaskV4@4
displayName: 'Terraform Plan'
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
environmentServiceNameAzureRM: 'azure-terraform-dev'
commandOptions: '-var="db_password=$(db-admin-password)" -var="api_key=$(api-key-external)" -out=$(Build.ArtifactStagingDirectory)/tfplan'
The Key Vault task fetches secrets and maps them to pipeline variables automatically. They are masked in logs. Combine this with Terraform's sensitive variable flag:
variable "db_password" {
type = string
sensitive = true
}
variable "api_key" {
type = string
sensitive = true
}
Terraform Output Consumption in Subsequent Stages
Terraform outputs are critical for passing infrastructure details to application deployment stages. For example, after Terraform provisions an App Service, you need its hostname for your deployment step.
Capturing Outputs
- task: AzureCLI@2
name: terraform_outputs
displayName: 'Capture Terraform Outputs'
inputs:
azureSubscription: 'azure-terraform-dev'
scriptType: 'bash'
addSpnToEnvironment: true
scriptLocation: 'inlineScript'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
inlineScript: |
export ARM_CLIENT_ID=$servicePrincipalId
export ARM_CLIENT_SECRET=$servicePrincipalKey
export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
export ARM_TENANT_ID=$tenantId
APP_SERVICE_NAME=$(terraform output -raw app_service_name)
RESOURCE_GROUP=$(terraform output -raw resource_group_name)
echo "##vso[task.setvariable variable=appServiceName;isOutput=true]$APP_SERVICE_NAME"
echo "##vso[task.setvariable variable=resourceGroupName;isOutput=true]$RESOURCE_GROUP"
Using Outputs in Later Stages
Reference the output variables in subsequent jobs using the stageDependencies syntax:
- stage: DeployApp
dependsOn: Apply_dev
variables:
appServiceName: $[ stageDependencies.Apply_dev.Apply.outputs['Apply.terraform_outputs.appServiceName'] ]
resourceGroupName: $[ stageDependencies.Apply_dev.Apply.outputs['Apply.terraform_outputs.resourceGroupName'] ]
jobs:
- job: Deploy
steps:
- task: AzureWebApp@1
inputs:
azureSubscription: 'azure-terraform-dev'
appName: '$(appServiceName)'
resourceGroupName: '$(resourceGroupName)'
package: '$(Pipeline.Workspace)/app/*.zip'
Drift Detection Pipelines
Infrastructure drift is when the actual state of your cloud resources diverges from what Terraform expects. Someone makes a manual change in the Azure portal, and suddenly your next terraform apply does something unexpected.
Build a scheduled pipeline that runs terraform plan and alerts you if there are changes:
# drift-detection.yml
trigger: none
schedules:
- cron: '0 6 * * *'
displayName: 'Daily Drift Detection - 6 AM UTC'
branches:
include:
- main
always: true
pool:
vmImage: 'ubuntu-latest'
steps:
- task: TerraformInstaller@1
inputs:
terraformVersion: '1.7.3'
- task: TerraformTaskV4@4
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
backendServiceArm: 'azure-terraform-prod'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'prod.terraform.tfstate'
- task: AzureCLI@2
displayName: 'Detect Drift'
inputs:
azureSubscription: 'azure-terraform-prod'
scriptType: 'bash'
addSpnToEnvironment: true
scriptLocation: 'inlineScript'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
inlineScript: |
export ARM_CLIENT_ID=$servicePrincipalId
export ARM_CLIENT_SECRET=$servicePrincipalKey
export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
export ARM_TENANT_ID=$tenantId
terraform plan -detailed-exitcode -out=drift-plan 2>&1 | tee plan-output.txt
EXIT_CODE=${PIPESTATUS[0]}
if [ $EXIT_CODE -eq 2 ]; then
echo "##vso[task.logissue type=warning]Infrastructure drift detected!"
echo "##vso[task.setvariable variable=driftDetected]true"
# Show what drifted
terraform show drift-plan
elif [ $EXIT_CODE -eq 0 ]; then
echo "No drift detected. Infrastructure matches configuration."
echo "##vso[task.setvariable variable=driftDetected]false"
else
echo "##vso[task.logissue type=error]Terraform plan failed!"
exit 1
fi
- task: Bash@3
displayName: 'Notify on Drift'
condition: eq(variables['driftDetected'], 'true')
inputs:
targetType: 'inline'
script: |
# Post to Slack, Teams, or create a work item
curl -X POST "$(TEAMS_WEBHOOK_URL)" \
-H "Content-Type: application/json" \
-d '{"text": "Infrastructure drift detected in production! Review the latest drift detection pipeline run."}'
The key is the -detailed-exitcode flag. Exit code 0 means no changes, exit code 2 means changes detected, and exit code 1 means an error. This lets you differentiate between a successful "no drift" run and a pipeline failure.
Modular Terraform in Pipelines
When your Terraform codebase grows, you will split it into modules. This affects your pipeline in a few ways.
Local Modules
If your modules live in the same repository, no special configuration is needed. Terraform init handles local module paths automatically:
module "networking" {
source = "./modules/networking"
environment = var.environment
address_space = var.vnet_address_space
}
module "compute" {
source = "./modules/compute"
environment = var.environment
subnet_id = module.networking.subnet_id
vm_size = var.vm_size
}
Private Module Registry
If you use a private Terraform module registry or pull modules from a private Git repository, you need to configure authentication in your pipeline:
steps:
- task: Bash@3
displayName: 'Configure Git Credentials for Modules'
inputs:
targetType: 'inline'
script: |
git config --global url."https://$(System.AccessToken)@dev.azure.com".insteadOf "https://dev.azure.com"
This rewrites Git URLs to include the pipeline's access token, allowing Terraform to clone private module repositories during init.
Caching Terraform Providers
Provider downloads can add 30-60 seconds to every pipeline run. Caching eliminates this overhead:
variables:
TF_PLUGIN_CACHE_DIR: $(Pipeline.Workspace)/.terraform-plugin-cache
steps:
- task: Cache@2
displayName: 'Cache Terraform Providers'
inputs:
key: 'terraform | "$(Agent.OS)" | $(System.DefaultWorkingDirectory)/infra/.terraform.lock.hcl'
path: '$(TF_PLUGIN_CACHE_DIR)'
cacheHitVar: 'CACHE_RESTORED'
- task: Bash@3
displayName: 'Create Plugin Cache Directory'
inputs:
targetType: 'inline'
script: mkdir -p $(TF_PLUGIN_CACHE_DIR)
- task: TerraformInstaller@1
inputs:
terraformVersion: '1.7.3'
- task: TerraformTaskV4@4
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
backendServiceArm: 'azure-terraform-dev'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'dev.terraform.tfstate'
env:
TF_PLUGIN_CACHE_DIR: $(TF_PLUGIN_CACHE_DIR)
The cache key is derived from .terraform.lock.hcl, which changes only when provider versions change. On cache hit, Terraform init skips provider downloads entirely.
Complete Working Example
Here is a complete multi-stage Azure Pipeline that ties everything together. It handles three environments with plan approvals, artifact passing, provider caching, and output consumption.
# azure-pipelines.yml
trigger:
branches:
include:
- main
paths:
include:
- infra/**
variables:
terraformVersion: '1.7.3'
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
TF_PLUGIN_CACHE_DIR: '$(Pipeline.Workspace)/.terraform-plugin-cache'
stages:
# ──────────────────────────────────
# DEV - Plan & Apply (auto-approve)
# ──────────────────────────────────
- stage: Plan_Dev
displayName: 'Plan - Dev'
pool:
vmImage: 'ubuntu-latest'
variables:
- group: terraform-vars-dev
jobs:
- job: Plan
steps:
- task: Cache@2
displayName: 'Cache Terraform Providers'
inputs:
key: 'terraform | "$(Agent.OS)" | $(workingDirectory)/.terraform.lock.hcl'
path: '$(TF_PLUGIN_CACHE_DIR)'
- task: Bash@3
displayName: 'Create Cache Dir'
inputs:
targetType: 'inline'
script: mkdir -p $(TF_PLUGIN_CACHE_DIR)
- task: TerraformInstaller@1
inputs:
terraformVersion: '$(terraformVersion)'
- task: TerraformTaskV4@4
displayName: 'Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDirectory)'
backendServiceArm: 'azure-terraform-dev'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'dev.terraform.tfstate'
env:
TF_PLUGIN_CACHE_DIR: $(TF_PLUGIN_CACHE_DIR)
- task: TerraformTaskV4@4
displayName: 'Validate'
inputs:
provider: 'azurerm'
command: 'validate'
workingDirectory: '$(workingDirectory)'
- task: AzureKeyVault@2
displayName: 'Fetch Secrets'
inputs:
azureSubscription: 'azure-terraform-dev'
KeyVaultName: 'kv-terraform-dev'
SecretsFilter: 'db-admin-password'
- task: TerraformTaskV4@4
displayName: 'Plan'
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(workingDirectory)'
environmentServiceNameAzureRM: 'azure-terraform-dev'
commandOptions: >-
-var="environment=$(environment)"
-var="db_password=$(db-admin-password)"
-out=$(Build.ArtifactStagingDirectory)/tfplan
- task: PublishPipelineArtifact@1
displayName: 'Publish Plan'
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/tfplan'
artifact: 'plan-dev'
- stage: Apply_Dev
displayName: 'Apply - Dev'
dependsOn: Plan_Dev
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: Apply
environment: 'terraform-dev'
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: TerraformInstaller@1
inputs:
terraformVersion: '$(terraformVersion)'
- task: TerraformTaskV4@4
displayName: 'Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDirectory)'
backendServiceArm: 'azure-terraform-dev'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'dev.terraform.tfstate'
- task: TerraformTaskV4@4
displayName: 'Apply'
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(workingDirectory)'
environmentServiceNameAzureRM: 'azure-terraform-dev'
commandOptions: '$(Pipeline.Workspace)/plan-dev/tfplan'
- task: AzureCLI@2
name: outputs
displayName: 'Capture Outputs'
inputs:
azureSubscription: 'azure-terraform-dev'
scriptType: 'bash'
addSpnToEnvironment: true
workingDirectory: '$(workingDirectory)'
scriptLocation: 'inlineScript'
inlineScript: |
export ARM_CLIENT_ID=$servicePrincipalId
export ARM_CLIENT_SECRET=$servicePrincipalKey
export ARM_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
export ARM_TENANT_ID=$tenantId
echo "##vso[task.setvariable variable=appServiceName;isOutput=true]$(terraform output -raw app_service_name)"
# ──────────────────────────────────
# STAGING - Plan & Apply (approval)
# ──────────────────────────────────
- stage: Plan_Staging
displayName: 'Plan - Staging'
dependsOn: Apply_Dev
pool:
vmImage: 'ubuntu-latest'
variables:
- group: terraform-vars-staging
jobs:
- job: Plan
steps:
- task: TerraformInstaller@1
inputs:
terraformVersion: '$(terraformVersion)'
- task: TerraformTaskV4@4
displayName: 'Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDirectory)'
backendServiceArm: 'azure-terraform-staging'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'staging.terraform.tfstate'
- task: TerraformTaskV4@4
displayName: 'Plan'
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(workingDirectory)'
environmentServiceNameAzureRM: 'azure-terraform-staging'
commandOptions: '-var="environment=$(environment)" -out=$(Build.ArtifactStagingDirectory)/tfplan'
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/tfplan'
artifact: 'plan-staging'
- stage: Apply_Staging
displayName: 'Apply - Staging'
dependsOn: Plan_Staging
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: Apply
environment: 'terraform-staging'
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: TerraformInstaller@1
inputs:
terraformVersion: '$(terraformVersion)'
- task: TerraformTaskV4@4
displayName: 'Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDirectory)'
backendServiceArm: 'azure-terraform-staging'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'staging.terraform.tfstate'
- task: TerraformTaskV4@4
displayName: 'Apply'
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(workingDirectory)'
environmentServiceNameAzureRM: 'azure-terraform-staging'
commandOptions: '$(Pipeline.Workspace)/plan-staging/tfplan'
# ──────────────────────────────────
# PROD - Plan & Apply (approval)
# ──────────────────────────────────
- stage: Plan_Prod
displayName: 'Plan - Prod'
dependsOn: Apply_Staging
pool:
vmImage: 'ubuntu-latest'
variables:
- group: terraform-vars-prod
jobs:
- job: Plan
steps:
- task: TerraformInstaller@1
inputs:
terraformVersion: '$(terraformVersion)'
- task: TerraformTaskV4@4
displayName: 'Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDirectory)'
backendServiceArm: 'azure-terraform-prod'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'prod.terraform.tfstate'
- task: TerraformTaskV4@4
displayName: 'Plan'
inputs:
provider: 'azurerm'
command: 'plan'
workingDirectory: '$(workingDirectory)'
environmentServiceNameAzureRM: 'azure-terraform-prod'
commandOptions: '-var="environment=$(environment)" -out=$(Build.ArtifactStagingDirectory)/tfplan'
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/tfplan'
artifact: 'plan-prod'
- stage: Apply_Prod
displayName: 'Apply - Prod'
dependsOn: Plan_Prod
pool:
vmImage: 'ubuntu-latest'
jobs:
- deployment: Apply
environment: 'terraform-prod'
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: TerraformInstaller@1
inputs:
terraformVersion: '$(terraformVersion)'
- task: TerraformTaskV4@4
displayName: 'Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: '$(workingDirectory)'
backendServiceArm: 'azure-terraform-prod'
backendAzureRmResourceGroupName: 'rg-terraform-state'
backendAzureRmStorageAccountName: 'stterraformstateab12'
backendAzureRmContainerName: 'tfstate'
backendAzureRmKey: 'prod.terraform.tfstate'
- task: TerraformTaskV4@4
displayName: 'Apply'
inputs:
provider: 'azurerm'
command: 'apply'
workingDirectory: '$(workingDirectory)'
environmentServiceNameAzureRM: 'azure-terraform-prod'
commandOptions: '$(Pipeline.Workspace)/plan-prod/tfplan'
This pipeline triggers only when files in the infra/ directory change, flows sequentially through dev, staging, and production, publishes plan artifacts for each environment, and uses Azure DevOps environments with approval gates to control when applies happen.
Common Issues and Troubleshooting
1. State Lock Timeout
Symptom: Error acquiring the state lock after a pipeline run was cancelled mid-apply.
Fix: You need to manually break the lease on the state blob. Do not force-unlock in a pipeline.
az storage blob lease break \
--blob-name dev.terraform.tfstate \
--container-name tfstate \
--account-name stterraformstateab12
Then investigate what the cancelled run was doing. The state may be in an inconsistent state.
2. Service Principal Permission Errors
Symptom: AuthorizationFailed: The client does not have authorization to perform action during plan or apply.
Fix: The service principal needs the correct RBAC roles. Contributor is the minimum for most resources, but some operations (like assigning roles or managing Key Vault access policies) require Owner or specific roles. Check the Terraform documentation for the resource you are creating to see which permissions are needed.
az role assignment create \
--role "Contributor" \
--assignee <sp-object-id> \
--scope /subscriptions/<subscription-id>
3. Plan File Incompatibility Between Stages
Symptom: Error: saved plan is stale when applying a plan artifact in a different stage.
Cause: The plan stage and apply stage are using different Terraform versions, different provider versions, or the source code changed between stages. This also happens if you re-run just the apply stage after pushing new commits.
Fix: Ensure both stages use the same Terraform version, the same checked-out code, and the plan was generated in the same pipeline run. If you need to re-apply, re-run the entire pipeline from the plan stage.
4. Backend Initialization Fails in Apply Stage
Symptom: Error: Backend initialization required, please run "terraform init" in the apply stage.
Cause: Deployment jobs in Azure DevOps run with a clean workspace. The .terraform directory from the plan stage does not carry over.
Fix: Always run terraform init in every stage, even if you already ran it in the plan stage. The init command is idempotent and fast when providers are cached.
5. Terraform Output Not Available in Downstream Stages
Symptom: Output variables from Terraform are empty in subsequent pipeline stages.
Cause: The variable was set with isOutput=true but the reference syntax is wrong. Azure DevOps has specific syntax for referencing output variables across stages and jobs.
Fix: Use the correct reference format: $[ stageDependencies.StageName.JobName.outputs['StepName.variableName'] ]. For deployment jobs, the job name is always the deployment name, not the strategy step.
Best Practices
Pin every version. Pin the Terraform version, pin provider versions in your
.terraform.lock.hcl, and pin the extension task versions in your YAML. Unpinned versions will break your pipeline on a random Tuesday morning.Use separate service connections per environment. A dev service principal should not have access to production resources. Create one service connection per environment and scope each to the appropriate subscription or resource group.
Never skip the plan artifact. Always output the plan to a file in the plan stage and consume that exact file in the apply stage. Running
terraform applywithout a saved plan means you might apply changes that were not reviewed.Run validate on pull requests. Create a separate, lightweight pipeline that runs
terraform initandterraform validateon every pull request. This catches syntax errors before they reach your main branch.Enable pipeline concurrency controls. Set
lockBehavior: sequentialon your environments to prevent two pipeline runs from applying to the same environment simultaneously. Terraform state locking helps, but sequential runs are safer.Keep state files small. If your Terraform configuration manages hundreds of resources, split it into multiple state files by domain (networking, compute, data). Large state files slow down every plan and increase the blast radius of mistakes.
Use
-compact-warningsin CI. Terraform can produce a lot of deprecation warnings that clutter your pipeline logs. The-compact-warningsflag condenses them into a summary.Tag your infrastructure. Always include a
managed_by = "terraform"tag on every resource. This makes it easy to identify which resources are under Terraform control and which were created manually.Rotate service principal credentials. Set a reminder to rotate the client secrets for your Terraform service principals. Expired credentials will silently fail your pipelines.
