Compliance Automation with Azure Policy

Overview

Azure Policy enforces organizational standards across your Azure resources automatically — no manual reviews, no audit checklists, no hoping developers follow the rules. When your Azure DevOps pipelines deploy infrastructure, Azure Policy evaluates every resource against your compliance rules and blocks non-compliant deployments before they reach production. I use Azure Policy as the enforcement layer for every compliance requirement that can be expressed as a rule: required tags, allowed VM sizes, network restrictions, encryption requirements, and more.

Prerequisites

• An Azure subscription with Owner or Policy Contributor permissions
• Azure DevOps organization with pipelines that deploy Azure resources
• Azure CLI installed locally
• Node.js 16 or later for automation scripts
• Basic understanding of Azure Resource Manager (ARM) templates or Terraform
• Familiarity with JSON policy definition syntax

Understanding Azure Policy

Azure Policy evaluates resources during creation, update, and on a regular compliance scan. Each policy definition contains a rule that checks resource properties against your requirements.

Policy Effects

Effect	Behavior	Use Case
Deny	Blocks non-compliant resource creation/update	Hard enforcement in production
Audit	Logs non-compliance but allows the operation	Monitoring before enforcement
AuditIfNotExists	Checks if a related resource exists	Verify diagnostic settings exist
DeployIfNotExists	Auto-deploys a related resource	Add missing diagnostic settings
Modify	Adds/changes resource properties	Auto-tag resources
Disabled	Policy exists but is not evaluated	Temporary suspension
Append	Adds properties during creation	Add required network rules
DenyAction	Blocks specific actions	Prevent resource deletion

Policy Definition Structure

{
  "properties": {
    "displayName": "Require environment tag on all resources",
    "description": "All resources must have an 'environment' tag with a valid value",
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "anyOf": [
          {
            "field": "tags['environment']",
            "exists": "false"
          },
          {
            "field": "tags['environment']",
            "notIn": ["production", "staging", "development", "test"]
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {}
  }
}

Creating Custom Policies

Required Tags Policy

# policies/require-tags.json
cat > require-tags.json << 'POLICY'
{
  "properties": {
    "displayName": "Require standard tags on all resources",
    "description": "Resources must have environment, team, and cost-center tags",
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "anyOf": [
          { "field": "tags['environment']", "exists": "false" },
          { "field": "tags['team']", "exists": "false" },
          { "field": "tags['cost-center']", "exists": "false" }
        ]
      },
      "then": {
        "effect": "[parameters('effect')]"
      }
    },
    "parameters": {
      "effect": {
        "type": "String",
        "allowedValues": ["Deny", "Audit"],
        "defaultValue": "Audit",
        "metadata": {
          "displayName": "Effect",
          "description": "Deny or Audit when tags are missing"
        }
      }
    }
  }
}
POLICY

# Create the policy definition
az policy definition create \
  --name "require-standard-tags" \
  --display-name "Require standard tags on all resources" \
  --rules require-tags.json \
  --mode Indexed

Restrict Resource Locations

{
  "properties": {
    "displayName": "Restrict resource locations to approved regions",
    "description": "Resources can only be deployed to East US and West US 2",
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "not": {
          "field": "location",
          "in": "[parameters('allowedLocations')]"
        }
      },
      "then": {
        "effect": "deny"
      }
    },
    "parameters": {
      "allowedLocations": {
        "type": "Array",
        "metadata": {
          "displayName": "Allowed locations",
          "description": "The list of Azure regions where resources can be deployed"
        },
        "defaultValue": ["eastus", "westus2"],
        "allowedValues": ["eastus", "eastus2", "westus", "westus2", "centralus"]
      }
    }
  }
}

Enforce HTTPS on Storage Accounts

{
  "properties": {
    "displayName": "Storage accounts must require HTTPS traffic",
    "description": "Blocks creation of storage accounts with HTTP access enabled",
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Storage/storageAccounts"
          },
          {
            "field": "Microsoft.Storage/storageAccounts/supportsHttpsTrafficOnly",
            "notEquals": "true"
          }
        ]
      },
      "then": {
        "effect": "deny"
      }
    }
  }
}

Enforce Encryption on SQL Databases

{
  "properties": {
    "displayName": "SQL databases must use TDE encryption",
    "description": "Transparent Data Encryption must be enabled on all SQL databases",
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Sql/servers/databases"
          },
          {
            "field": "Microsoft.Sql/servers/databases/transparentDataEncryption.status",
            "notEquals": "Enabled"
          }
        ]
      },
      "then": {
        "effect": "auditIfNotExists",
        "details": {
          "type": "Microsoft.Sql/servers/databases/transparentDataEncryption",
          "existenceCondition": {
            "field": "Microsoft.Sql/transparentDataEncryption/status",
            "equals": "Enabled"
          }
        }
      }
    }
  }
}

Policy Initiatives (Policy Sets)

Group related policies into initiatives for easier management:

# Create an initiative definition
az policy set-definition create \
  --name "security-baseline" \
  --display-name "Security Baseline Initiative" \
  --description "Core security requirements for all Azure resources" \
  --definitions '[
    {
      "policyDefinitionId": "/subscriptions/{sub-id}/providers/Microsoft.Authorization/policyDefinitions/require-standard-tags",
      "parameters": {
        "effect": { "value": "Deny" }
      }
    },
    {
      "policyDefinitionId": "/providers/Microsoft.Authorization/policyDefinitions/404c3081-a854-4457-ae30-26a93ef643f9",
      "parameters": {}
    },
    {
      "policyDefinitionId": "/providers/Microsoft.Authorization/policyDefinitions/b2982f36-99f2-4db5-8eff-283140c09693",
      "parameters": {}
    }
  ]'

Assigning Initiatives

# Assign at subscription level
az policy assignment create \
  --name "security-baseline-assignment" \
  --display-name "Security Baseline - Production" \
  --scope "/subscriptions/{sub-id}" \
  --policy-set-definition "security-baseline" \
  --enforcement-mode "Default" \
  --location "eastus" \
  --identity-type "SystemAssigned"

# Assign at resource group level (narrower scope)
az policy assignment create \
  --name "security-baseline-rg-prod" \
  --display-name "Security Baseline - Production RG" \
  --scope "/subscriptions/{sub-id}/resourceGroups/rg-production" \
  --policy-set-definition "security-baseline" \
  --enforcement-mode "Default"

Pipeline Integration

Pre-Deployment Policy Check

Run a compliance check before deploying to catch violations early:

// scripts/check-policy-compliance.js
var cp = require("child_process");

var SUBSCRIPTION = process.env.AZURE_SUBSCRIPTION_ID;
var RESOURCE_GROUP = process.env.RESOURCE_GROUP;

function checkCompliance(callback) {
    var cmd = "az policy state summarize" +
        " --subscription " + SUBSCRIPTION +
        " --resource-group " + RESOURCE_GROUP +
        " --output json";

    cp.exec(cmd, { maxBuffer: 10 * 1024 * 1024 }, function(err, stdout, stderr) {
        if (err) return callback(new Error("Policy check failed: " + stderr));

        try {
            var summary = JSON.parse(stdout);
            callback(null, summary);
        } catch (e) {
            callback(new Error("Failed to parse policy summary"));
        }
    });
}

function getDetailedViolations(callback) {
    var cmd = "az policy state list" +
        " --subscription " + SUBSCRIPTION +
        " --resource-group " + RESOURCE_GROUP +
        " --filter \"complianceState eq 'NonCompliant'\"" +
        " --output json";

    cp.exec(cmd, { maxBuffer: 10 * 1024 * 1024 }, function(err, stdout, stderr) {
        if (err) return callback(new Error("Policy state query failed: " + stderr));

        try {
            var violations = JSON.parse(stdout);
            callback(null, violations);
        } catch (e) {
            callback(new Error("Failed to parse policy violations"));
        }
    });
}

console.log("Checking policy compliance...");
console.log("Subscription: " + SUBSCRIPTION);
console.log("Resource Group: " + RESOURCE_GROUP + "\n");

checkCompliance(function(err, summary) {
    if (err) {
        console.error(err.message);
        process.exit(1);
    }

    var results = summary.results || {};
    var nonCompliant = results.nonCompliantResources || 0;
    var compliant = (results.totalResources || 0) - nonCompliant;

    console.log("Compliance Summary:");
    console.log("  Total resources: " + (results.totalResources || 0));
    console.log("  Compliant: " + compliant);
    console.log("  Non-compliant: " + nonCompliant);

    if (nonCompliant === 0) {
        console.log("\nAll resources are compliant. Deployment can proceed.");
        process.exit(0);
    }

    console.log("\nNon-compliant resources detected. Fetching details...\n");

    getDetailedViolations(function(err2, violations) {
        if (err2) {
            console.error(err2.message);
            process.exit(1);
        }

        violations.forEach(function(v) {
            console.log("  Resource: " + (v.resourceId || "").split("/").pop());
            console.log("  Policy: " + (v.policyDefinitionName || "unknown"));
            console.log("  Effect: " + (v.policyDefinitionAction || "unknown"));
            console.log("  Details: " + (v.complianceReasonPhrase || "none"));
            console.log("");
        });

        var denyViolations = violations.filter(function(v) {
            return v.policyDefinitionAction === "deny";
        });

        if (denyViolations.length > 0) {
            console.error("BLOCKING: " + denyViolations.length + " deny-effect violations will block deployment.");
            process.exit(1);
        } else {
            console.log("WARNING: " + violations.length + " audit-effect violations found. Deployment can proceed but issues should be addressed.");
            process.exit(0);
        }
    });
});

Pipeline with Policy Gates

# azure-pipelines-compliant.yml
trigger:
  branches:
    include:
      - main

pool:
  vmImage: "ubuntu-latest"

variables:
  - group: Azure-Deployment-Settings

stages:
  - stage: Validate
    displayName: "Validate Infrastructure"
    jobs:
      - job: PolicyCheck
        steps:
          - task: AzureCLI@2
            displayName: "Check current compliance state"
            inputs:
              azureSubscription: "Azure-Production"
              scriptType: "bash"
              scriptLocation: "inlineScript"
              inlineScript: |
                node scripts/check-policy-compliance.js
            env:
              AZURE_SUBSCRIPTION_ID: $(SubscriptionId)
              RESOURCE_GROUP: $(ResourceGroup)

          - task: AzureCLI@2
            displayName: "What-if deployment validation"
            inputs:
              azureSubscription: "Azure-Production"
              scriptType: "bash"
              scriptLocation: "inlineScript"
              inlineScript: |
                az deployment group what-if \
                  --resource-group $(ResourceGroup) \
                  --template-file infrastructure/main.bicep \
                  --parameters infrastructure/parameters.prod.json \
                  --result-format FullResourcePayloads

  - stage: Deploy
    displayName: "Deploy Infrastructure"
    dependsOn: Validate
    jobs:
      - deployment: InfraDeploy
        environment: production
        strategy:
          runOnce:
            deploy:
              steps:
                - task: AzureCLI@2
                  displayName: "Deploy with ARM/Bicep"
                  inputs:
                    azureSubscription: "Azure-Production"
                    scriptType: "bash"
                    scriptLocation: "inlineScript"
                    inlineScript: |
                      az deployment group create \
                        --resource-group $(ResourceGroup) \
                        --template-file infrastructure/main.bicep \
                        --parameters infrastructure/parameters.prod.json \
                        --name "deploy-$(Build.BuildNumber)"

                - task: AzureCLI@2
                  displayName: "Post-deploy compliance check"
                  inputs:
                    azureSubscription: "Azure-Production"
                    scriptType: "bash"
                    scriptLocation: "inlineScript"
                    inlineScript: |
                      # Trigger a policy evaluation scan
                      az policy state trigger-scan \
                        --resource-group $(ResourceGroup) \
                        --no-wait

                      echo "Policy evaluation triggered. Results available in ~15 minutes."

Automated Remediation

Some policies can automatically fix non-compliant resources using the DeployIfNotExists or Modify effects.

Auto-Tag Resources

{
  "properties": {
    "displayName": "Auto-add environment tag based on resource group",
    "description": "Automatically adds an environment tag based on resource group naming convention",
    "mode": "Indexed",
    "policyRule": {
      "if": {
        "allOf": [
          {
            "field": "tags['environment']",
            "exists": "false"
          },
          {
            "value": "[resourceGroup().name]",
            "contains": "prod"
          }
        ]
      },
      "then": {
        "effect": "modify",
        "details": {
          "roleDefinitionIds": [
            "/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
          ],
          "operations": [
            {
              "operation": "addOrReplace",
              "field": "tags['environment']",
              "value": "production"
            }
          ]
        }
      }
    }
  }
}

Remediation Tasks

Trigger remediation for existing non-compliant resources:

# Create a remediation task
az policy remediation create \
  --name "fix-missing-tags-$(date +%Y%m%d)" \
  --policy-assignment "security-baseline-assignment" \
  --definition-reference-id "require-standard-tags" \
  --resource-group "rg-production" \
  --resource-discovery-mode "ExistingNonCompliant"

# Check remediation progress
az policy remediation show \
  --name "fix-missing-tags-20260210" \
  --resource-group "rg-production" \
  --output table

Automated Remediation Script

// scripts/remediate-policies.js
var cp = require("child_process");

var SUBSCRIPTION = process.env.AZURE_SUBSCRIPTION_ID;

function runAzCommand(args, callback) {
    var cmd = "az " + args + " --output json";
    cp.exec(cmd, { maxBuffer: 10 * 1024 * 1024 }, function(err, stdout, stderr) {
        if (err) return callback(new Error(stderr || err.message));
        try { callback(null, JSON.parse(stdout)); }
        catch (e) { callback(null, stdout); }
    });
}

function getNonCompliantPolicies(callback) {
    runAzCommand(
        "policy state summarize --subscription " + SUBSCRIPTION +
        " --filter \"complianceState eq 'NonCompliant'\"",
        function(err, summary) {
            if (err) return callback(err);

            var policies = (summary.policyAssignments || []).reduce(function(acc, assignment) {
                (assignment.policyDefinitions || []).forEach(function(def) {
                    if (def.results && def.results.nonCompliantResources > 0) {
                        acc.push({
                            assignmentId: assignment.policyAssignmentId,
                            definitionId: def.policyDefinitionId,
                            definitionName: def.policyDefinitionReferenceId,
                            effect: def.effect,
                            nonCompliant: def.results.nonCompliantResources
                        });
                    }
                });
                return acc;
            }, []);

            callback(null, policies);
        }
    );
}

function createRemediation(assignmentId, definitionRefId, callback) {
    var name = "auto-remediate-" + Date.now();
    runAzCommand(
        "policy remediation create" +
        " --name " + name +
        " --policy-assignment " + assignmentId +
        " --definition-reference-id " + definitionRefId +
        " --resource-discovery-mode ExistingNonCompliant" +
        " --subscription " + SUBSCRIPTION,
        function(err, result) {
            if (err) return callback(err);
            callback(null, { name: name, status: result.provisioningState });
        }
    );
}

console.log("Scanning for non-compliant policies...\n");

getNonCompliantPolicies(function(err, policies) {
    if (err) {
        console.error("Scan failed:", err.message);
        process.exit(1);
    }

    // Only auto-remediate policies with DeployIfNotExists or Modify effects
    var remediable = policies.filter(function(p) {
        return p.effect === "deployifnotexists" || p.effect === "modify";
    });

    console.log("Total non-compliant policies: " + policies.length);
    console.log("Auto-remediable: " + remediable.length + "\n");

    if (remediable.length === 0) {
        console.log("No auto-remediable policies found.");
        return;
    }

    var completed = 0;
    remediable.forEach(function(p) {
        console.log("Remediating: " + (p.definitionName || p.definitionId.split("/").pop()));
        console.log("  Non-compliant resources: " + p.nonCompliant);

        createRemediation(p.assignmentId, p.definitionName, function(err2, result) {
            completed++;
            if (err2) {
                console.log("  [FAIL] " + err2.message);
            } else {
                console.log("  [OK] Remediation task: " + result.name + " (" + result.status + ")");
            }

            if (completed === remediable.length) {
                console.log("\nAll remediation tasks created. Monitor progress with:");
                console.log("  az policy remediation list --subscription " + SUBSCRIPTION);
            }
        });
    });
});

Compliance Dashboard with Node.js

Build a compliance dashboard that queries policy state and generates HTML reports:

// scripts/compliance-dashboard.js
var cp = require("child_process");
var fs = require("fs");

var SUBSCRIPTION = process.env.AZURE_SUBSCRIPTION_ID;

function runAz(args, callback) {
    cp.exec("az " + args + " --output json", { maxBuffer: 10 * 1024 * 1024 }, function(err, stdout, stderr) {
        if (err) return callback(new Error(stderr));
        try { callback(null, JSON.parse(stdout)); }
        catch (e) { callback(new Error("Parse error")); }
    });
}

runAz("policy state summarize --subscription " + SUBSCRIPTION, function(err, summary) {
    if (err) {
        console.error("Failed:", err.message);
        process.exit(1);
    }

    var results = summary.results || {};
    var total = results.totalResources || 0;
    var nonCompliant = results.nonCompliantResources || 0;
    var compliant = total - nonCompliant;
    var rate = total > 0 ? ((compliant / total) * 100).toFixed(1) : "0.0";

    var assignments = summary.policyAssignments || [];

    var html = '<!DOCTYPE html><html><head><title>Compliance Dashboard</title>';
    html += '<style>body{font-family:sans-serif;margin:2rem}';
    html += '.stat{display:inline-block;padding:1rem 2rem;margin:0.5rem;border-radius:8px;text-align:center}';
    html += '.stat .number{font-size:2rem;font-weight:bold}';
    html += '.stat .label{font-size:0.8rem;color:#666}';
    html += '.compliant{background:#d4edda;color:#155724}';
    html += '.non-compliant{background:#f8d7da;color:#721c24}';
    html += '.total{background:#cce5ff;color:#004085}';
    html += '.rate{background:#fff3cd;color:#856404}';
    html += 'table{border-collapse:collapse;width:100%;margin-top:1rem}';
    html += 'th,td{border:1px solid #ddd;padding:8px;text-align:left}';
    html += 'th{background:#f4f4f4}</style></head><body>';

    html += '<h1>Azure Policy Compliance Dashboard</h1>';
    html += '<p>Generated: ' + new Date().toISOString() + '</p>';
    html += '<p>Subscription: ' + SUBSCRIPTION + '</p>';

    html += '<div>';
    html += '<div class="stat total"><div class="number">' + total + '</div><div class="label">Total Resources</div></div>';
    html += '<div class="stat compliant"><div class="number">' + compliant + '</div><div class="label">Compliant</div></div>';
    html += '<div class="stat non-compliant"><div class="number">' + nonCompliant + '</div><div class="label">Non-Compliant</div></div>';
    html += '<div class="stat rate"><div class="number">' + rate + '%</div><div class="label">Compliance Rate</div></div>';
    html += '</div>';

    html += '<h2>Policy Assignments</h2>';
    html += '<table><tr><th>Assignment</th><th>Compliant</th><th>Non-Compliant</th><th>Total</th></tr>';

    assignments.forEach(function(a) {
        var ar = a.results || {};
        var aTotal = ar.totalResources || 0;
        var aNonCompliant = ar.nonCompliantResources || 0;
        var aName = (a.policyAssignmentId || "").split("/").pop();

        html += '<tr>';
        html += '<td>' + aName + '</td>';
        html += '<td>' + (aTotal - aNonCompliant) + '</td>';
        html += '<td style="color:' + (aNonCompliant > 0 ? 'red' : 'green') + '">' + aNonCompliant + '</td>';
        html += '<td>' + aTotal + '</td>';
        html += '</tr>';
    });

    html += '</table></body></html>';

    var outputPath = "compliance-dashboard.html";
    fs.writeFileSync(outputPath, html);
    console.log("Dashboard generated: " + outputPath);
    console.log("Compliance rate: " + rate + "% (" + compliant + "/" + total + " resources)");
});

Common Issues and Troubleshooting

"RequestDisallowedByPolicy" when deploying resources

{
  "error": {
    "code": "RequestDisallowedByPolicy",
    "message": "Resource 'my-storage-account' was disallowed by policy. Policy: 'Require standard tags'.",
    "details": [{
      "code": "RequestDisallowedByPolicy",
      "target": "tags",
      "message": "Resource 'my-storage-account' was disallowed by policy."
    }]
  }
}

A deny-effect policy is blocking the deployment. Check the policy name in the error message and ensure your resource template includes the required properties. For tag policies, add tags to the resource definition. For location policies, deploy to an allowed region.

Policy compliance scan shows stale data

Azure Policy evaluates compliance on a schedule — typically every 24 hours for existing resources and immediately for new deployments. To force an immediate scan, trigger it manually:

az policy state trigger-scan --resource-group rg-production --no-wait

The scan takes 15-30 minutes for a typical resource group. Do not rely on compliance state being instant after a deployment.

Remediation task stuck in "Evaluating" state

Remediation 'fix-missing-tags' has been in 'Evaluating' state for over 2 hours.

Remediation tasks depend on the policy compliance scan finding non-compliant resources first. If the scan has not completed, the remediation waits. Also verify the managed identity assigned to the policy assignment has the necessary RBAC permissions to modify the target resources.

Custom policy definition shows syntax errors

PolicyDefinitionInvalidPolicyRule: The policy definition rule is invalid.

Common JSON syntax issues: missing commas, unquoted field names, mismatched brackets, or using single quotes instead of double quotes. Validate your JSON with jq . < policy.json before creating the definition. Also ensure the mode is correct — use Indexed for resource-level policies and All for subscription-level policies.

Best Practices

• Start with audit effect, then switch to deny. Deploy new policies in audit mode first. Review compliance results for false positives. Once you are confident the policy is correct, switch to deny effect to enforce.

• Use initiatives to group related policies. Assigning individual policies becomes unmanageable at scale. Group them into initiatives (security baseline, cost management, networking, etc.) and assign the initiative once.

• Test policies against existing resources before enforcement. Run a compliance scan in audit mode against your production subscription. A deny-effect policy on existing non-compliant resources does not break them, but it blocks updates — which can be just as disruptive.

• Assign policies at the management group level for organization-wide enforcement. Subscription-level assignments only cover one subscription. Management group assignments cascade to all child subscriptions automatically.

• Include policy validation in your CI/CD pipeline. The pre-deployment compliance check script catches violations before they hit the deployment stage. This saves time and prevents partially failed deployments.

• Version control your policy definitions. Store policy JSON in a Git repository alongside your infrastructure code. Changes to compliance rules should go through the same review process as code changes.

• Use exemptions sparingly and with expiration dates. When a resource legitimately cannot comply with a policy, create a time-limited exemption instead of disabling the policy. Set the exemption to expire in 30-90 days and require renewal.

References

• Azure Policy Overview
• Azure Policy Definition Structure
• Built-in Policy Definitions
• Azure Policy Remediation
• Azure Policy as Code
• Azure CLI Policy Commands