Webhook Automation with Azure DevOps
A comprehensive guide to webhook automation in Azure DevOps, covering service hooks configuration, custom webhook receivers, event filtering, payload processing, secure webhook endpoints, retry handling, and complete working examples for build, release, and work item event automation.
Webhook Automation with Azure DevOps
Overview
Webhooks are the backbone of event-driven automation in Azure DevOps. Every time a build completes, a work item changes, a pull request is opened, or code is pushed, Azure DevOps can fire an HTTP POST to any URL you specify. This turns Azure DevOps from a tool you interact with manually into an event source that triggers downstream systems automatically. I have built webhook-driven automation for everything from deployment orchestration to compliance auditing, and the patterns in this article come from those real systems.
Prerequisites
- Azure DevOps organization with project admin permissions (needed to configure service hooks)
- A publicly accessible HTTP endpoint for receiving webhooks (or ngrok for local development)
- Node.js 16 or later for webhook receiver examples
- Familiarity with Azure DevOps service hooks (Project Settings > Service hooks)
- Basic understanding of HTTP, JSON, and express.js
Service Hooks Fundamentals
Azure DevOps service hooks are the mechanism for outbound webhooks. They monitor specific events and send HTTP POST requests with event payloads to configured URLs.
Supported Event Types
Azure DevOps fires webhooks for these event categories:
Build Events:
build.complete— Build finished (any result)
Code Events:
git.push— Code pushed to a repositorygit.pullrequest.created— New pull request openedgit.pullrequest.updated— Pull request updated (new commits, votes, status changes)git.pullrequest.merged— Pull request completed and merged
Work Item Events:
workitem.created— New work item createdworkitem.updated— Work item fields changedworkitem.deleted— Work item deletedworkitem.restored— Work item restored from recycle binworkitem.commented— Comment added to work item
Release Events:
ms.vss-release.release-created-event— New release createdms.vss-release.deployment-started-event— Deployment stage startedms.vss-release.deployment-completed-event— Deployment stage finishedms.vss-release.deployment-approval-pending-event— Approval waitingms.vss-release.deployment-approval-completed-event— Approval given or rejected
Pipeline Events:
ms.vss-pipelines.run-state-changed-event— Pipeline run status changedms.vss-pipelines.stage-state-changed-event— Pipeline stage status changed
Configuring a Service Hook
Navigate to Project Settings > Service hooks > Create subscription:
- Select the service type: Web Hooks
- Choose the event trigger (e.g., "Build completed")
- Apply filters (specific definitions, branches, build results)
- Enter the webhook URL
- Optionally set HTTP headers for authentication
- Test the hook and save
You can also create service hooks via the REST API:
// create-service-hook.js
var apiClient = require("./api-client");
var client = apiClient.createClient(process.env.AZURE_ORG, process.env.AZURE_PAT);
var subscription = {
publisherId: "tfs",
eventType: "build.complete",
resourceVersion: "1.0",
consumerId: "webHooks",
consumerActionId: "httpRequest",
publisherInputs: {
projectId: process.env.PROJECT_ID,
buildStatus: "failed" // Only failed builds
},
consumerInputs: {
url: "https://your-webhook-receiver.com/hooks/builds",
httpHeaders: "X-Webhook-Secret:YOUR_SECRET_HERE",
resourceDetailsToSend: "all",
messagesToSend: "all",
detailedMessagesToSend: "all"
}
};
client.post("/_apis/hooks/subscriptions?api-version=7.1", subscription, function (err, result) {
if (err) { return console.error("Failed to create hook:", err.message); }
console.log("Created service hook: " + result.id);
console.log("Event: " + result.eventType);
console.log("URL: " + result.consumerInputs.url);
});
Building a Webhook Receiver
Basic Express Receiver
// webhook-receiver.js
var express = require("express");
var crypto = require("crypto");
var app = express();
app.use(express.json({ limit: "5mb" }));
var WEBHOOK_SECRET = process.env.WEBHOOK_SECRET || "";
var PORT = process.env.PORT || 4100;
// --- Middleware: Verify webhook authenticity ---
function verifyWebhook(req, res, next) {
if (!WEBHOOK_SECRET) {
return next(); // No secret configured, skip verification
}
var receivedSecret = req.headers["x-webhook-secret"];
if (!receivedSecret) {
console.warn("Missing X-Webhook-Secret header from " + req.ip);
return res.status(401).json({ error: "Missing authentication header" });
}
// Constant-time comparison to prevent timing attacks
var expected = Buffer.from(WEBHOOK_SECRET);
var received = Buffer.from(receivedSecret);
if (expected.length !== received.length || !crypto.timingSafeEqual(expected, received)) {
console.warn("Invalid webhook secret from " + req.ip);
return res.status(401).json({ error: "Invalid secret" });
}
next();
}
app.use("/hooks", verifyWebhook);
// --- Event handlers ---
var handlers = {};
function registerHandler(eventType, handler) {
if (!handlers[eventType]) {
handlers[eventType] = [];
}
handlers[eventType].push(handler);
}
function dispatchEvent(eventType, payload) {
var eventHandlers = handlers[eventType] || [];
console.log("Dispatching " + eventType + " to " + eventHandlers.length + " handler(s)");
eventHandlers.forEach(function (handler) {
try {
handler(payload);
} catch (err) {
console.error("Handler error for " + eventType + ": " + err.message);
}
});
}
// --- Routes ---
app.post("/hooks/builds", function (req, res) {
var payload = req.body;
var eventType = payload.eventType;
console.log("\n[" + new Date().toISOString() + "] Build event: " + eventType);
if (eventType === "build.complete") {
var resource = payload.resource;
console.log(" Build: #" + resource.buildNumber);
console.log(" Definition: " + resource.definition.name);
console.log(" Result: " + resource.result);
console.log(" Branch: " + (resource.sourceBranch || "").replace("refs/heads/", ""));
dispatchEvent("build.complete", payload);
}
res.status(200).json({ received: true });
});
app.post("/hooks/code", function (req, res) {
var payload = req.body;
var eventType = payload.eventType;
console.log("\n[" + new Date().toISOString() + "] Code event: " + eventType);
if (eventType === "git.push") {
var resource = payload.resource;
var commits = resource.commits || [];
console.log(" Repo: " + resource.repository.name);
console.log(" Branch: " + (resource.refUpdates[0] || {}).name);
console.log(" Commits: " + commits.length);
commits.forEach(function (c) {
console.log(" - " + c.commitId.substring(0, 8) + " " + c.comment);
});
dispatchEvent("git.push", payload);
}
if (eventType === "git.pullrequest.created" || eventType === "git.pullrequest.updated") {
var prResource = payload.resource;
console.log(" PR #" + prResource.pullRequestId + ": " + prResource.title);
console.log(" Author: " + prResource.createdBy.displayName);
console.log(" " + prResource.sourceRefName + " -> " + prResource.targetRefName);
dispatchEvent(eventType, payload);
}
res.status(200).json({ received: true });
});
app.post("/hooks/workitems", function (req, res) {
var payload = req.body;
var eventType = payload.eventType;
console.log("\n[" + new Date().toISOString() + "] Work item event: " + eventType);
if (eventType === "workitem.created" || eventType === "workitem.updated") {
var resource = payload.resource;
var fields = resource.fields || {};
console.log(" #" + resource.id + ": " + fields["System.Title"]);
console.log(" Type: " + fields["System.WorkItemType"]);
console.log(" State: " + fields["System.State"]);
if (payload.resource.revision && payload.resource.revision.fields) {
var changedFields = Object.keys(payload.resource.revision.fields);
console.log(" Changed fields: " + changedFields.join(", "));
}
dispatchEvent(eventType, payload);
}
res.status(200).json({ received: true });
});
app.post("/hooks/releases", function (req, res) {
var payload = req.body;
var eventType = payload.eventType;
console.log("\n[" + new Date().toISOString() + "] Release event: " + eventType);
dispatchEvent(eventType, payload);
res.status(200).json({ received: true });
});
// Health check
app.get("/health", function (req, res) {
res.json({ status: "ok", uptime: process.uptime(), handlers: Object.keys(handlers) });
});
// --- Start ---
app.listen(PORT, function () {
console.log("Webhook receiver listening on port " + PORT);
console.log("Endpoints:");
console.log(" POST /hooks/builds — Build events");
console.log(" POST /hooks/code — Push and PR events");
console.log(" POST /hooks/workitems — Work item events");
console.log(" POST /hooks/releases — Release events");
console.log(" GET /health — Health check");
});
module.exports = { app: app, registerHandler: registerHandler };
Event-Driven Automation Patterns
Auto-Create Bug from Failed Build
When a build fails on the main branch, automatically create a bug work item assigned to the person who triggered the build:
// automations/failed-build-bug.js
var receiver = require("../webhook-receiver");
var apiClient = require("../api-client");
var client = apiClient.createClient(process.env.AZURE_ORG, process.env.AZURE_PAT);
var PROJECT = process.env.AZURE_PROJECT;
receiver.registerHandler("build.complete", function (payload) {
var resource = payload.resource;
var branch = (resource.sourceBranch || "").replace("refs/heads/", "");
// Only create bugs for failed builds on main
if (resource.result !== "failed" || branch !== "main") {
return;
}
var requestedBy = resource.requestedFor ? resource.requestedFor.uniqueName : "";
var buildUrl = resource._links && resource._links.web ? resource._links.web.href : "";
var patchDoc = [
{ op: "add", path: "/fields/System.Title", value: "Build failure: " + resource.definition.name + " #" + resource.buildNumber },
{ op: "add", path: "/fields/System.Description", value:
"<p>Build <a href=\"" + buildUrl + "\">#" + resource.buildNumber + "</a> failed on branch <strong>" + branch + "</strong>.</p>" +
"<p>Definition: " + resource.definition.name + "</p>" +
"<p>Triggered by: " + requestedBy + "</p>" +
"<p>Time: " + resource.finishTime + "</p>"
},
{ op: "add", path: "/fields/System.AssignedTo", value: requestedBy },
{ op: "add", path: "/fields/Microsoft.VSTS.Common.Priority", value: 1 },
{ op: "add", path: "/fields/System.Tags", value: "auto-created;build-failure" }
];
client.patch("/" + PROJECT + "/_apis/wit/workitems/$Bug?api-version=7.1", patchDoc, function (err, result) {
if (err) {
console.error("Failed to create bug for build failure:", err.message);
return;
}
console.log("Created bug #" + result.id + " for failed build #" + resource.buildNumber);
});
});
Auto-Label PRs by File Path
Automatically add labels or tags to pull requests based on which files were changed:
// automations/pr-auto-label.js
var receiver = require("../webhook-receiver");
var apiClient = require("../api-client");
var client = apiClient.createClient(process.env.AZURE_ORG, process.env.AZURE_PAT);
var PROJECT = process.env.AZURE_PROJECT;
var pathLabels = [
{ pattern: /^src\/api\//, label: "api" },
{ pattern: /^src\/ui\//, label: "frontend" },
{ pattern: /^infrastructure\//, label: "infrastructure" },
{ pattern: /^tests\//, label: "tests" },
{ pattern: /\.sql$/, label: "database" },
{ pattern: /Dockerfile|docker-compose/, label: "docker" },
{ pattern: /package\.json|yarn\.lock/, label: "dependencies" }
];
receiver.registerHandler("git.pullrequest.created", function (payload) {
var pr = payload.resource;
var repoId = pr.repository.id;
var prId = pr.pullRequestId;
// Get the list of changed files in the PR
client.get("/" + PROJECT + "/_apis/git/repositories/" + repoId +
"/pullRequests/" + prId + "/iterations?api-version=7.1", function (err, data) {
if (err) { return console.error("Failed to get PR iterations:", err.message); }
var latestIteration = data.value[data.value.length - 1];
if (!latestIteration) { return; }
client.get("/" + PROJECT + "/_apis/git/repositories/" + repoId +
"/pullRequests/" + prId + "/iterations/" + latestIteration.id +
"/changes?api-version=7.1", function (err2, changes) {
if (err2) { return console.error("Failed to get PR changes:", err2.message); }
var labels = {};
var changeEntries = changes.changeEntries || [];
changeEntries.forEach(function (change) {
var filePath = change.item.path || "";
pathLabels.forEach(function (rule) {
if (rule.pattern.test(filePath)) {
labels[rule.label] = true;
}
});
});
var labelList = Object.keys(labels);
if (labelList.length === 0) { return; }
console.log("PR #" + prId + " labels: " + labelList.join(", "));
// Add labels to the PR
var labelPayload = labelList.map(function (l) { return { name: l }; });
client.post("/" + PROJECT + "/_apis/git/repositories/" + repoId +
"/pullRequests/" + prId + "/labels?api-version=7.1", labelPayload,
function (err3) {
if (err3) { console.error("Failed to add labels:", err3.message); }
else { console.log("Added " + labelList.length + " label(s) to PR #" + prId); }
}
);
});
});
});
Deployment Audit Trail
Record every deployment event to a log store for compliance:
// automations/deployment-audit.js
var receiver = require("../webhook-receiver");
var fs = require("fs");
var path = require("path");
var AUDIT_LOG_DIR = process.env.AUDIT_LOG_DIR || path.join(__dirname, "../audit-logs");
if (!fs.existsSync(AUDIT_LOG_DIR)) {
fs.mkdirSync(AUDIT_LOG_DIR, { recursive: true });
}
function logAuditEvent(event) {
var date = new Date();
var fileName = "audit-" + date.toISOString().split("T")[0] + ".jsonl";
var filePath = path.join(AUDIT_LOG_DIR, fileName);
var entry = JSON.stringify(event) + "\n";
fs.appendFileSync(filePath, entry);
}
receiver.registerHandler("build.complete", function (payload) {
var resource = payload.resource;
logAuditEvent({
timestamp: new Date().toISOString(),
eventType: "build.complete",
buildId: resource.id,
buildNumber: resource.buildNumber,
definition: resource.definition.name,
result: resource.result,
requestedBy: resource.requestedFor ? resource.requestedFor.displayName : "unknown",
sourceBranch: resource.sourceBranch,
repository: resource.repository ? resource.repository.name : "unknown"
});
});
["ms.vss-release.deployment-started-event",
"ms.vss-release.deployment-completed-event",
"ms.vss-release.deployment-approval-completed-event"].forEach(function (eventType) {
receiver.registerHandler(eventType, function (payload) {
var resource = payload.resource;
var environment = resource.environment || resource.releaseEnvironment || {};
logAuditEvent({
timestamp: new Date().toISOString(),
eventType: eventType,
releaseId: resource.release ? resource.release.id : resource.id,
releaseName: resource.release ? resource.release.name : resource.name,
environment: environment.name || "unknown",
status: environment.status || resource.status || "unknown",
approvedBy: resource.approval ? resource.approval.approvedBy.displayName : null
});
});
});
Webhook Security
HMAC Signature Verification
Azure DevOps service hooks support sending custom HTTP headers. Use a shared secret and verify it on the receiver side. For stronger security, implement HMAC-based verification:
// webhook-security.js
var crypto = require("crypto");
function generateHmacSignature(body, secret) {
return crypto.createHmac("sha256", secret)
.update(typeof body === "string" ? body : JSON.stringify(body))
.digest("hex");
}
function createHmacMiddleware(secret) {
return function (req, res, next) {
var signature = req.headers["x-webhook-signature"];
if (!signature) {
return res.status(401).json({ error: "Missing signature" });
}
// Need raw body for HMAC calculation
var rawBody = JSON.stringify(req.body);
var expected = generateHmacSignature(rawBody, secret);
if (!crypto.timingSafeEqual(Buffer.from(signature, "hex"), Buffer.from(expected, "hex"))) {
console.warn("Invalid HMAC signature from " + req.ip);
return res.status(401).json({ error: "Invalid signature" });
}
next();
};
}
module.exports = {
generateHmacSignature: generateHmacSignature,
createHmacMiddleware: createHmacMiddleware
};
IP Allowlisting
Azure DevOps sends webhooks from specific IP ranges. Restrict your receiver to only accept requests from these ranges:
// ip-filter.js
// Azure DevOps Services IP ranges (check Microsoft docs for current list)
var ALLOWED_RANGES = [
"13.107.6.0/24",
"13.107.9.0/24",
"13.107.42.0/24",
"13.107.43.0/24"
];
function ipInRange(ip, cidr) {
var parts = cidr.split("/");
var rangeIp = parts[0];
var mask = parseInt(parts[1], 10);
var ipNum = ip.split(".").reduce(function (acc, octet) { return (acc << 8) + parseInt(octet, 10); }, 0) >>> 0;
var rangeNum = rangeIp.split(".").reduce(function (acc, octet) { return (acc << 8) + parseInt(octet, 10); }, 0) >>> 0;
var maskNum = (-1 << (32 - mask)) >>> 0;
return (ipNum & maskNum) === (rangeNum & maskNum);
}
function isAllowedIP(ip) {
return ALLOWED_RANGES.some(function (range) {
return ipInRange(ip, range);
});
}
module.exports = { isAllowedIP: isAllowedIP };
Retry Handling and Reliability
Azure DevOps retries failed webhook deliveries with exponential backoff. A service hook subscription retries up to 4 times with increasing delays. Your receiver should be idempotent — processing the same event twice should produce the same result.
// idempotency.js
var processedEvents = {};
function isProcessed(eventId) {
if (processedEvents[eventId]) {
return true;
}
processedEvents[eventId] = Date.now();
// Clean up old entries every hour
var now = Date.now();
var oneHour = 60 * 60 * 1000;
Object.keys(processedEvents).forEach(function (key) {
if (now - processedEvents[key] > oneHour) {
delete processedEvents[key];
}
});
return false;
}
// Usage in handler
function handleEvent(req, res) {
var eventId = req.headers["x-vss-subscriptionid"] + "-" +
(req.body.id || req.body.resource.id || Date.now());
if (isProcessed(eventId)) {
console.log("Duplicate event " + eventId + ", skipping");
return res.status(200).json({ received: true, duplicate: true });
}
// Process the event...
res.status(200).json({ received: true });
}
For production systems, replace the in-memory store with Redis or a database table to handle restarts and multiple receiver instances.
Complete Working Example: Multi-Automation Webhook Server
This ties everything together — a production-grade webhook server with multiple automations, security, logging, and health monitoring:
// server.js
var express = require("express");
var fs = require("fs");
var path = require("path");
var app = express();
app.use(express.json({ limit: "5mb" }));
var PORT = process.env.PORT || 4100;
var WEBHOOK_SECRET = process.env.WEBHOOK_SECRET;
// --- Request logging ---
app.use(function (req, res, next) {
if (req.path.startsWith("/hooks")) {
console.log("[" + new Date().toISOString() + "] " + req.method + " " + req.path +
" from " + req.ip + " (" + (req.headers["content-length"] || 0) + " bytes)");
}
next();
});
// --- Secret verification ---
app.use("/hooks", function (req, res, next) {
if (!WEBHOOK_SECRET) { return next(); }
var received = req.headers["x-webhook-secret"];
if (received !== WEBHOOK_SECRET) {
console.warn("Rejected: invalid secret from " + req.ip);
return res.status(401).json({ error: "Unauthorized" });
}
next();
});
// --- Event metrics ---
var metrics = {
received: 0,
processed: 0,
errors: 0,
byType: {}
};
function trackEvent(eventType) {
metrics.received++;
metrics.byType[eventType] = (metrics.byType[eventType] || 0) + 1;
}
// --- Generic webhook endpoint ---
app.post("/hooks/:category", function (req, res) {
var category = req.params.category;
var payload = req.body;
var eventType = payload.eventType || "unknown";
trackEvent(eventType);
console.log(" Event: " + eventType + " (category: " + category + ")");
// Load automations dynamically
var automationDir = path.join(__dirname, "automations");
if (fs.existsSync(automationDir)) {
var files = fs.readdirSync(automationDir);
files.forEach(function (file) {
if (file.endsWith(".js")) {
try {
var automation = require(path.join(automationDir, file));
if (typeof automation.handle === "function") {
automation.handle(eventType, payload);
metrics.processed++;
}
} catch (err) {
console.error("Automation " + file + " error: " + err.message);
metrics.errors++;
}
}
});
}
res.status(200).json({ received: true, eventType: eventType });
});
// --- Health and metrics ---
app.get("/health", function (req, res) {
res.json({
status: "ok",
uptime: Math.round(process.uptime()),
metrics: metrics
});
});
app.get("/metrics", function (req, res) {
res.json(metrics);
});
app.listen(PORT, function () {
console.log("Webhook automation server on port " + PORT);
console.log("Secret verification: " + (WEBHOOK_SECRET ? "enabled" : "DISABLED"));
// List loaded automations
var automationDir = path.join(__dirname, "automations");
if (fs.existsSync(automationDir)) {
var files = fs.readdirSync(automationDir).filter(function (f) { return f.endsWith(".js"); });
console.log("Loaded automations: " + files.join(", "));
}
});
Example automation module:
// automations/notify-on-critical-bug.js
var https = require("https");
var SLACK_WEBHOOK = process.env.SLACK_CRITICAL_BUGS_WEBHOOK;
function handle(eventType, payload) {
if (eventType !== "workitem.created") { return; }
var resource = payload.resource;
var fields = resource.fields || {};
if (fields["System.WorkItemType"] !== "Bug") { return; }
if (fields["Microsoft.VSTS.Common.Priority"] > 1) { return; }
// Priority 1 bug — send immediate notification
var message = {
text: "🚨 *Critical Bug Created*\n" +
"#" + resource.id + ": " + fields["System.Title"] + "\n" +
"Assigned to: " + (fields["System.AssignedTo"] || "unassigned") + "\n" +
"Area: " + (fields["System.AreaPath"] || "")
};
if (!SLACK_WEBHOOK) {
console.log("Would notify Slack: " + message.text);
return;
}
var url = new URL(SLACK_WEBHOOK);
var body = JSON.stringify(message);
var req = https.request({
hostname: url.hostname,
path: url.pathname,
method: "POST",
headers: { "Content-Type": "application/json" }
});
req.on("error", function (err) { console.error("Slack notify failed:", err.message); });
req.write(body);
req.end();
}
module.exports = { handle: handle };
Common Issues and Troubleshooting
Service hook shows "Disabled" status after repeated failures
Azure DevOps disables service hooks after multiple consecutive delivery failures (usually 10). The hook stops firing until you manually re-enable it. Navigate to Project Settings > Service hooks, find the disabled subscription, and click Enable. Fix the root cause — your receiver was likely down or returning non-200 status codes.
Webhook payload missing expected fields
TypeError: Cannot read property 'displayName' of undefined
Different event types have different payload structures. A build.complete event has resource.requestedFor.displayName, but a workitem.updated event does not. Always check for null/undefined before accessing nested properties. Use the "Test" button in service hook configuration to see the exact payload structure for your event type.
Webhook events arriving with significant delay
Service hook delivery is not guaranteed to be instant. Under heavy load, Azure DevOps may queue webhook deliveries. Typical latency is under 5 seconds, but during outages or high-traffic periods it can be minutes. Do not build workflows that depend on sub-second webhook delivery.
Multiple service hooks firing for the same event
If you have overlapping service hook subscriptions (e.g., two hooks both listening to build.complete without filters), both fire for every build. Audit your service hooks list regularly and remove duplicates. Use filters to narrow which events each hook receives.
Local development: webhooks cannot reach localhost
Azure DevOps cannot send webhooks to localhost or private IPs. Use ngrok or a similar tunneling tool for development:
ngrok http 4100
# Forwarding: https://abc123.ngrok.io -> http://localhost:4100
# Use the ngrok URL as your service hook endpoint
Best Practices
Always return 200 status codes quickly. Process webhook payloads asynchronously after responding. If your handler takes more than 10 seconds, Azure DevOps marks it as failed and retries, potentially causing duplicate processing.
Implement idempotency for all handlers. Webhooks can be delivered more than once due to retries, network issues, or Azure DevOps internal processing. Use a unique event identifier to detect and skip duplicates.
Filter events at the service hook level, not in your receiver. Azure DevOps supports filtering by build definition, branch, work item type, and more. Filtering at the source reduces unnecessary HTTP traffic and processing.
Monitor service hook health in Azure DevOps. The service hooks page shows delivery success rates and recent failures. Set up a periodic check (or use the service hooks API) to alert when hooks become disabled.
Use separate webhook URLs per event category. Route builds to
/hooks/builds, PRs to/hooks/code, etc. This makes it easier to debug, scale, and apply different security policies per event type.Store webhook payloads for debugging. Log the full JSON payload of every incoming webhook for at least 7 days. When automation behaves unexpectedly, the raw payload is the first thing you need to diagnose the issue.
Secure your webhook endpoints. Use shared secrets, HMAC signatures, or IP allowlisting. An unsecured webhook endpoint is an attack vector — anyone who discovers the URL can trigger your automations with crafted payloads.