Test Plans

Selenium Integration with Azure DevOps

A practical guide to integrating Selenium WebDriver tests with Azure DevOps, covering Selenium setup with Node.js, browser driver management, headless execution in pipelines, test result publishing, screenshot capture on failure, parallel execution, and cross-browser testing with configurations.

Selenium Integration with Azure DevOps

Overview

Selenium WebDriver is the most widely adopted browser automation framework for UI testing. It supports Chrome, Firefox, Edge, and Safari through standardized driver interfaces, and runs on every major platform. Integrating Selenium tests with Azure DevOps means running those tests automatically in your CI/CD pipeline, publishing results to the Test tab, capturing screenshots on failure, and managing browser drivers across different agent environments. The integration is not plug-and-play -- headless browser configuration, driver version management, and pipeline agent capabilities all require explicit setup.

I have run Selenium test suites in Azure Pipelines ranging from 10 tests against a single browser to 500 tests across a 4-browser matrix. The patterns are consistent regardless of scale: configure the driver, run headless, capture artifacts on failure, publish results in JUnit format. This article covers the complete integration from local development through pipeline execution, using Node.js and selenium-webdriver as the primary stack.

Prerequisites

  • Node.js 18+ installed locally and available on pipeline agents
  • Azure DevOps organization with Azure Pipelines enabled
  • Basic understanding of Selenium WebDriver concepts (locators, waits, actions)
  • A web application accessible from the pipeline agent (deployed or running locally)
  • Familiarity with YAML pipeline syntax
  • npm or yarn for dependency management

Setting Up Selenium with Node.js

Project Structure

selenium-tests/
├── package.json
├── tests/
│   ├── login.test.js
│   ├── dashboard.test.js
│   └── checkout.test.js
├── pages/
│   ├── loginPage.js
│   ├── dashboardPage.js
│   └── checkoutPage.js
├── utils/
│   ├── driver-factory.js
│   ├── reporter.js
│   └── screenshots.js
├── results/
│   └── (generated test results)
├── screenshots/
│   └── (captured on failure)
└── jest.config.js

Dependencies

{
  "name": "selenium-tests",
  "version": "1.0.0",
  "scripts": {
    "test": "jest --forceExit --detectOpenHandles",
    "test:chrome": "BROWSER=chrome jest --forceExit",
    "test:firefox": "BROWSER=firefox jest --forceExit",
    "test:headless": "HEADLESS=true jest --forceExit"
  },
  "devDependencies": {
    "selenium-webdriver": "4.17.0",
    "jest": "29.7.0",
    "jest-junit": "16.0.0",
    "chromedriver": "121.0.0",
    "geckodriver": "4.3.0"
  }
}

Browser Driver Factory

The driver factory centralizes browser initialization and handles headless configuration:

// utils/driver-factory.js
var webdriver = require("selenium-webdriver");
var chrome = require("selenium-webdriver/chrome");
var firefox = require("selenium-webdriver/firefox");

function createDriver(browserName) {
  var browser = browserName || process.env.BROWSER || "chrome";
  var headless = process.env.HEADLESS === "true" || process.env.CI === "true";

  var builder = new webdriver.Builder();

  if (browser === "chrome") {
    var chromeOptions = new chrome.Options();
    if (headless) {
      chromeOptions.addArguments("--headless=new");
      chromeOptions.addArguments("--no-sandbox");
      chromeOptions.addArguments("--disable-dev-shm-usage");
      chromeOptions.addArguments("--disable-gpu");
      chromeOptions.addArguments("--window-size=1920,1080");
    }
    builder.forBrowser("chrome").setChromeOptions(chromeOptions);
  } else if (browser === "firefox") {
    var firefoxOptions = new firefox.Options();
    if (headless) {
      firefoxOptions.addArguments("--headless");
      firefoxOptions.addArguments("--width=1920");
      firefoxOptions.addArguments("--height=1080");
    }
    builder.forBrowser("firefox").setFirefoxOptions(firefoxOptions);
  } else {
    throw new Error("Unsupported browser: " + browser);
  }

  return builder.build();
}

module.exports = { createDriver: createDriver };

The --no-sandbox and --disable-dev-shm-usage flags are critical for Chrome in CI environments. Without them, Chrome crashes on Linux agents because the default sandbox configuration requires kernel capabilities that pipeline agents do not have. The --disable-dev-shm-usage flag prevents crashes caused by the limited /dev/shm shared memory partition on containerized agents.

Page Object Pattern

Page objects encapsulate page-specific interactions:

// pages/loginPage.js
var webdriver = require("selenium-webdriver");
var By = webdriver.By;
var until = webdriver.until;

function LoginPage(driver) {
  this.driver = driver;
  this.url = process.env.BASE_URL || "http://localhost:3000";

  this.emailInput = By.css('input[name="email"]');
  this.passwordInput = By.css('input[name="password"]');
  this.submitButton = By.css('button[type="submit"]');
  this.errorMessage = By.css(".alert-danger");
  this.welcomeMessage = By.css(".welcome-text");
}

LoginPage.prototype.navigate = function () {
  return this.driver.get(this.url + "/login");
};

LoginPage.prototype.login = function (email, password) {
  var self = this;
  return self.driver
    .wait(until.elementLocated(self.emailInput), 10000)
    .then(function () {
      return self.driver.findElement(self.emailInput);
    })
    .then(function (el) {
      return el.clear().then(function () {
        return el.sendKeys(email);
      });
    })
    .then(function () {
      return self.driver.findElement(self.passwordInput);
    })
    .then(function (el) {
      return el.clear().then(function () {
        return el.sendKeys(password);
      });
    })
    .then(function () {
      return self.driver.findElement(self.submitButton);
    })
    .then(function (el) {
      return el.click();
    });
};

LoginPage.prototype.getErrorMessage = function () {
  var self = this;
  return self.driver
    .wait(until.elementLocated(self.errorMessage), 5000)
    .then(function () {
      return self.driver.findElement(self.errorMessage).getText();
    });
};

LoginPage.prototype.getWelcomeMessage = function () {
  var self = this;
  return self.driver
    .wait(until.elementLocated(self.welcomeMessage), 10000)
    .then(function () {
      return self.driver.findElement(self.welcomeMessage).getText();
    });
};

module.exports = LoginPage;

Screenshot Capture on Failure

Capture screenshots automatically when tests fail:

// utils/screenshots.js
var fs = require("fs");
var path = require("path");

var screenshotDir = path.join(__dirname, "..", "screenshots");

function ensureDir() {
  if (!fs.existsSync(screenshotDir)) {
    fs.mkdirSync(screenshotDir, { recursive: true });
  }
}

function captureScreenshot(driver, testName) {
  ensureDir();
  var filename = testName.replace(/[^a-zA-Z0-9]/g, "_") + "_" + Date.now() + ".png";
  var filepath = path.join(screenshotDir, filename);

  return driver.takeScreenshot().then(function (data) {
    fs.writeFileSync(filepath, data, "base64");
    console.log("Screenshot saved: " + filepath);
    return filepath;
  }).catch(function (err) {
    console.error("Failed to capture screenshot: " + err.message);
    return null;
  });
}

module.exports = { captureScreenshot: captureScreenshot };

Writing Tests with Jest

// tests/login.test.js
var driverFactory = require("../utils/driver-factory");
var screenshots = require("../utils/screenshots");
var LoginPage = require("../pages/loginPage");

var driver;
var loginPage;

beforeAll(function () {
  driver = driverFactory.createDriver();
  loginPage = new LoginPage(driver);
});

afterAll(function () {
  if (driver) {
    return driver.quit();
  }
});

afterEach(function () {
  var testName = expect.getState().currentTestName;
  var testPassed = expect.getState().snapshotState &&
    !expect.getState().snapshotState._uncheckedKeys;

  // Capture screenshot on failure
  if (expect.getState().isExpectingAssertionError) {
    return screenshots.captureScreenshot(driver, testName);
  }
});

describe("Login Page", function () {
  test("should display login form", function () {
    return loginPage.navigate().then(function () {
      return driver.findElement(loginPage.emailInput);
    }).then(function (el) {
      return el.isDisplayed();
    }).then(function (displayed) {
      expect(displayed).toBe(true);
    });
  });

  test("should login with valid credentials", function () {
    return loginPage.navigate().then(function () {
      return loginPage.login("[email protected]", "ValidP@ss123");
    }).then(function () {
      return loginPage.getWelcomeMessage();
    }).then(function (message) {
      expect(message).toContain("Welcome");
    });
  });

  test("should show error for invalid credentials", function () {
    return loginPage.navigate().then(function () {
      return loginPage.login("[email protected]", "wrongpassword");
    }).then(function () {
      return loginPage.getErrorMessage();
    }).then(function (message) {
      expect(message).toContain("Invalid");
    });
  });

  test("should require email field", function () {
    return loginPage.navigate().then(function () {
      return loginPage.login("", "ValidP@ss123");
    }).then(function () {
      return loginPage.getErrorMessage();
    }).then(function (message) {
      expect(message).toContain("required");
    });
  });
});

Jest Configuration

// jest.config.js
module.exports = {
  testTimeout: 60000,
  reporters: [
    "default",
    [
      "jest-junit",
      {
        outputDirectory: "results",
        outputName: "junit-results.xml",
        suiteName: "Selenium UI Tests",
        classNameTemplate: "{classname}",
        titleTemplate: "{title}",
      },
    ],
  ],
};

Azure Pipeline Configuration

Basic Pipeline with Chrome

trigger:
  branches:
    include:
      - main
      - feature/*

pool:
  vmImage: ubuntu-latest

variables:
  BASE_URL: "https://staging.example.com"
  HEADLESS: "true"
  CI: "true"

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: "20.x"
    displayName: Use Node.js 20.x

  - script: npm ci
    workingDirectory: selenium-tests
    displayName: Install dependencies

  - script: npm test
    workingDirectory: selenium-tests
    displayName: Run Selenium tests
    env:
      BROWSER: chrome
      BASE_URL: $(BASE_URL)
    continueOnError: true

  - task: PublishTestResults@2
    inputs:
      testResultsFormat: JUnit
      testResultsFiles: "**/results/junit-results.xml"
      testRunTitle: "Selenium UI Tests - Chrome"
      mergeTestResults: true
    condition: always()
    displayName: Publish test results

  - task: PublishBuildArtifacts@1
    inputs:
      pathToPublish: selenium-tests/screenshots
      artifactName: failure-screenshots
    condition: always()
    displayName: Publish failure screenshots

Cross-Browser Pipeline with Matrix Strategy

trigger:
  branches:
    include:
      - main

pool:
  vmImage: ubuntu-latest

strategy:
  matrix:
    Chrome:
      BROWSER: chrome
      BROWSER_DISPLAY: Chrome
    Firefox:
      BROWSER: firefox
      BROWSER_DISPLAY: Firefox

variables:
  BASE_URL: "https://staging.example.com"
  HEADLESS: "true"
  CI: "true"

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: "20.x"

  - script: npm ci
    workingDirectory: selenium-tests
    displayName: Install dependencies

  - script: npm test
    workingDirectory: selenium-tests
    displayName: Run tests on $(BROWSER_DISPLAY)
    env:
      BROWSER: $(BROWSER)
    continueOnError: true

  - task: PublishTestResults@2
    inputs:
      testResultsFormat: JUnit
      testResultsFiles: "**/results/junit-results.xml"
      testRunTitle: "Selenium - $(BROWSER_DISPLAY)"
      mergeTestResults: true
    condition: always()

  - task: PublishBuildArtifacts@1
    inputs:
      pathToPublish: selenium-tests/screenshots
      artifactName: screenshots-$(BROWSER_DISPLAY)
    condition: always()

The matrix strategy runs Chrome and Firefox tests in parallel on separate agents. Each browser gets its own test run title in the Azure DevOps Tests tab.

Handling Browser Driver Versions

Browser drivers must match the installed browser version. On Azure-hosted agents, browser versions update regularly. Instead of pinning driver versions in package.json, use dynamic driver management:

// utils/driver-factory.js (updated with dynamic driver management)
var webdriver = require("selenium-webdriver");
var chrome = require("selenium-webdriver/chrome");
var firefox = require("selenium-webdriver/firefox");

function createDriver(browserName) {
  var browser = browserName || process.env.BROWSER || "chrome";
  var headless = process.env.HEADLESS === "true" || process.env.CI === "true";

  var builder = new webdriver.Builder();

  if (browser === "chrome") {
    var chromeOptions = new chrome.Options();
    if (headless) {
      chromeOptions.addArguments("--headless=new");
      chromeOptions.addArguments("--no-sandbox");
      chromeOptions.addArguments("--disable-dev-shm-usage");
      chromeOptions.addArguments("--disable-gpu");
      chromeOptions.addArguments("--window-size=1920,1080");
    }

    // Use Chrome for Testing - auto-manages driver version
    var service = new chrome.ServiceBuilder();
    builder
      .forBrowser("chrome")
      .setChromeOptions(chromeOptions)
      .setChromeService(service);
  } else if (browser === "firefox") {
    var firefoxOptions = new firefox.Options();
    if (headless) {
      firefoxOptions.addArguments("--headless");
      firefoxOptions.addArguments("--width=1920");
      firefoxOptions.addArguments("--height=1080");
    }

    var firefoxService = new firefox.ServiceBuilder();
    builder
      .forBrowser("firefox")
      .setFirefoxOptions(firefoxOptions)
      .setGeckoService(firefoxService);
  }

  return builder.build();
}

module.exports = { createDriver: createDriver };

Starting with Selenium 4.6+, the Selenium Manager component automatically downloads the correct driver version for the installed browser. This eliminates the driver version mismatch problem that plagued earlier Selenium versions.

Waiting Strategies

Unreliable waits are the leading cause of flaky Selenium tests. Use explicit waits, never implicit waits or sleep():

var webdriver = require("selenium-webdriver");
var until = webdriver.until;
var By = webdriver.By;

// BAD: Implicit wait - applies globally, masks real timing issues
// driver.manage().setTimeouts({ implicit: 10000 });

// BAD: Hard-coded sleep - wastes time or fails if too short
// driver.sleep(3000);

// GOOD: Explicit wait for specific condition
function waitForElement(driver, locator, timeout) {
  timeout = timeout || 10000;
  return driver.wait(until.elementLocated(locator), timeout).then(function () {
    return driver.wait(
      until.elementIsVisible(driver.findElement(locator)),
      timeout
    );
  });
}

// GOOD: Wait for element to be clickable
function waitForClickable(driver, locator, timeout) {
  timeout = timeout || 10000;
  return driver.wait(until.elementLocated(locator), timeout).then(function () {
    return driver.wait(
      until.elementIsEnabled(driver.findElement(locator)),
      timeout
    );
  });
}

// GOOD: Custom condition - wait for AJAX to complete
function waitForAjax(driver, timeout) {
  timeout = timeout || 15000;
  return driver.wait(function () {
    return driver.executeScript(
      "return document.readyState === 'complete' && " +
        "(typeof jQuery === 'undefined' || jQuery.active === 0)"
    );
  }, timeout);
}

// GOOD: Wait for URL change
function waitForNavigation(driver, expectedUrlPart, timeout) {
  timeout = timeout || 10000;
  return driver.wait(until.urlContains(expectedUrlPart), timeout);
}

Complete Working Example

A complete Selenium test suite integrated with Azure Pipelines, including page objects, screenshot capture, retry logic, and multi-browser execution:

// full-test-suite.js
var webdriver = require("selenium-webdriver");
var chrome = require("selenium-webdriver/chrome");
var firefox = require("selenium-webdriver/firefox");
var fs = require("fs");
var path = require("path");
var By = webdriver.By;
var until = webdriver.until;

var BASE_URL = process.env.BASE_URL || "http://localhost:3000";
var BROWSER = process.env.BROWSER || "chrome";
var HEADLESS = process.env.HEADLESS === "true" || process.env.CI === "true";
var RESULTS_DIR = path.join(__dirname, "results");
var SCREENSHOTS_DIR = path.join(__dirname, "screenshots");

// Ensure output directories exist
[RESULTS_DIR, SCREENSHOTS_DIR].forEach(function (dir) {
  if (!fs.existsSync(dir)) {
    fs.mkdirSync(dir, { recursive: true });
  }
});

function createDriver() {
  var builder = new webdriver.Builder();

  if (BROWSER === "chrome") {
    var opts = new chrome.Options();
    if (HEADLESS) {
      opts.addArguments("--headless=new", "--no-sandbox", "--disable-dev-shm-usage", "--window-size=1920,1080");
    }
    builder.forBrowser("chrome").setChromeOptions(opts);
  } else if (BROWSER === "firefox") {
    var fOpts = new firefox.Options();
    if (HEADLESS) {
      fOpts.addArguments("--headless", "--width=1920", "--height=1080");
    }
    builder.forBrowser("firefox").setFirefoxOptions(fOpts);
  }

  return builder.build();
}

function TestRunner() {
  this.driver = null;
  this.results = [];
  this.startTime = null;
}

TestRunner.prototype.setup = function () {
  this.driver = createDriver();
  this.startTime = Date.now();
  console.log("Browser: " + BROWSER + (HEADLESS ? " (headless)" : ""));
  console.log("Base URL: " + BASE_URL);
  return Promise.resolve();
};

TestRunner.prototype.teardown = function () {
  var self = this;
  var duration = ((Date.now() - self.startTime) / 1000).toFixed(1);
  var passed = self.results.filter(function (r) { return r.status === "passed"; }).length;
  var failed = self.results.filter(function (r) { return r.status === "failed"; }).length;

  console.log("\n=== Results ===");
  console.log("Total: " + self.results.length + " | Passed: " + passed + " | Failed: " + failed);
  console.log("Duration: " + duration + "s");

  self.writeJUnitXml();

  if (self.driver) {
    return self.driver.quit().then(function () {
      return failed > 0 ? 1 : 0;
    });
  }
  return Promise.resolve(failed > 0 ? 1 : 0);
};

TestRunner.prototype.runTest = function (name, testFn) {
  var self = this;
  var start = Date.now();
  console.log("\n  Running: " + name);

  return testFn(self.driver)
    .then(function () {
      var duration = Date.now() - start;
      self.results.push({ name: name, status: "passed", duration: duration });
      console.log("  PASSED (" + duration + "ms)");
    })
    .catch(function (err) {
      var duration = Date.now() - start;
      self.results.push({
        name: name,
        status: "failed",
        duration: duration,
        error: err.message,
      });
      console.log("  FAILED: " + err.message);

      // Capture screenshot on failure
      var filename = name.replace(/[^a-zA-Z0-9]/g, "_") + ".png";
      return self.driver.takeScreenshot().then(function (data) {
        fs.writeFileSync(path.join(SCREENSHOTS_DIR, filename), data, "base64");
        console.log("  Screenshot: " + filename);
      }).catch(function () {
        // Ignore screenshot errors
      });
    });
};

TestRunner.prototype.writeJUnitXml = function () {
  var totalTime = ((Date.now() - this.startTime) / 1000).toFixed(3);
  var failures = this.results.filter(function (r) { return r.status === "failed"; }).length;

  var xml = '<?xml version="1.0" encoding="UTF-8"?>\n';
  xml += '<testsuites>\n';
  xml += '<testsuite name="Selenium - ' + BROWSER + '" tests="' + this.results.length +
    '" failures="' + failures + '" time="' + totalTime + '">\n';

  this.results.forEach(function (result) {
    var time = (result.duration / 1000).toFixed(3);
    xml += '  <testcase name="' + escapeXml(result.name) + '" classname="selenium.' +
      BROWSER + '" time="' + time + '">\n';
    if (result.status === "failed") {
      xml += '    <failure message="' + escapeXml(result.error) + '">' +
        escapeXml(result.error) + '</failure>\n';
    }
    xml += '  </testcase>\n';
  });

  xml += '</testsuite>\n</testsuites>\n';
  fs.writeFileSync(path.join(RESULTS_DIR, "junit-results.xml"), xml);
};

function escapeXml(str) {
  return String(str)
    .replace(/&/g, "&amp;")
    .replace(/</g, "&lt;")
    .replace(/>/g, "&gt;")
    .replace(/"/g, "&quot;");
}

// Define tests
var runner = new TestRunner();

runner.setup()
  .then(function () {
    return runner.runTest("Homepage loads successfully", function (driver) {
      return driver.get(BASE_URL).then(function () {
        return driver.wait(until.titleContains("Grizzly"), 10000);
      }).then(function () {
        return driver.getTitle();
      }).then(function (title) {
        if (!title) {
          throw new Error("Page title is empty");
        }
      });
    });
  })
  .then(function () {
    return runner.runTest("Navigation links are visible", function (driver) {
      return driver.get(BASE_URL).then(function () {
        return driver.wait(until.elementLocated(By.css("nav")), 10000);
      }).then(function () {
        return driver.findElements(By.css("nav a"));
      }).then(function (links) {
        if (links.length < 3) {
          throw new Error("Expected at least 3 nav links, found " + links.length);
        }
      });
    });
  })
  .then(function () {
    return runner.runTest("Articles page loads and displays articles", function (driver) {
      return driver.get(BASE_URL + "/articles").then(function () {
        return driver.wait(until.elementLocated(By.css(".article-card, .blog-card, article")), 15000);
      }).then(function () {
        return driver.findElements(By.css(".article-card, .blog-card, article"));
      }).then(function (articles) {
        if (articles.length === 0) {
          throw new Error("No articles found on the page");
        }
      });
    });
  })
  .then(function () {
    return runner.runTest("Contact page form is functional", function (driver) {
      return driver.get(BASE_URL + "/contact").then(function () {
        return driver.wait(until.elementLocated(By.css("form")), 10000);
      }).then(function () {
        return driver.findElement(By.css('input[name="name"], input[name="email"], input[type="email"]'));
      }).then(function (el) {
        return el.isDisplayed();
      }).then(function (displayed) {
        if (!displayed) {
          throw new Error("Contact form fields are not visible");
        }
      });
    });
  })
  .then(function () {
    return runner.teardown();
  })
  .then(function (exitCode) {
    process.exit(exitCode);
  })
  .catch(function (err) {
    console.error("Fatal error: " + err.message);
    process.exit(1);
  });

Running locally:

$ BROWSER=chrome HEADLESS=true node full-test-suite.js
Browser: chrome (headless)
Base URL: http://localhost:3000

  Running: Homepage loads successfully
  PASSED (1240ms)

  Running: Navigation links are visible
  PASSED (890ms)

  Running: Articles page loads and displays articles
  PASSED (2100ms)

  Running: Contact page form is functional
  PASSED (1050ms)

=== Results ===
Total: 4 | Passed: 4 | Failed: 0
Duration: 5.3s

Common Issues and Troubleshooting

ChromeDriver Version Mismatch

Error: SessionNotCreatedError: session not created: This version of ChromeDriver
only supports Chrome version 120. Current browser version is 121.0.6167.85

This happens when the chromedriver npm package version does not match the Chrome browser version on the pipeline agent. Azure-hosted agents update Chrome regularly. Fix by either: (a) removing the chromedriver dependency and relying on Selenium Manager (Selenium 4.6+) for automatic driver management, or (b) using npx chromedriver --version to detect the installed Chrome version and installing the matching driver dynamically in your pipeline.

Chrome Crashes on Linux Pipeline Agents

Error: WebDriverError: unknown error: Chrome failed to start: crashed.
  (chrome not reachable)

Add --no-sandbox and --disable-dev-shm-usage to Chrome options. The sandbox requires kernel capabilities not available in containerized pipeline agents, and the default shared memory partition is too small for Chrome. Also verify that Chrome is actually installed on the agent -- Microsoft-hosted ubuntu-latest agents include Chrome, but self-hosted agents may not.

Elements Not Found Despite Being Visible

Error: NoSuchElementError: no such element: Unable to locate element: {"method":"css selector","selector":"#submit-btn"}

The element has not loaded when Selenium tries to find it. Never rely on driver.findElement() alone for dynamic content. Always use driver.wait(until.elementLocated(...)) with an explicit timeout. Also check if the element is inside an iframe -- Selenium cannot find elements in iframes without first switching to the iframe context with driver.switchTo().frame().

Tests Pass Locally But Fail in Pipeline

Differences between local and pipeline environments: (a) screen resolution -- headless Chrome defaults to 800x600, add --window-size=1920,1080, (b) font rendering -- affects screenshot comparisons, (c) timing -- pipeline agents may be slower than your local machine, increase wait timeouts, (d) network -- the application URL must be reachable from the pipeline agent.

Firefox geckodriver Not Found

Error: The geckodriver executable could not be found on the current PATH

Install geckodriver via npm (npm install geckodriver) or use Selenium Manager. On Microsoft-hosted agents, Firefox is pre-installed but geckodriver may not be on the PATH. Add a pipeline step to verify: geckodriver --version || npx geckodriver --version.

Best Practices

  • Always run headless in CI. GUI mode requires a display server (Xvfb on Linux) and is slower. Headless mode is faster, more reliable, and does not require display configuration. Use --headless=new for Chrome (the new headless mode is more compatible than the old --headless flag).

  • Capture screenshots on every failure. Publish screenshots as build artifacts. A failing test with a screenshot is debuggable; a failing test with only a stack trace requires reproducing the issue locally, which wastes developer time.

  • Use explicit waits everywhere. Never use driver.sleep() or implicit waits. Explicit waits poll for a specific condition and return as soon as the condition is met, making tests both faster and more reliable.

  • Implement the Page Object pattern. Page objects encapsulate page interactions, making tests readable and maintenance manageable. When a page's HTML structure changes, you update one page object instead of every test that touches that page.

  • Set continueOnError: true on the test step. This allows the publish and artifact steps to run even when tests fail. Without it, a test failure skips result publishing, and you lose the diagnostic data you need to debug the failure.

  • Keep Selenium tests focused on critical paths. Selenium tests are slow compared to unit and API tests. Use them for critical user workflows (login, checkout, signup) and visual validation. Do not test business logic with Selenium -- use faster test types for that.

  • Run cross-browser tests on merge, not on every PR. The matrix strategy multiplies build time by the number of browsers. Run Chrome-only tests on PR builds for fast feedback, and run the full browser matrix on merge to main.

  • Pin browser versions in self-hosted agents. Microsoft-hosted agents update browsers automatically, which is convenient but can break tests unexpectedly. For self-hosted agents, pin the browser version and update deliberately.

References

Powered by Contentful