Continuous Testing in CI/CD Pipelines

Running tests locally is optional discipline. Running tests in CI is enforced discipline. The difference between a team that ships reliable software and one that does not is almost always the CI pipeline.

Continuous testing means every commit triggers automated tests. No code reaches production without passing unit tests, integration tests, linting, and whatever quality gates your team defines. This guide covers how to build testing pipelines that are fast, reliable, and actually useful — not just green checkmarks that nobody trusts.

Prerequisites

• A Git repository with tests (Jest, Mocha, or similar)
• GitHub, GitLab, or similar CI platform
• Understanding of testing fundamentals

Test Pipeline Architecture

A well-structured pipeline runs tests in stages from fastest to slowest:

Stage 1: Lint + Type Check     (10-30 seconds)
Stage 2: Unit Tests            (30-120 seconds)
Stage 3: Integration Tests     (1-5 minutes)
Stage 4: E2E Tests             (5-15 minutes)
Stage 5: Performance Tests     (5-30 minutes, optional)

Each stage is a gate. If linting fails, there is no reason to run unit tests. If unit tests fail, integration tests will likely fail too. This saves CI minutes and gives developers fast feedback.

GitHub Actions: Complete Pipeline

Basic Test Workflow

# .github/workflows/test.yml
name: Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npm run lint

  unit-tests:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npm test -- --coverage
      - uses: actions/upload-artifact@v4
        with:
          name: coverage-report
          path: coverage/

  integration-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
          POSTGRES_DB: testdb
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npm run test:integration
        env:
          DATABASE_URL: postgresql://test:test@localhost:5432/testdb

Matrix Testing Across Node Versions

unit-tests:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      node-version: [18, 20, 22]
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
        cache: "npm"
    - run: npm ci
    - run: npm test

Parallel Test Splitting

Split tests across multiple runners for faster feedback:

unit-tests:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      shard: [1, 2, 3, 4]
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: 20
        cache: "npm"
    - run: npm ci
    - run: npx jest --shard=${{ matrix.shard }}/4

GitLab CI: Complete Pipeline

# .gitlab-ci.yml
stages:
  - lint
  - test
  - integration
  - deploy

variables:
  NODE_VERSION: "20"

.node-cache:
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
    policy: pull

install:
  stage: .pre
  image: node:${NODE_VERSION}
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
    policy: push
  script:
    - npm ci

lint:
  stage: lint
  image: node:${NODE_VERSION}
  extends: .node-cache
  script:
    - npm run lint
  allow_failure: false

unit-tests:
  stage: test
  image: node:${NODE_VERSION}
  extends: .node-cache
  script:
    - npm test -- --coverage
  artifacts:
    reports:
      junit: junit.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
  coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'

integration-tests:
  stage: integration
  image: node:${NODE_VERSION}
  extends: .node-cache
  services:
    - postgres:15
  variables:
    POSTGRES_USER: test
    POSTGRES_PASSWORD: test
    POSTGRES_DB: testdb
    DATABASE_URL: postgresql://test:test@postgres:5432/testdb
  script:
    - npm run test:integration
  artifacts:
    reports:
      junit: integration-junit.xml

Dependency Caching

Caching node_modules or the npm cache dramatically speeds up pipelines.

GitHub Actions Caching

- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: "npm"  # Built-in caching

# Or manual caching for more control
- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

Caching Test Results

Cache expensive test setup (database migrations, compiled assets):

- uses: actions/cache@v4
  with:
    path: .jest-cache
    key: jest-${{ hashFiles('src/**/*.js') }}

- run: npm test -- --cacheDirectory=.jest-cache

Test Reporting

JUnit XML Reports

Most CI platforms parse JUnit XML for rich test reporting.

Jest:

npm install --save-dev jest-junit

{
  "scripts": {
    "test:ci": "jest --ci --reporters=default --reporters=jest-junit"
  },
  "jest-junit": {
    "outputDirectory": ".",
    "outputName": "junit.xml"
  }
}

Mocha:

npm install --save-dev mocha-junit-reporter

{
  "scripts": {
    "test:ci": "mocha --reporter mocha-junit-reporter --reporter-options mochaFile=junit.xml"
  }
}

GitHub Actions Test Summary

- run: npm run test:ci

- name: Test Report
  uses: dorny/test-reporter@v1
  if: success() || failure()
  with:
    name: Jest Tests
    path: junit.xml
    reporter: jest-junit

Coverage Reporting

- run: npm test -- --coverage

- name: Coverage Summary
  if: github.event_name == 'pull_request'
  run: |
    echo "## Coverage Report" >> $GITHUB_STEP_SUMMARY
    echo '```' >> $GITHUB_STEP_SUMMARY
    cat coverage/coverage-summary.json | node -e "
      var data = '';
      process.stdin.on('data', function(c) { data += c; });
      process.stdin.on('end', function() {
        var report = JSON.parse(data);
        var total = report.total;
        console.log('Lines:      ' + total.lines.pct + '%');
        console.log('Branches:   ' + total.branches.pct + '%');
        console.log('Functions:  ' + total.functions.pct + '%');
        console.log('Statements: ' + total.statements.pct + '%');
      });
    " >> $GITHUB_STEP_SUMMARY
    echo '```' >> $GITHUB_STEP_SUMMARY

Quality Gates

Required Status Checks

Configure GitHub branch protection to require passing checks:

1. Settings > Branches > Branch protection rules
2. Require status checks to pass before merging
3. Select the test jobs that must pass

Coverage Thresholds

Fail the build when coverage drops:

{
  "jest": {
    "coverageThreshold": {
      "global": {
        "branches": 80,
        "functions": 80,
        "lines": 80,
        "statements": 80
      }
    }
  }
}

Lint as a Gate

lint:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: 20
        cache: "npm"
    - run: npm ci
    - run: npm run lint
    # If lint fails, all downstream jobs are skipped

Handling Flaky Tests

Flaky tests — tests that sometimes pass and sometimes fail without code changes — erode trust in the pipeline.

Retry Strategy

# GitHub Actions — retry the entire job
unit-tests:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: 20
        cache: "npm"
    - run: npm ci
    - name: Run tests with retry
      uses: nick-fields/retry@v2
      with:
        timeout_minutes: 10
        max_attempts: 3
        command: npm test

Jest Retry Per-Test

// jest.config.js
module.exports = {
  // Retry failed tests up to 2 times
  // Only use this for known-flaky tests while fixing them
  retryTimes: process.env.CI ? 2 : 0
};

Quarantining Flaky Tests

// Tag flaky tests so they can be tracked and fixed
var flakyDescribe = process.env.SKIP_FLAKY ? describe.skip : describe;

flakyDescribe("WebSocket reconnection", function() {
  // This test is flaky due to timing issues
  // Tracked in issue #234
  test("reconnects after disconnect", function() {
    // ...
  });
});

# Run stable tests as a gate, flaky tests as informational
stable-tests:
  runs-on: ubuntu-latest
  steps:
    - run: SKIP_FLAKY=1 npm test

flaky-tests:
  runs-on: ubuntu-latest
  continue-on-error: true  # Do not block the pipeline
  steps:
    - run: npm test -- --testPathPattern="flaky"

Optimizing Pipeline Speed

Run Only Affected Tests

# Only test changed files
- name: Get changed files
  id: changed
  run: |
    echo "files=$(git diff --name-only ${{ github.event.before }} HEAD -- 'src/**/*.js' | tr '\n' ' ')" >> $GITHUB_OUTPUT

- name: Run affected tests
  run: npx jest --findRelatedTests ${{ steps.changed.outputs.files }}

Skip Tests for Documentation-Only Changes

on:
  push:
    paths-ignore:
      - "**.md"
      - "docs/**"
      - ".gitignore"
      - "LICENSE"

Conditional E2E Tests

Run expensive tests only when relevant code changes:

e2e-tests:
  needs: unit-tests
  if: |
    contains(github.event.head_commit.message, '[e2e]') ||
    github.event_name == 'push' && github.ref == 'refs/heads/main'
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - run: npm ci
    - run: npx playwright install --with-deps
    - run: npm run test:e2e

Database Testing in CI

PostgreSQL Service

services:
  postgres:
    image: postgres:15
    env:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: app_test
    ports:
      - 5432:5432
    options: >-
      --health-cmd pg_isready
      --health-interval 10s
      --health-timeout 5s
      --health-retries 5

steps:
  - run: npm ci
  - name: Run migrations
    run: npm run db:migrate
    env:
      DATABASE_URL: postgresql://test:test@localhost:5432/app_test
  - name: Run tests
    run: npm run test:integration
    env:
      DATABASE_URL: postgresql://test:test@localhost:5432/app_test

MongoDB Service

services:
  mongo:
    image: mongo:7
    ports:
      - 27017:27017

steps:
  - run: npm ci
  - run: npm run test:integration
    env:
      MONGO_URL: mongodb://localhost:27017/test

Redis Service

services:
  redis:
    image: redis:7
    ports:
      - 6379:6379
    options: >-
      --health-cmd "redis-cli ping"
      --health-interval 10s
      --health-timeout 5s
      --health-retries 5

Complete Pipeline Example

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  # Stage 1: Fast checks
  lint-and-typecheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck

  # Stage 2: Unit tests (parallel shards)
  unit-tests:
    needs: lint-and-typecheck
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npx jest --shard=${{ matrix.shard }}/3 --ci --coverage --reporters=default --reporters=jest-junit
        env:
          JEST_JUNIT_OUTPUT_DIR: ./reports
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results-shard-${{ matrix.shard }}
          path: |
            reports/
            coverage/

  # Stage 3: Integration tests
  integration-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
          POSTGRES_DB: app_test
        ports:
          - 5432:5432
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
      redis:
        image: redis:7
        ports:
          - 6379:6379
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npm run db:migrate
        env:
          DATABASE_URL: postgresql://test:test@localhost:5432/app_test
      - run: npm run test:integration
        env:
          DATABASE_URL: postgresql://test:test@localhost:5432/app_test
          REDIS_URL: redis://localhost:6379

  # Stage 4: E2E tests (only on main or when requested)
  e2e-tests:
    needs: integration-tests
    if: github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'run-e2e')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run build
      - run: npm run test:e2e
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: e2e-screenshots
          path: test-results/

  # Summary job — required for branch protection
  test-summary:
    needs: [lint-and-typecheck, unit-tests, integration-tests]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Check results
        run: |
          if [ "${{ needs.lint-and-typecheck.result }}" != "success" ] ||
             [ "${{ needs.unit-tests.result }}" != "success" ] ||
             [ "${{ needs.integration-tests.result }}" != "success" ]; then
            echo "One or more required jobs failed"
            exit 1
          fi
          echo "All required checks passed"

Common Issues and Troubleshooting

Tests pass locally but fail in CI

Environment differences: different Node version, missing environment variables, or different OS:

Fix: Match Node version in CI to local development. List all required environment variables in your pipeline config. Use npm ci instead of npm install in CI for deterministic installs.

Pipeline is too slow

Running all tests sequentially takes too long:

Fix: Split tests into parallel shards. Cache node_modules between runs. Skip expensive tests on non-critical branches. Use paths-ignore to skip pipelines for documentation changes.

Database tests fail intermittently in CI

The database service is not ready when tests start:

Fix: Add health checks to service containers. Wait for the database to be ready before running migrations. Add retry logic for initial database connections.

CI minutes are expensive

Testing every commit on every branch consumes resources:

Fix: Use concurrency groups to cancel redundant runs. Limit matrix testing to main branch pushes. Run E2E tests only on main or when explicitly requested. Use paths filters to skip unrelated changes.

Best Practices

• Fail fast. Run linting and type checking before tests. If the code does not compile or has style issues, there is no need to run the test suite.
• Cache aggressively. node_modules, test results, and build artifacts should be cached between pipeline runs. The first run is slow; subsequent runs should reuse cached data.
• Use npm ci in CI, not npm install. npm ci installs from package-lock.json exactly, ensuring deterministic builds. npm install can resolve to different versions.
• Report test results in CI-native format. JUnit XML, Cobertura coverage, and test summaries give developers rich feedback directly in the CI interface.
• Cancel redundant pipeline runs. When a developer pushes multiple commits quickly, cancel the older runs. Only the latest commit matters.
• Keep the main branch green. If tests fail on main, fixing them is the team's top priority. A broken main branch means nobody can deploy.
• Separate required and optional checks. Unit tests and linting should be required. Performance tests and E2E tests can be informational. Do not block merges on tests that are flaky or slow.
• Monitor pipeline duration. Track how long your pipeline takes over time. If it grows from 5 minutes to 15 minutes, investigate and optimize before developers start ignoring it.
• Test the pipeline itself. When you change CI configuration, verify the changes work. A broken pipeline that silently passes is worse than no pipeline at all.

References

• GitHub Actions Documentation
• GitLab CI/CD Documentation
• Jest CI Configuration
• jest-junit Reporter
• GitHub Branch Protection