Skip to main content
VertaaUX Docs
CLI

CI/CD Integration

Integrate VertaaUX audits into your CI/CD pipeline

CI/CD Integration

Automate UX audits in your CI/CD pipeline to catch regressions before they reach production. The CLI is designed for CI environments with exit codes, structured output, and baseline comparisons.

Why CI/CD Integration?

BenefitDescription
Catch regressionsFail builds when UX score drops
Prevent new issuesBlock PRs that introduce errors
Track improvementsCompare against baselines over time
Team accountabilityVisibility into UX quality trends

Quick Start

The simplest CI integration:

# In any CI system
- run: npm install -g @vertaaux/cli
- run: vertaa audit -u $PREVIEW_URL --fail-on error
  env:
    VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}

GitHub Actions

Basic Workflow

# .github/workflows/ux-audit.yml
name: UX Audit

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install VertaaUX CLI
        run: npm install -g @vertaaux/cli

      - name: Run UX Audit
        run: vertaa audit -u ${{ vars.PREVIEW_URL }} --fail-on error
        env:
          VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}

With Preview Deployments (Vercel)

name: UX Audit

on:
  deployment_status:

jobs:
  audit:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install VertaaUX CLI
        run: npm install -g @vertaaux/cli

      - name: Run UX Audit
        run: |
          vertaa audit \
            -u ${{ github.event.deployment_status.target_url }} \
            --fail-on error \
            --threshold 80 \
            --format json \
            -o results.json
        env:
          VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}

      - name: Upload Results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: ux-audit-results
          path: results.json

With PR Comments

name: UX Audit

on:
  pull_request:
    branches: [main]

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install VertaaUX CLI
        run: npm install -g @vertaaux/cli

      - name: Run UX Audit
        id: audit
        run: |
          vertaa audit \
            -u ${{ vars.PREVIEW_URL }} \
            --baseline baseline.json \
            --format json \
            -o results.json
        env:
          VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}
        continue-on-error: true

      - name: Generate PR Comment
        run: vertaa comment --file results.json --format github > comment.md

      - name: Post PR Comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const comment = fs.readFileSync('comment.md', 'utf8');
            github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: comment
            });

      - name: Check Audit Result
        if: steps.audit.outcome == 'failure'
        run: exit 1

With Baseline Management

name: UX Audit with Baseline

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install VertaaUX CLI
        run: npm install -g @vertaaux/cli

      - name: Run Audit with Baseline
        run: |
          vertaa audit \
            -u ${{ vars.PREVIEW_URL }} \
            --baseline baseline.json \
            --max-new-errors 0 \
            --max-new-warnings 5 \
            --fail-on error
        env:
          VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}

  # Update baseline on main branch pushes
  update-baseline:
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install VertaaUX CLI
        run: npm install -g @vertaaux/cli

      - name: Update Baseline
        run: |
          vertaa baseline \
            -u ${{ vars.PRODUCTION_URL }} \
            --update baseline.json
        env:
          VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}

      - name: Commit Baseline
        run: |
          git config user.name "GitHub Actions"
          git config user.email "actions@github.com"
          git add baseline.json
          git diff --staged --quiet || git commit -m "chore: update UX baseline"
          git push

GitLab CI

# .gitlab-ci.yml
stages:
  - test

ux-audit:
  stage: test
  image: node:20
  script:
    - npm install -g @vertaaux/cli
    - vertaa audit -u $CI_ENVIRONMENT_URL --fail-on error --threshold 80
  variables:
    VERTAA_API_KEY: $VERTAA_API_KEY
  only:
    - merge_requests
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.preview.example.com

With Artifacts

ux-audit:
  stage: test
  image: node:20
  script:
    - npm install -g @vertaaux/cli
    - vertaa audit -u $CI_ENVIRONMENT_URL --format json -o results.json
    - vertaa audit -u $CI_ENVIRONMENT_URL --format html -o report.html
  artifacts:
    paths:
      - results.json
      - report.html
    reports:
      junit: results.json
    expire_in: 1 week
  variables:
    VERTAA_API_KEY: $VERTAA_API_KEY

CircleCI

# .circleci/config.yml
version: 2.1

jobs:
  ux-audit:
    docker:
      - image: cimg/node:20.0
    steps:
      - checkout
      - run:
          name: Install VertaaUX CLI
          command: npm install -g @vertaaux/cli
      - run:
          name: Run UX Audit
          command: |
            vertaa audit \
              -u $PREVIEW_URL \
              --fail-on error \
              --threshold 80
          environment:
            VERTAA_API_KEY: $VERTAA_API_KEY
      - store_artifacts:
          path: results.json

workflows:
  ux-quality:
    jobs:
      - ux-audit:
          filters:
            branches:
              only:
                - main
                - /feature\/.*/

Azure DevOps

# azure-pipelines.yml
trigger:
  branches:
    include:
      - main

pr:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'

  - script: npm install -g @vertaaux/cli
    displayName: 'Install VertaaUX CLI'

  - script: |
      vertaa audit \
        -u $(PREVIEW_URL) \
        --fail-on error \
        --threshold 80 \
        --format junit \
        -o results.xml
    displayName: 'Run UX Audit'
    env:
      VERTAA_API_KEY: $(VERTAA_API_KEY)

  - task: PublishTestResults@2
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'results.xml'
    condition: always()

Jenkins

// Jenkinsfile
pipeline {
    agent any

    environment {
        VERTAA_API_KEY = credentials('vertaaux-api-key')
    }

    stages {
        stage('Install') {
            steps {
                sh 'npm install -g @vertaaux/cli'
            }
        }

        stage('UX Audit') {
            steps {
                sh '''
                    vertaa audit \
                        -u ${PREVIEW_URL} \
                        --fail-on error \
                        --threshold 80 \
                        --format json \
                        -o results.json
                '''
            }
            post {
                always {
                    archiveArtifacts artifacts: 'results.json'
                }
            }
        }
    }
}

Quality Gates

Exit Codes

CodeMeaningUse Case
0SuccessAll checks passed
1Issues found--fail-on triggered or --max-new-* exceeded
2ErrorNetwork error, invalid URL, auth failure
3Score below threshold--threshold value not met

Common Gate Configurations

Strict (production releases):

vertaa audit -u $URL \
  --fail-on error \
  --threshold 90 \
  --max-new-errors 0 \
  --max-new-warnings 0

Balanced (PR checks):

vertaa audit -u $URL \
  --fail-on error \
  --threshold 75 \
  --baseline baseline.json \
  --max-new-errors 0

Lenient (feature branches):

vertaa audit -u $URL \
  --fail-on error  # Only block on critical errors

Using with Preview Deployments

Vercel

# Wait for Vercel deployment
- name: Wait for Vercel
  uses: patrickedqvist/wait-for-vercel-preview@v1.3.1
  id: vercel
  with:
    token: ${{ secrets.GITHUB_TOKEN }}
    max_timeout: 300

- name: Run Audit
  run: vertaa audit -u ${{ steps.vercel.outputs.url }}

Netlify

- name: Wait for Netlify
  uses: jakepartusch/wait-for-netlify-action@v1.4
  id: netlify
  with:
    site_name: 'your-site-name'
    max_timeout: 300

- name: Run Audit
  run: vertaa audit -u ${{ steps.netlify.outputs.url }}

AWS Amplify

- name: Get Preview URL
  id: amplify
  run: |
    URL=$(aws amplify get-branch \
      --app-id ${{ vars.AMPLIFY_APP_ID }} \
      --branch-name ${{ github.head_ref }} \
      --query 'branch.displayName' \
      --output text)
    echo "url=https://$URL.${{ vars.AMPLIFY_APP_ID }}.amplifyapp.com" >> $GITHUB_OUTPUT

- name: Run Audit
  run: vertaa audit -u ${{ steps.amplify.outputs.url }}

Incremental Audits

For large sites, audit only changed routes:

vertaa audit \
  -u $PREVIEW_URL \
  --incremental \
  --baseline baseline.json

The --incremental flag detects changed files from Git and only audits affected routes.

Policy-Based CI

Use policies for branch-specific rules:

# vertaa.policy.yml
version: 1
branches:
  main:
    pattern: "^main$"
    assertions:
      fail_on: error
      overall_score: 85
      max_new_errors: 0
  feature:
    pattern: "^feature/.*$"
    assertions:
      fail_on: error
      overall_score: 70

Then simply run:

# Policy automatically applies based on current branch
vertaa audit -u $PREVIEW_URL

Structured Output

For CI parsing, use JSON logs:

vertaa audit -u $URL --json-logs --format json

Output includes machine-readable progress and results:

{"event":"start","timestamp":"2026-01-15T10:00:00Z","url":"https://example.com"}
{"event":"progress","percent":25,"pagesAudited":1,"totalPages":4}
{"event":"progress","percent":50,"pagesAudited":2,"totalPages":4}
{"event":"progress","percent":75,"pagesAudited":3,"totalPages":4}
{"event":"complete","score":85,"issues":{"error":2,"warning":5,"info":3}}

Caching

Speed up CI by caching the CLI:

GitHub Actions

- name: Cache CLI
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: vertaaux-cli-${{ runner.os }}

GitLab CI

cache:
  key: vertaaux-cli
  paths:
    - node_modules/

Troubleshooting

"Unauthorized" Error

Verify your API key is set correctly:

- run: echo "API key length: ${#VERTAA_API_KEY}"
  env:
    VERTAA_API_KEY: ${{ secrets.VERTAA_API_KEY }}

Timeout Issues

Increase timeout for slow preview deployments:

vertaa audit -u $URL --timeout 120000  # 2 minutes per page

Preview Not Ready

Add a wait step before auditing:

- name: Wait for Preview
  run: |
    for i in {1..30}; do
      curl -s -o /dev/null -w "%{http_code}" $PREVIEW_URL | grep -q "200" && exit 0
      sleep 10
    done
    exit 1

Policy Configuration

Branch-specific quality rules

Baseline Management

Track improvements over time

Audit Command

All audit options

Configuration

CLI defaults with .vertaaux.yml

Was this page helpful?

All Commands

Complete reference for all VertaaUX CLI commands

SDK Overview

Official TypeScript/JavaScript client for the VertaaUX API

On this page

CI/CD IntegrationWhy CI/CD Integration?Quick StartGitHub ActionsBasic WorkflowWith Preview Deployments (Vercel)With PR CommentsWith Baseline ManagementGitLab CIWith ArtifactsCircleCIAzure DevOpsJenkinsQuality GatesExit CodesCommon Gate ConfigurationsUsing with Preview DeploymentsVercelNetlifyAWS AmplifyIncremental AuditsPolicy-Based CIStructured OutputCachingGitHub ActionsGitLab CITroubleshooting"Unauthorized" ErrorTimeout IssuesPreview Not ReadyRelated