Skip to main content

Testing Lighthouse scores and page performance in Azure DevOps pipelines

This guide explains how to run DebugBear tests in CI using Azure DevOps.

Prerequisites​

Before getting started makes sure that:

If you are not able to deploy your branch consider using ngrok and the --ngrokWebPort 4040 option of the DebugBear CLI.

Creating a pipeline​

Open the Pipelines tab of your project on Azure DevOps, then click New pipeline.

Create Azure DevOps pipeline

Select the type of repo and the repo you want to use.

Select repo

Then click Starter pipeline

Create starter pipeline

Setting up the DEBUGBEAR_API_KEY environment variable​

Click Variables and then New Variable.

Add Azure DevOps environment variable

Set the variable name to DEBUGBEAR_API_KEY and enter the API key obtained from the Project API tab on the DebugBear website.

Configure Azure DevOps environment variable

Running DebugBear in the pipeline​

Update the pipeline YAML code as follows. Replace the page ID and URL to match your project.

trigger:
 branches:
    include:
    - '*' # Run on every branch

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    npm install debugbear
    ./node_modules/.bin/debugbear \
      --pageId=123 \
      --inferBuildInfo \
      --waitForResult \
      --baseBranch=master \
      --fail \
      https://my-branch.example.com/page.html
  displayName: 'Run DebugBear test'

This code assumes that your source code already contains an npm package. If that isn't the case you need to run npm init -y before running npm install debugbear.

The following options are passed to the DebugBear command-line interface:

  • pageId – Tells DebugBear which page in your project to test
  • inferBuildInfo – Pick up details like commit hash and message from the CI environment
  • waitForResult – Wait until the test is complete before exiting the CLI
  • fail – If the test status is failure then exit with a non-zero exit code, failing the pipeline
  • https://my-branch.example.com/ – override the default test URL with the URL where the current branch is deployed

Once the test is complete DebugBear will print a summary of changes compared to the base branch, or compared to the scheduled tests if no build exists on the branch.

DebugBear CLI output in pipeline

Failing the build if a performance budget is exceeded​

By default all tests on DebugBear finish with a neutral status. Set up performance budgets to get success and failure statuses and fail the DevOps pipeline.

DebugBear performance budget to fail builds

The Azure DevOps pipeline will now fail if the budget is exceeded.

Failed pipeline based on web performance

Last updated on