New
I18n with StorybookAutomate with Chromatic
Star74,085
Back to UI Testing Handbook
React
Chapters
  • Introduction
  • Visual
  • Composition
  • Interaction
  • Accessibility
  • User flow
  • Automate
  • Workflow
  • Conclusion

How to automate UI tests with Github Actions

Speed up your workflow and ship higher quality of code

Developers spend 4-8 hrs a week fixing bugs. Things only get worse if a bug sneaks its way into production. It takes 5-10x longer to fix it. That's why UI testing is integral to delivering high-quality experiences, but it can also be a huge time sink. It's too much work to run all your tests manually after every change.

You can automate your workflow to trigger tests when a developer pushes code. The tests execute in the background and report results on completion. That allows you to detect regressions automatically.

This chapter shows you how to implement such a workflow with Github Actions. Along the way, I'll point out ways to optimize your test runs.

Continuous UI testing

Reviewing code is a big part of being a developer. It helps catch bugs early and maintains high code quality.

To ensure that a pull request (PR) won't break production, you’d typically pull code and run the test suite locally. That disrupts your workflow and takes a lot of time. With Continuous Integration (CI), you get all the benefits of testing without any manual intervention.

You can tweak the UI, build a new feature, or update a dependency. When you open a pull request, the CI server will automatically run comprehensive UI tests—visual, composition, accessibility, interaction and user flows.

You’ll get test results via PR badges, which provide a summary of all the checks.

At a glance, you can tell if the pull request passed all quality checks. If yes, move on to reviewing the actual code. If not, dive into the logs to find out what’s wrong.

"Testing gives me full confidence for automated dependency updates. If tests pass, we merge them in."

Simon Taggart, Principal Engineer at Twilio

Tutorial

The previous five chapters demonstrated how to test the various aspects of the Taskbox UI. Building on that, we’ll set up continuous integration using GitHub Actions.

Set up CI

Create a .github/workflows/ui-tests.yml file in your repository to get started. A workflow is a set of jobs that you want to automate. It is triggered by events such as pushing a commit or creating a pull request.

Our workflow will run when code is pushed to any branch of our repository and it’ll have three jobs:

  • Run interaction and the accessibility tests with the Storybook test runner
  • Run visual and composition tests with Chromatic
  • Run user flow tests with Cypress
Copy
.github/workflows/ui-tests.yml
name: 'UI Tests'

on: push

jobs:
  # Run interaction and accessibility tests
  interaction-and-accessibility:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - name: Install dependencies
        run: yarn
      - name: Install Playwright
        run: npx playwright install --with-deps
      - name: Build Storybook
        run: yarn build-storybook --quiet
      - name: Serve Storybook and run tests
        run: |
          npx concurrently -k -s first -n "SB,TEST" -c "magenta,blue" \
            "npx http-server storybook-static --port 6006 --silent" \
            "npx wait-on tcp:6006 && yarn test-storybook"
  # Run visual and composition tests with Chromatic
  visual-and-composition:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0 # Required to retrieve git history
      - name: Install dependencies
        run: yarn
      - name: Publish to Chromatic
        uses: chromaui/action@v1
        with:
          # Grab this from the Chromatic manage page
          projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
  # Run user flow tests with Cypress
  user-flow:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install dependencies
        run: yarn
      - name: Cypress run
        uses: cypress-io/github-action@v2
        with:
          start: npm start

Couple of things to note here. For the test runner, we're using a combination of concurrently, http-server and wait-on libraries to build and serve the Storybook to run tests against it.

And to run Chromatic, you’ll need the CHROMATIC_PROJECT_TOKEN. You can grab it from the Chromatic manage page and add it to your repository secrets.

get project token from Chromatic add secret to your repository

Finally, create a new commit, push your changes to GitHub, and you should see your workflow in action!

Cache dependencies

Each job runs independently, which means the CI server has to install dependencies in all three jobs. That slows down the test run. We can cache dependencies and only run yarn install if the lock file changes to avoid that. Let’s update the workflow to include the install-cache job.

Copy
.github/workflows/ui-tests.yml
name: 'UI Tests'

on: push

jobs:
  # Install and cache npm dependencies
  install-cache:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Commit
        uses: actions/checkout@v2
      - name: Cache yarn dependencies and cypress
        uses: actions/cache@v2
        id: yarn-cache
        with:
          path: |
            ~/.cache/Cypress
            node_modules
          key: ${{ runner.os }}-yarn-v1-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-v1
      - name: Install dependencies if cache invalid
        if: steps.yarn-cache.outputs.cache-hit != 'true'
        run: yarn
  # Run interaction and accessibility tests
  interaction-and-accessibility:
    runs-on: ubuntu-latest
    needs: install-cache
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - name: Restore yarn dependencies
        uses: actions/cache@v2
        id: yarn-cache
        with:
          path: |
            ~/.cache/Cypress
            node_modules
          key: ${{ runner.os }}-yarn-v1-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-v1
     - name: Install Playwright
       run: npx playwright install --with-deps
     - name: Build Storybook
       run: yarn build-storybook --quiet
     - name: Serve Storybook and run tests
       run: |
         npx concurrently -k -s first -n "SB,TEST" -c "magenta,blue" \
           "npx http-server storybook-static --port 6006 --silent" \
           "npx wait-on tcp:6006 && yarn test-storybook"
  # Run visual and composition tests with Chromatic
  visual-and-composition:
    runs-on: ubuntu-latest
    needs: install-cache
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0 # Required to retrieve git history
      - name: Restore yarn dependencies
        uses: actions/cache@v2
        id: yarn-cache
        with:
          path: |
            ~/.cache/Cypress
            node_modules
          key: ${{ runner.os }}-yarn-v1-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-v1
      - name: Publish to Chromatic
        uses: chromaui/action@v1
        with:
          # Grab this from the Chromatic manage page
          projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
  # Run user flow tests with Cypress
  user-flow:
    runs-on: ubuntu-latest
    needs: install-cache
    steps:
      - uses: actions/checkout@v2
      - name: Restore yarn dependencies
        uses: actions/cache@v2
        id: yarn-cache
        with:
          path: |
            ~/.cache/Cypress
            node_modules
          key: ${{ runner.os }}-yarn-v1-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-v1
      - name: Cypress run
        uses: cypress-io/github-action@v2
        with:
          start: npm start

We also tweaked the other three jobs to wait for the install-cache job to complete to use the cached dependencies. Push another commit to re-run the workflow.

Success! You’ve automated your testing workflow. When you open up a PR it’ll run the test runner, Chromatic and Cypress in parallel and display the results on the PR page itself.

Mastering the UI testing workflow

The testing workflow starts by isolating components using Storybook. Then run checks while you code to get a faster feedback loop. Finally, execute your entire test suite using continuous integration.

Chapter 8 illustrates this complete workflow in action. We'll see how to test a new feature before shipping it to prod.

Keep your code in sync with this chapter. View 8f0a4cf on GitHub.
Is this free guide helping you? Tweet to give kudos and help other devs find it.
Next Chapter
Workflow
A testing workflow that doesn’t slow you down
✍️ Edit on GitHub – PRs welcome!
Join the community
5,772 developers and counting
WhyWhy StorybookComponent-driven UI
Open source software
Storybook

Maintained by
Chromatic
Special thanks to Netlify and CircleCI