Test runner

Storybook test runner turns all of your stories into executable tests. It is powered by Jest and Playwright.

• For those without a play function: it verifies whether the story renders without any errors.
• For those with a play function: it also checks for errors in the play function and that all assertions passed.

These tests run in a live browser and can be executed via the command line or your CI server.

Setup

The test-runner is a standalone, framework-agnostic utility that runs parallel to your Storybook. You will need to take some additional steps to set it up properly. Detailed below is our recommendation to configure and execute it.

Run the following command to install it.

npm install @storybook/test-runner --save-dev

Update your package.json scripts and enable the test runner.

{
  "scripts": {
    "test-storybook": "test-storybook"
  }
}

Start your Storybook with:

npm run storybook

Finally, open a new terminal window and run the test-runner with:

npm run test-storybook

Configure

Test runner offers zero-config support for Storybook. However, you can run test-storybook --eject for more fine-grained control. It generates a test-runner-jest.config.js file at the root of your project, which you can modify. Additionally, you can extend the generated configuration file and provide testEnvironmentOptions as the test runner also uses jest-playwright under the hood.

CLI Options

The test-runner is powered by Jest and accepts a subset of its CLI options (for example, --watch, --maxWorkers). If you're already using any of those flags in your project, you should be able to migrate them into Storybook's test-runner without any issues. Listed below are all the available flags and examples of using them.

npm run test-storybook -- --watch

Run tests against a deployed Storybook

By default, the test-runner assumes that you're running it against a locally served Storybook on port 6006. If you want to define a target URL to run against deployed Storybooks, you can use the --url flag:

npm run test-storybook -- --url https://the-storybook-url-here.com

Alternatively, you can set the TARGET_URL environment variable and run the test-runner:

TARGET_URL=https://the-storybook-url-here.com yarn test-storybook

Set up CI to run tests

You can also configure the test-runner to run tests on a CI environment. Documented below are some recipes to help you get started.

Run against deployed Storybooks via Github Actions deployment

If you're publishing your Storybook with services such as Vercel or Netlify, they emit a deployment_status event in GitHub Actions. You can use it and set the deployment_status.target_url as the TARGET_URL environment variable. Here's how:

name: Storybook Tests
on: deployment_status
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - name: Install dependencies
        run: yarn
      - name: Install Playwright
        run: npx playwright install --with-deps
      - name: Run Storybook tests
        run: yarn test-storybook
        env:
          TARGET_URL: '${{ github.event.deployment_status.target_url }}'

Run against non-deployed Storybooks

You can use your CI provider (for example, GitHub Actions, GitLab Pipelines, CircleCI) to build and run the test runner against your built Storybook. Here's a recipe that relies on third-party libraries, that is to say, concurrently, http-server, and wait-on to build Storybook and run tests with the test-runner.

name: 'Storybook Tests'
on: push
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - name: Install dependencies
        run: yarn
      - name: Install Playwright
        run: npx playwright install --with-deps
      - name: Build Storybook
        run: yarn build-storybook --quiet
      - name: Serve Storybook and run tests
        run: |
          npx concurrently -k -s first -n "SB,TEST" -c "magenta,blue" \
            "npx http-server storybook-static --port 6006 --silent" \
            "npx wait-on tcp:6006 && yarn test-storybook"

What's the difference between Chromatic and Test runner?

The test-runner is a generic testing tool that can run locally or on CI and be configured or extended to run all kinds of tests.

Chromatic is a cloud-based service that runs visual and interaction tests (and soon accessibility tests) without setting up the test runner. It also syncs with your Git provider and manages access control for private projects.

However, you might want to pair the test runner and Chromatic in some cases.

• Use it locally and Chromatic on your CI.
• Use Chromatic for visual and interaction tests and run other custom tests using the test runner.

Advanced configuration

Test hook API (experimental)

The test-runner renders a story and executes its play function if one exists. However, certain behaviors are impossible to achieve via the play function, which executes in the browser. For example, if you want the test-runner to take visual snapshots for you, this is possible via Playwright/Jest but must be executed in Node.

The test-runner exports test hooks that can be overridden globally to enable use cases like visual or DOM snapshots. These hooks give you access to the test lifecycle before and after the story is rendered. Listed below are the available hooks and an overview of how to use them.

To enable the hooks API, you'll need to add a new configuration file inside your Storybook directory and set them up as follows:

import type { TestRunnerConfig } from '@storybook/test-runner';
 
const config: TestRunnerConfig = {
  // Hook that is executed before the test runner starts running tests
  setup() {
    // Add your configuration here.
  },
  /* Hook to execute before a story is rendered.
   * The page argument is the Playwright's page object for the story.
   * The context argument is a Storybook object containing the story's id, title, and name.
   */
  async preRender(page, context) {
    // Add your configuration here.
  },
  /* Hook to execute after a story is rendered.
   * The page argument is the Playwright's page object for the story
   * The context argument is a Storybook object containing the story's id, title, and name.
   */
  async postRender(page, context) {
    // Add your configuration here.
  },
};
 
module.exports = config;

When the test-runner executes, your existing tests will go through the following lifecycle:

• The setup function is executed before all the tests run.
• The context object is generated containing the required information.
• Playwright navigates to the story's page.
• The preRender function is executed.
• The story is rendered, and any existing play functions are executed.
• The postRender function is executed.

Helpers

The test-runner exports a few helpers that can be used to make your tests more readable and maintainable by accessing Storybook's internals (e.g., args, parameters). Listed below are the available helpers and an overview of how to use them.

import type { TestRunnerConfig } from '@storybook/test-runner';
 
import { getStoryContext } from '@storybook/test-runner';
 
const config: TestRunnerConfig = {
  // Hook that is executed before the test runner starts running tests
  setup() {
    // Add your configuration here.
  },
  /* Hook to execute before a story is rendered.
   * The page argument is the Playwright's page object for the story.
   * The context argument is a Storybook object containing the story's id, title, and name.
   */
  async preRender(page, context) {
    // Add your configuration here.
  },
  /* Hook to execute after a story is rendered.
   * The page argument is the Playwright's page object for the story
   * The context argument is a Storybook object containing the story's id, title, and name.
   */
  async postRender(page, context) {
    // Get the entire context of a story, including parameters, args, argTypes, etc.
    const storyContext = await getStoryContext(page, context);
 
    // Add your configuration here.
  },
};
 
module.exports = config;

Stories.json mode

The test-runner transforms your story files into tests when testing a local Storybook. For a remote Storybook, it uses the Storybook's stories.json file (a static index of all the stories) to run the tests.

Why?

Suppose you run into a situation where the local and remote Storybooks appear out of sync, or you might not even have access to the code. In that case, the stories.json file is guaranteed to be the most accurate representation of the deployed Storybook you are testing. To test a local Storybook using this feature, use the --stories-json flag as follows:

npm run test-storybook -- --stories-json

If you need to disable it, use the --no-stories-json flag:

npm run test-storybook -- --no-stories-json

How do I check if my Storybook has a stories.json file?

Stories.json mode requires a stories.json file. Open a browser window and navigate to your deployed Storybook instance (for example, https://your-storybook-url-here.com/stories.json). You should see a JSON file that starts with a "v": 3 key, immediately followed by another key called "stories", which contains a map of story IDs to JSON objects. If that is the case, your Storybook supports stories.json mode.

Troubleshooting

The test runner seems flaky and keeps timing out

If your tests time out with the following message:

Timeout - Async callback was not invoked within the 15000 ms timeout specified by jest.setTimeout

It might be that Playwright couldn't handle testing the number of stories you have in your project. Perhaps you have a large number of stories, or your CI environment has a really low RAM configuration. In such cases, you should limit the number of workers that run in parallel by adjusting your command as follows:

{
  "scripts": {
    "test-storybook:ci": "yarn test-storybook --maxWorkers=2"
  }
}

The error output in the CLI is too short

By default, the test runner truncates error outputs at 1000 characters, and you can check the full output directly in Storybook in the browser. However, if you want to change that limit, you can do so by setting the DEBUG_PRINT_LIMIT environment variable to a number of your choosing, for example, DEBUG_PRINT_LIMIT=5000 yarn test-storybook.

Run the test runner in other CI environments

As the test runner is based on Playwright, you might need to use specific docker images or other configurations depending on your CI setup. In that case, you can refer to the Playwright CI docs for more information.

Learn about other UI tests

• Test runner to automate test execution
• Visual tests for appearance
• Accessibility tests for accessibility
• Interaction tests for user behavior simulation
• Coverage tests for measuring code coverage
• Snapshot tests for rendering errors and warnings
• Import stories in other tests for other tools