Visual Tests

No Storybook tutorial would be complete without testing. Testing is essential to creating high-quality UIs. In modular systems, minuscule tweaks can result in major regressions. So far, we have encountered three types of tests:

• Manual tests rely on developers to manually look at a component to verify it for correctness. They help us sanity check a component’s appearance as we build.

• Accessibility tests with a11y addon verify that the component is accessible to everyone. They're great for allowing us to collect information about how people with certain types of disabilities use our components.

• Component tests with the play function verify that the component behaves as expected when interacting with it. They're great for testing the behavior of a component when it's in use.

“But does it look right?”

Unfortunately, the aforementioned testing methods alone aren’t enough to prevent UI bugs. UIs are tricky to test because design is subjective and nuanced. Manual tests are, well, manual. Other UI tests, such as snapshot tests, trigger too many false positives, and pixel-level unit tests are poorly valued. A complete Storybook testing strategy also includes visual regression tests.

Visual testing for Storybook

Visual regression tests, also called visual tests, are designed to catch changes in appearance. They work by capturing screenshots of every story and comparing them commit-to-commit to surface changes. It's perfect for verifying graphical elements like layout, color, size, and contrast.

Storybook is a fantastic tool for visual regression testing because every story is essentially a test specification. Each time we write or update a story, we get a spec for free!

There are several tools for visual regression testing. We recommend Chromatic, a free publishing service made by the Storybook maintainers that runs visual tests in a lightning-fast cloud browser environment. It also allows us to publish Storybook online, as we saw in the previous chapter.

Catch a UI change

Visual regression testing relies on comparing images of the newly rendered UI code to the baseline images. If a UI change is caught, we'll get notified.

Let's see how it works by tweaking the background of the Task component.

Start by creating a new branch for this change:

git checkout -b change-task-background

Change src/components/Task.vue to the following:

<template>
  <div :class="classes">
    <label
      :for="'checked' + task.id"
      :aria-label="'archiveTask-' + task.id"
      class="checkbox"
    >
      <input
        type="checkbox"
        :checked="isChecked"
        disabled
        :name="'checked' + task.id"
        :id="'archiveTask-' + task.id"
      />
      <span class="checkbox-custom" @click="archiveTask" />
    </label>
    <label :for="'title-' + task.id" :aria-label="task.title" class="title">
      <input
        type="text"
        readonly
        :value="task.title"
        :id="'title-' + task.id"
        name="title"
        placeholder="Input title"
+       style="background-color: red" />
      />
    </label>
    <button
      v-if="!isChecked"
      class="pin-button"
      @click="pinTask"
      :id="'pinTask-' + task.id"
      :aria-label="'pinTask-' + task.id"
    >
      <span class="icon-star" />
    </button>
  </div>
</template>

This yields a new background color for the item.

Add the file:

git add .

Commit it:

git commit -m "change task background to red"

And push the changes to the remote repo:

git push -u origin change-task-background

Finally, open your GitHub repository and open a pull request for the change-task-background branch.

Add a descriptive text to your pull request and click Create pull request. Click on the "🟡 UI Tests" PR check at the bottom of the page.

It will show you the UI changes caught by your commit.

There are a lot of changes! The component hierarchy where Task is a child of TaskList and Inbox means one small tweak snowballs into major regressions. This circumstance is precisely why developers need visual regression testing in addition to other testing methods.

Review changes

Visual regression testing ensures components don’t change by accident. But it’s still up to us to determine whether changes are intentional or not.

If a change is intentional, we'll need to update the baseline to compare future tests to the latest version of the story. If a change is unintentional, it needs to be fixed.

Since modern apps are constructed from components, it’s important that we test at the level of the component. Doing so helps us pinpoint the root cause of a change, the component, instead of reacting to symptoms of a change: the screens and composite components.

Merge changes

When we’ve finished reviewing, we’re ready to merge UI changes with confidence--knowing that updates won’t accidentally introduce bugs. Accept the changes if you like the new red background. If not, revert to the previous state.

Storybook helps us build components; testing helps us maintain them. This tutorial covers four types of UI testing: manual, accessibility, interaction, and visual regression. You can automate the last three by adding them to a CI as we've just finished setting up, and it helps us ship components without worrying about stowaway bugs. The whole workflow is illustrated below.