New
Build your own Storybook GPTVisual test with Chromatic
Star81,174
Back to blog

Storybook Ecosystem CI

Protecting Storybook users from package upgrade headaches

loading
Michael Shilman
@mshilman

Upgrading npm packages can be a bit like crossing a minefield. Even a tiny patch upgrade can blow up your entire stack. Unfortunately, this kind of issue is common in modern open-source software. Packages often break due to bugs in their code, bugs in their dependencies, or incompatible transitive dependency upgrades.

As maintainers of Storybook—one of the most popular UI development tools—we're all too familiar with the terrors of package upgrades. That's why we've created Storybook Ecosystem CI, an early warning system to help protect our users from this chaotic environment.

  • 🚥 Public status page that reports across a matrix of officially supported configurations
  • ✅ Suite of tests that run on every commit
  • 🔔 Notification system to alert us of critical failures

Read on to learn more about what we're testing and how it's already paying dividends to improve Storybook's stability.

Wait, but why?

Storybook sits in the middle of one of the fastest-moving areas of software development, and our users expect us to work with the latest versions of everything.

The front-end ecosystem moves fast. Frameworks and libraries release regular updates, and new build tools constantly emerge. Keeping up is no small task due to the broad range of configurations that Storybook supports, combined with all its features.

Storybook has an enormous compatibility matrix

  • 10+ renderers: React, Vue, Angular, Web components, Svelte, Ember, HTML, and more
  • 2 languages: JS, TS
  • 2 builders: Webpack, Vite
  • 3 package managers: npm, yarn, pnpm
  • Meta-frameworks like NextJS, SvelteKit, etc.

Plus, various versions of all of the above (React 16+, Vue2/3, npm6/7/8, TS 3+ but also taking advantage of the latest 4.9 satisfies feature, and on and on! 😅)

Your project uses one combination of those options, e.g., NextJS + React + Webpack + yarn. Whereas, Storybook supports a combinatorial matrix of project configurations.

It's hard to make big changes to Storybook and ensure all project configurations continue to work. Storybook Ecosystem CI simplifies this process by testing this configuration matrix continuously and catching issues early. It is partially inspired by Vite’s Ecosystem CI, which is worth checking out.

Status page

The centerpiece of the Ecosystem CI is a public status page. A broken package is analogous to a “service outage” for users of that package, and we’re trying to maximize Storybook’s “uptime.” So we took design inspiration from popular status dashboards:

Each “heartbeat” bar in our visualization summarizes a daily CI run against an extensive test suite. Clicking on the bar shows which tests failed and gives you a link to see the full CI run for even more details.

As a Storybook user, or potential user evaluating Storybook for your project, you can see whether your configuration is supported and the recent stability of that configuration.

Sandbox testing

The way we test each of these configurations is by creating a sandbox. This is a new standalone project based on a generator script:

'nextjs/default-js': {
  script: 'yarn create next-app nextjs-default-js --javascript --eslint',
}

Our sandbox script then installs Storybook like a user would, adds a suite of stories, and runs a series of tests (Chromatic snapshots, Storybook test-runner and Playwright e2e).

When we converted our stories to Component Story Format 3 (CSF3), we found that we can write generic stories that don’t contain any framework/renderer-specific code. For example:

export default {
  // React/Vue/Angular/etc. component
  component: globalThis.components.Button,
}

export const Actions = {
  play: async ({ args, canvasElement }) => {
    await fireEvent.click(within(canvasElement).getByRole('button'));
    await expect(args.onSuccess).toHaveBeenCalled();
  },
};

This allows us to scalably test new frameworks (React, Vue, Angular, Web-components, Svelte, etc.) without having to duplicate code/work for each additional configuration.

Notification system

We test so many configurations so that we can be notified as soon as they break, whether due to changes in Storybook or updates in the broader ecosystem. Inspired by Vite’s own ecosystem CI, we’ve implemented a notification system that posts to the Storybook Discord whenever CI breaks:

Early wins

Ecosystem CI has already caught multiple breakages in the weeks that it’s been deployed. With Ecosystem CI, we had failing reproductions in CI and a process for resolving the issues. In most cases, we fixed the breakage within 24 hours.

Previously, these breakages would’ve been reported by the community via GitHub issues, often without reproductions, and it would take a few days to a week to resolve them in an ad-hoc manner.

Get involved

The frontend ecosystem is diverse and ever-changing and it’s hard to keep up. Storybook Ecosystem CI is our attempt to systematically tackle this problem, to create the best experience for our users. It was created by Michael Arestad, Yann Braga, Tom Coleman, Norbert de Langen, Kasper Peulen, Michael Shilman (me!), and the rest of the Storybook team.

Ecosystem CI is currently running against Storybook 7.0 beta. Try out the beta in your project with npx storybook@next init, or upgrade an existing Storybook by following the 7.0 migration guide. Join us on GitHub or chat with us on Discord. Finally, follow @storybookjs on Twitter for the latest news.

Join the Storybook mailing list

Get the latest news, updates and releases

6,388 developers and counting

We’re hiring!

Join the team behind Storybook and Chromatic. Build tools that are used in production by 100s of thousands of developers. Remote-first.

View jobs

Popular posts

Vuetify in Storybook

Get the most out of Vuetify in Storybook with out of the box Vite support
loading
Shaun Evening
Join the community
6,388 developers and counting
WhyWhy StorybookComponent-driven UI
Open source software
Storybook

Maintained by
Chromatic
Special thanks to Netlify and CircleCI