Skip to content

web: enable cross-theme screenshots in Chromatic

Warren Gifford requested to merge vb/chromatic-cross-theme-screenshots into main

Created by: valerybugakov

Changes

Added three more variants of each existing story to test cross-theme visual regressions:

  1. Default theme + Dark mode.
  2. Redesign theme + Light mode.
  3. Redesign theme + Dark mode.

They are not displayed in the deployed version of Storybook. It's possible to explore how components look with different themes by using toolbar toggles. Stories added are meant for Chromatic snapshots only.

Closes #19955 (closed).

Approach

  1. Additional story file Chromatic.story is added along with a separate story glob specific to this file.
  2. Logic in Chromatic.story is wrapped in isChromatic() check to avoid applying it in other environments.
  3. When Storybook executes this file, all our stories are already added to the Storybook internal store. We create three more variations of each of the stories by wrapping them into a decorator function which takes care of CSS classes toggling based on received theme options.

As a result, we have four variants of each story which are different only by CSS classes applied to <body>. Any new story added to the Storybook will be captured in Chromatic in all these variants.

Alternatives

I've explored ways to achieve the same result by adding a custom decorator or addon.

  1. With a global decorator, it's possible to receive all the stories and add variations to the store on the first render call. Still, then the decorator will be executed for each story without any value.
  2. With a custom addon, it wasn't possible to reliably receive all the decorated stories. StoryStore is not available on window for addons. Channels/events usage allows receiving only serialized info about added stories, which means that we would need to create storyFn from it manually, which is error-prone.

Pricing

This PR quadruples the number of snapshots we take in Chromatic. It's temporary; after the redesign release, we will cut this number by half, but still, even without cross-theme screenshots, we exceed the number of screenshots per month by five times.

Sourcegraph has used 824,335 snapshots so far this month. Sourcegraph's plan includes 150,000 snapshots a month.

We're already on the most expensive plan publicly available. Is it time to explore enterprise options that Chromatic can offer to us?

Pricing breakdown

This month we created 850k snapshots:

  • 649$ for 150k snapshots included;
  • 5600$ for 700k extra snapshots, each for 0.008$;

After the PR, we will have around 3400k snapshots per month:

  • 649$ for 150k snapshots included;
  • 26000$ for 3250k extra snapshots, each for 0.008$;

After the redesign release in June, we will cut the number of snapshots by half:

  • 649$ for 150k snapshots included;
  • 12400$ for 1550k extra snapshots, each for 0.008$;

Merge request reports