A/B testing with Sanity and LaunchDarkly

We use Sanity to build fast websites that rank on the Google search engine and attract users.

We know that A/B testing is a key great way to find out what content fits better with your target users, so integrating it on Sanity is a great way to test your content with real users before going live with them.

This article will be a demo where we show how our clients usually create A/B tests from Sanity and control them from an A/B testing platform called LaunchDarkly.

You can also jump straight to the Youtube video if you prefer:

We have a homepage in a light theme, but we think that users might prefer a dark variation of the page.

To test that, we’ll create an A/B test with the 2 variations, serve each variation to 50% of users, and then decide which variation to keep.

On the "A/B Tests" tab, we create a new test and call it "Homepage theme".

Since we want to A/B test the homepage, we assign it the slug "/".

In the "Variations" field we set the existing light-themed homepage as the "Control".

We add a "Variant" variation and assign it a page that is identical to the existing light-themed homepage, but we set the theme to "Dark".

On LaunchDarkly, we create a new feature flag with the key "homepage", as indicated on the Sanity A/B test page.

We give it 2 string variations: Control and Variant.

Once the flag is created, we configure it to serve the Control homepage to 50% of users and the Variant homepage to the other half.

Once we publish both the LaunchDarkly and Sanity changes, we can test if the A/B test is working as configured by opening 2 incognito windows.

We'll notice that one of them is served the dark theme while the other is served the light one.

Let's say we ran the A/B test for a while and realized that most users prefer the light theme (Control).

We can now delete the A/B test and the dark-themed version (Variant) from Sanity.

Once this change is published, 100% of users will be served the light-themed variation.