We use Sanity to build fast websites that rank on the Google search engine and attract users.
As a web design company, we know that A/B testing is a key great way to find out what content fits better with your target users, so integrating it on Sanity is a great way to test your content with real users before going live with them.
This article will be a demo where we show how our clients usually create A/B tests from Sanity and control them from an A/B testing platform called LaunchDarkly.
You can also jump straight to the Youtube video if you prefer:
We have a homepage in a light theme, but we think that users might prefer a dark variation of the page.
To test that, we’ll create an A/B test with the 2 variations, serve each variation to 50% of users, and then decide which variation to keep.
On the "A/B Tests" tab, we create a new test and call it "Homepage theme".
Since we want to A/B test the homepage, we assign it the slug "/".
In the "Variations" field we set the existing light-themed homepage as the "Control".
We add a "Variant" variation and assign it a page that is identical to the existing light-themed homepage, but we set the theme to "Dark".
On LaunchDarkly, we create a new feature flag with the key "homepage", as indicated on the Sanity A/B test page.
We give it 2 string variations: Control and Variant.
Once the flag is created, we configure it to serve the Control homepage to 50% of users and the Variant homepage to the other half.
Once we publish both the LaunchDarkly and Sanity changes, we can test if the A/B test is working as configured by opening 2 incognito windows.
We'll notice that one of them is served the dark theme while the other is served the light one.
Let's say we ran the A/B test for a while and realized that most users prefer the light theme (Control).
We can now delete the A/B test and the dark-themed version (Variant) from Sanity.
Once this change is published, 100% of users will be served the light-themed variation.
When you run an A/B test, you want to do so against unique users. If you reload the page during the same session, you're identified as the same user and thus served the same variation during the A/B test.
If the website has authentication in place, we use users' ids to identify them. For anonymous users or for websites that don't have authentication, we generate a unique UUID and store it in the browser's cookies.
You first have to ask yourself: what user action makes my page successful? For example, it could be pressing a Call-To-Action button or subscribing to your service.
Once you know what action makes a page variation successful, you need to have proper analytics in place to track that action.