For Website or in-product surveys, you can conduct a split test by running different variants of one survey which will be randomly shown to visitors of your website or web app. We achieve this by assigning website visitors different cookie values.
With A/B/X testing, you can:
see which version of a survey gets higher response rates;
experiment with offering various incentives to your visitors;
test out different Call-to-Actions after answering the survey.
In this article, you'll learn how to set up A/B/X testing and run various variants of your website or in-product survey to avoid researcher and response bias.
Step 1. Enable the A/B Testing integration
1. Find the A/B Testing integration. You can open it in two ways:
From Integrations: in the left-side menu, go to Integrations and search for A/B Testing:
from your survey: open your Website survey, go to the Connect step, and search for A/B Testing:
2. Click Connect to activate the A/B Testing feature.
3. Once connected via Integration, go to the Settings tab. Name your cookie and choose how many groups you want to split your visitors into:
We will generate a cookie with numerous values depending on how many variants you’d like to test. This creates randomly picked segments of website visitors for your A/B/X test.
For example: Test A: 1-2 is an A/B test. We'll give every visitor the Test B: 1-4 is an A/B/C/D test. Your visitors will be split into 4 groups of 25% each. |
💡 To run several A/B/X tests at once, click + Add cookie. In the screenshot above, two tests are run at the same time: Test A and Test B.
4. Don't forget to click Save. The cookie generation and assignment will be enabled and we’ll start splitting your website visitors by assigning everyone one of the cookie's values.
Step 2. Create and target surveys
The created cookies should now be used to target your surveys.
1. Create a new Website or in-product survey.
You can either set up a survey of your own (start from scratch), select a survey from our templates library (start from template), create a survey with AI, or import questions you prepared before:
2. Go to the Audience tab, click +Add audience and choose the audience type:
3. Give your audience a name and click Create Audience.
💡 To easily tell your A/B surveys apart, name each survey and its Audience after the cookie and its value, for example, Test A-1 and Test A-2.
4. Click Add filter and choose Cookies under User properties.
5. Pick the cookie you created in the A/B Testing integration, choose equals, and enter its value. In this example, the first survey includes visitors whose cookie Test 1 equals 1.
❗️The cookie name must be exactly the same as in the Integrations settings, where you decided on the cookie name and the number of values.
6. When you are ready, click Create audience.
Remember to enable the audience in order to display the survey properly.
❗️Create as many surveys as cookie values you've picked. For example, the Test B cookie has values 1-4, so you need 4 surveys.
Now, open the next survey and repeat steps 1 to 6. This time, create the Audience for the same cookie with the value 2, then 3, and so on. You can name the Audiences Test B-2, Test B-3, or anything else you like.
Survey throttling in A/B/X testing
If you don't want one visitor to see several surveys from your test when they come back to your website, open each survey and go to Configure ➡️ Frequency. Turn on Override global throttling settings and set Time since the last survey seen, for example, to 30 days:
In this example, the survey ignores the global throttling and waits at least 30 days after the visitor has seen another survey before it shows up.
💡 You can also set up web survey throttling for the whole workspace in Settings ➡️ Throttling. Web surveys throttling sets the minimum time before the same respondent can see the next survey from your workspace:
This applies to all Website surveys in your workspace. To use different rules for a single survey, turn on Override global throttling settings in the survey's Configure ➡️ Frequency step.
❗️The Global throttling applies to each domain separately - meaning that, if you show website surveys on various domains, and a visitor enters them both, they may still see another survey in the second website. That's because domains don't share Local Storage.
Step 3. Analyze responses
When your surveys have collected some responses, you can see them in the Analyze tab in your Survicate Panel. By comparing the results from all your surveys, you can determine how survey questions, survey design, or even the answer choices influence the results.
Take a look at our survey results guide to make the most of your visitors' answers 🚀
Troubleshooting
I don't see the survey even though I belong to the enabled Audience
If you don't see the survey, please check if you have the Survicate tracking code installed on your website.
It's also possible you have already answered it. Please try changing the survey's Frequency settings.
Best practices
Keep everything except the tested element the same
For reliable results, show all surveys from one test on the same pages and with the same triggers. Change only what you want to test, for example, the question wording or the incentive. Learn more in Set up pages where your survey is triggered and Adjust when you'd like to show your Website survey.
Combine the cookie with other filters
Add more filters to the same audience to narrow down your test. For example, combine Cookies with Device to run the test only for desktop visitors. See all available filters in Define your Website survey's audience by targeting based on visitor filters.
📞 If you have any questions about setting up the Survey A/B/X Testing - feel free to reach out to our team via chat or email: support@survicate.com.