Hi friends,
We have been asked many times about the easiest way to perform multivariate testing of landing pages. Something that will work when testing small page variations just to get information about how each small change performs.
You can create separate landing pages with slight variations, then split-test traffic between them in order to see which ones perform better, but there is an easier way to do multivariate testing with a single physical landing page.
Let’s say that you want to test 3 headlines, 2 images and 3 buttons (CTA). This means that you will have to generate 3x2x3 = 18 different pages for such variations.
But what if there was a way to test all these 18 page variations within a single landing page? You would probably think that it must be complicated or that it must require coding skills… Well, this is super-easy and it can be done with no coding at all in MV Lab.
We recently did a small case study on how to use MV Lab and see how each small variation of a page improves the CTR.
I will explain all the steps in this thread with a real campaign in order to show how MV Lab can be used and what the multivariate testing results were after 3 days tests.
What I was looking for specifically was how to track multivariate results and the CTR of different page versions.
I was absolutely sure that the campaign won’t be profitable and that the offer chosen is not one of the best, but this campaign was not about making $$$, it was just for learning multivariate testing in an easy way.
So my focus was on the landing page and how to adjust it based on the metrics tracked.
Traffic Source: PropellerAds
Tracking Tool: CPV Lab Pro + MV Lab
Affiliate Network: Traffic Company
Type of Offer: not great, not terrible
My initial landing page was pretty simple and it looked like this:
I wanted to see how small changes in this page will improve CTR and I planned on performing multivariate tests for the following items:
So let’s see how the MV Lab approach is and how much time this saved us.
There are a few requirements for using MV Lab for multivariate testing:
There was one extra thing to do in my campaign: I have setup one extra token for each of my snippets in the multivariate test. These extra tokens will receive information about the variations displayed for each visitor.
Now using the visual editor for MV Lab included in CPV Lab Pro, I have defined each snippet and the variations that I want to be rotated for it.
I can also set a Share (number of visitors) for each variation, but for now I left the Shares blank in order to have traffic distributed evenly.
I’ve copied the code and pasted it at the beginning of the HTML code in my landing page, before the <html> tag and any other tags. This code will quickly execute before the page is loaded and set the variations for each visitor.
While editing the landing page, I have also defined the places where I want the variations to be rotated.
I just replaced the actual fields with the snippets.
These will be the 3 snippets and I need to use the wildcard ##snippet_X## (where X will be the snippet ID) in the exact place where I want it to be replaced by the actual variation.
Now the configuration of MV Lab in my landing page is complete. All I have to do is to upload the landing page on the server and it’s ready for traffic.
You can see our page at the URL below. Refresh the page several times to see how different variations will appear in the 3 snippets with each page refresh… this is how the page variations was rotated between my visitors (and yes, the url has a token to help me count how many times you clicked it
)
Lucky Tiger Promo
After running a few hours of traffic in this campaign, I had the first results.
As mentioned before, I was focusing on the Clicks from the landing page and the CTR of the page for different variations of the 3 snippets that were defined. I just wanted to increase the page CTR.
The initial overall results of the page from CPV Lab Pro are as follows:
So that’s a 8.33% CTR for my page across all variations.
But let’s dig deeper and see how my multivariate test went. Let’s look first at the first snippet which rotated 3 texts for the subtitle: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “YOUR LOCATION”.
Well, this brings some information into what happened in my landing page. I can see that 2 of my Subtitle variations were getting higher than average CTR, while the third variation wasn’t performing at all.
Subtitle 3 is a certain candidate for removal from my landing page.
Let’s look now at the second snippet, the background of the page, which I’ve tested in 2 variations: with and without a background image.
Just like with snippet 1, we have a clear underperforming variation here, the one without a background image (var2-black). Not as drastically, but still pretty big difference between the 2 variations from here.
Let’s check now the third snippet from my test which is the button background color. This was tested in 3 variations: “LawnGreen”, “Yellow, “Red”.
The CTR for each of the 3 variations for the button background is pretty similar, there is a very small variation here. This means that the button background color didn’t really have an influence of the clicks received in my page, so I focused on the other 2 variables (Subtitle and Image).
If I run a combined report for the 3 variables (Subtitle, Image and Button) sorted by CTR descending, then things look like this:
First Conclusions:
Based on this initial traffic, I’ve identified that the third variation for the Subtitle snippet (“YOUR LOCATION”) isn’t working at all, no matter what other variations it was paired with. This is a good moment to remove this variation from the landing page as it isn’t bringing results.
Also there is a clear difference between the background image variations. The variation containing a background image performs better and traffic should be increased to it. Or maybe the variation with no background image should be removed.
The 3 background colors for buttons didn’t make a difference for Clicks and CTR, so they can be kept as they are.
Next Actions:
1. Normally I would have removed the third variation for the Subtitle snippet, but instead I changed it to something else in order to test it further.
I changed this variation from “YOUR LOCATION” to “<visitor’s city>” where this value will be dynamic based on the actual visitor location.
Let’s see if the CTR will improved when I showed the exact visitors’ location in the landing page instead of the generic “YOUR LOCATION” text.
It’s pretty easy to write the actual location for visitors in my landing page with MV Lab.
First I configured CPV Lab Pro to pass visitors’ city in the landing page URL in the “city” parameter by ending my Landing Page URL with “?city={!mm_city!}”.
The {!mm_city!} placeholder will get replaced by the actual city of each visitor and now MV Lab will write the actual city for each visitor when this Variation 3 will be displayed.
2. Another change I made was to send more visitors to the variation with background image. I’ve set 75% of traffic to go to variation 1 (with background image) and 25% traffic to go to variation 2 (no background):
3. Snippet 3 (background color for button) remain unchanged.
I will stop this post here as it got very long.
I will post again later to show the next results so you can see what changed.
Cheers,
Julia
We have been asked many times about the easiest way to perform multivariate testing of landing pages. Something that will work when testing small page variations just to get information about how each small change performs.
You can create separate landing pages with slight variations, then split-test traffic between them in order to see which ones perform better, but there is an easier way to do multivariate testing with a single physical landing page.
Let’s say that you want to test 3 headlines, 2 images and 3 buttons (CTA). This means that you will have to generate 3x2x3 = 18 different pages for such variations.
But what if there was a way to test all these 18 page variations within a single landing page? You would probably think that it must be complicated or that it must require coding skills… Well, this is super-easy and it can be done with no coding at all in MV Lab.
We recently did a small case study on how to use MV Lab and see how each small variation of a page improves the CTR.
I will explain all the steps in this thread with a real campaign in order to show how MV Lab can be used and what the multivariate testing results were after 3 days tests.
What I was looking for specifically was how to track multivariate results and the CTR of different page versions.
I was absolutely sure that the campaign won’t be profitable and that the offer chosen is not one of the best, but this campaign was not about making $$$, it was just for learning multivariate testing in an easy way.
So my focus was on the landing page and how to adjust it based on the metrics tracked.
Traffic Source: PropellerAds
Tracking Tool: CPV Lab Pro + MV Lab
Affiliate Network: Traffic Company
Type of Offer: not great, not terrible
My initial landing page was pretty simple and it looked like this:
I wanted to see how small changes in this page will improve CTR and I planned on performing multivariate tests for the following items:
- Subtitle text – test 3 variations: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “YOUR LOCATION”
- Background image – test 2 variations: with background image and without background image
- Button color – test 3 colors: “LawnGreen”, “Yellow, “Red”. These 3 will be the snippets from my page that I will want to rotate multiple variations for.
So let’s see how the MV Lab approach is and how much time this saved us.
There are a few requirements for using MV Lab for multivariate testing:
- You need to have access to the page HTML code, so you need a page that is hosted on your server and not a page from a landing page builder tool (but a little birdie told me that a version compatible with Wordpress is being prepared)
- You need a PHP page – if you have a page with *.htm or *html extension, then you can simply rename it to have a *.php extension
- The mvlab.php file generated in your MV Lab account should be uploaded on the server, in the same location as your landing page
- CPV Lab Pro as the tracker in order to see the results of your tests directly in the tracker reports
There was one extra thing to do in my campaign: I have setup one extra token for each of my snippets in the multivariate test. These extra tokens will receive information about the variations displayed for each visitor.
Now using the visual editor for MV Lab included in CPV Lab Pro, I have defined each snippet and the variations that I want to be rotated for it.
I can also set a Share (number of visitors) for each variation, but for now I left the Shares blank in order to have traffic distributed evenly.
I’ve copied the code and pasted it at the beginning of the HTML code in my landing page, before the <html> tag and any other tags. This code will quickly execute before the page is loaded and set the variations for each visitor.
While editing the landing page, I have also defined the places where I want the variations to be rotated.
I just replaced the actual fields with the snippets.
These will be the 3 snippets and I need to use the wildcard ##snippet_X## (where X will be the snippet ID) in the exact place where I want it to be replaced by the actual variation.
Now the configuration of MV Lab in my landing page is complete. All I have to do is to upload the landing page on the server and it’s ready for traffic.
You can see our page at the URL below. Refresh the page several times to see how different variations will appear in the 3 snippets with each page refresh… this is how the page variations was rotated between my visitors (and yes, the url has a token to help me count how many times you clicked it
)Lucky Tiger Promo
After running a few hours of traffic in this campaign, I had the first results.
As mentioned before, I was focusing on the Clicks from the landing page and the CTR of the page for different variations of the 3 snippets that were defined. I just wanted to increase the page CTR.
The initial overall results of the page from CPV Lab Pro are as follows:
So that’s a 8.33% CTR for my page across all variations.
But let’s dig deeper and see how my multivariate test went. Let’s look first at the first snippet which rotated 3 texts for the subtitle: “BLACK FRIDAY”, “NEXT 10 MINUTES”, “YOUR LOCATION”.
Well, this brings some information into what happened in my landing page. I can see that 2 of my Subtitle variations were getting higher than average CTR, while the third variation wasn’t performing at all.
Subtitle 3 is a certain candidate for removal from my landing page.
Let’s look now at the second snippet, the background of the page, which I’ve tested in 2 variations: with and without a background image.
Just like with snippet 1, we have a clear underperforming variation here, the one without a background image (var2-black). Not as drastically, but still pretty big difference between the 2 variations from here.
Let’s check now the third snippet from my test which is the button background color. This was tested in 3 variations: “LawnGreen”, “Yellow, “Red”.
The CTR for each of the 3 variations for the button background is pretty similar, there is a very small variation here. This means that the button background color didn’t really have an influence of the clicks received in my page, so I focused on the other 2 variables (Subtitle and Image).
If I run a combined report for the 3 variables (Subtitle, Image and Button) sorted by CTR descending, then things look like this:
First Conclusions:
Based on this initial traffic, I’ve identified that the third variation for the Subtitle snippet (“YOUR LOCATION”) isn’t working at all, no matter what other variations it was paired with. This is a good moment to remove this variation from the landing page as it isn’t bringing results.
Also there is a clear difference between the background image variations. The variation containing a background image performs better and traffic should be increased to it. Or maybe the variation with no background image should be removed.
The 3 background colors for buttons didn’t make a difference for Clicks and CTR, so they can be kept as they are.
Next Actions:
1. Normally I would have removed the third variation for the Subtitle snippet, but instead I changed it to something else in order to test it further.
I changed this variation from “YOUR LOCATION” to “<visitor’s city>” where this value will be dynamic based on the actual visitor location.
Let’s see if the CTR will improved when I showed the exact visitors’ location in the landing page instead of the generic “YOUR LOCATION” text.
It’s pretty easy to write the actual location for visitors in my landing page with MV Lab.
First I configured CPV Lab Pro to pass visitors’ city in the landing page URL in the “city” parameter by ending my Landing Page URL with “?city={!mm_city!}”.
The {!mm_city!} placeholder will get replaced by the actual city of each visitor and now MV Lab will write the actual city for each visitor when this Variation 3 will be displayed.
2. Another change I made was to send more visitors to the variation with background image. I’ve set 75% of traffic to go to variation 1 (with background image) and 25% traffic to go to variation 2 (no background):
3. Snippet 3 (background color for button) remain unchanged.
I will stop this post here as it got very long.
I will post again later to show the next results so you can see what changed.
Cheers,
Julia
