What adds value to your business? That can be a tough question to answer, especially when it comes to website optimization.
A while back, I worked at a company where everyone was convinced that one member of a team was the main value driver for the team’s multimillion dollar project. At the same time, this employee had a coworker that most of the team thought wasn’t bringing much to the table.
However, when the “rock star” employee suddenly quit, the project suddenly went into overdrive. It turned out that—despite all the value that had been attributed to her—she had actually been holding the team back.
A few months later, everything ground to a halt when the “unimportant” employee left. To everyone’s surprise, his efforts had been critical to the project’s success and his knowledge and skills proved irreplaceable.
This sort of story is far from uncommon. As human beings, we tend to assign value to people or things that we like—how else do you explain our obsession with celebrities?
We’ve never met these people, but we believe all sorts of positive or negative things about them based off how they are portrayed in the media.
Everyone liked employee #1. Therefore, everyone believed she was producing value.
No one thought much of employee #2. So, no one attributed much worth to his contributions.
Were either of these assumptions founded in good data? No, but the whole team believed that they knew where all the value was coming from.
Assigning value to the wrong team elements slowed and eventually crippled a multimillion dollar project. Are you doing the same thing to your website?
When optimizing your website, it’s important to identify how specific site elements affect your overall conversion rate.
At its core, website optimization is a relatively straightforward concept. You remove elements of your site and then split test visitors to both the original and the variant.
If the conversion rate on the variant improves, the removed element was hurting your conversion rate. If the conversion rate decreases, the removed element was helping you get conversions.
In essence, this is much like what happened with the company I mentioned previously.
When element #1 (the favorite employee) was gone, things improved. The element was hurting the team. When element #2 (Mr. Nobody) was gone, the project fell apart. The element was helping the team.
What Do I Do With It?
The great thing about website optimization is that it gives you the data you need to make the right decisions.
Inevitably, you’ll discover some of your favorite elements are like employee #1—you might like them, but they’re hurting your business. On the other hand, there will be the elements that are unexpectedly, sometimes inexplicably driving major portions of your conversion rate.
For example, SurvivalLife.com started their website with the kind of clean, minimalistic design most experts recommend. However, after testing their website, they eventually discovered that a messy website sold far better to their target audience.
This revelation ultimately helped them to build their website to $1 million/month in sales.
Conventional wisdom isn’t always data-driven wisdom—existence testing shows you what is working, which is not always what common sense says should work.
As humans, we tend to believe that what works for us works for everyone. However, to run a successful website, you need to figure out what works for your target audience, not for you. That’s the power of existence testing.
What to Test
Okay, if you’ve made it this far, you’re probably sold on the idea of website optimization. The question is, how do you do it?
Guess what? You’re already using it.
If you think about it, website optimization is about more than just determining the value of page components. You can test any aspect of your conversion funnel, but two of the easiest areas to test are pages and page elements.
Do Your Website Pages Matter?
Most companies test the importance of their own homepage without even realizing it.
How? Landing pages.
A landing page is effectively an A/B test against the homepage. Typically, a good landing page that maintains the scent for potential customers has a higher conversion rate than the more generic homepage, so most companies assume that they’re better off with a landing page.
Conventional wisdom, right? Years of experimentation support the idea that landing pages work best, but how many companies actually compare the conversion rates of their home and landing pages?
At Disruptive, we’ve found that the homepages of some companies work better than their landing pages.
In all seriousness, for these companies, their homepage is actually an important contributory part of their conversion funnel.
Without testing landing pages, these companies’ homepages would have been left in the same situation as the ignored employee #2 from our story—undervalued and underutilized.
For example, one of our clients wanted to find out whether or not sending visitors to two separate landing pages would result in more sales.
Both items sold well, but they wanted to split test and see which landing page worked better. We tried testing both pages, but we also threw the homepage into the mix as a comparator.
To our surprise, the homepage won.
It turned out that their customers were interested in both products and what they really wanted was to get to the homepage. Visiting the website was part of their decision-making process, so diverting them into a single channel actually deflected a good portion of the potential conversions.
The homepage mattered to this audience.
So, if you want an easy place to start existence testing, look at your sales process. How are different pages contributing? What happens if you remove a step? What if you add a step?
Page-level existence testing can provide powerful insights into how major changes to your marketing funnel affect your conversion rate.
Which Page Elements Matter?
Once you’ve got a good sense for which pages contribute most to your conversion process, it’s time to get more granular.
By hiding specific elements on a page and comparing conversion rates between the original and the modified page, you can determine how design aspects of your page are affecting your customers.
That Sounds Nice, But Does It Work?
For example, after a series of tests on a client’s homepage, we discovered that eliminating their sliding promotion header increased revenue-per-visitor by 25%. When we hid a navigation bar from one side of their page, revenue went up by 19%.
Eliminating these pet design elements increased profits by $2 million a year!
We’re not the only ones to see impressive results from existence testing. EA removed the promo banner from their SimCity microsite and saw a 43% improvement in purchase rate. Impact eliminated their sidebar and increased their conversion rate by 71%.
In addition to directly improving your conversion rates, page element testing can be a phenomenal way to come up with further testing ideas.
Remember, the only failed test is the one you don’t learn from.
Once you know what works, you can replicate those elements on landing pages or elsewhere on your site. You can also try to milk site components by tweaking things like size, copy, color, imagery or location.
If a particular page element doesn’t affect your conversion rate, you can also try making tweaks to see if you can make those page elements more valuable to your conversion process.
And, of course, if a page element is hurting your site, you should remove it completely or test a replacement.
Overall, testing your page elements is a great way to identify low-hanging fruit on your website. You’d be surprised how big of a difference seemingly small changes to your site can have on your overall profitability.
How to Test
At this point you’re probably thinking, This all sounds awesome, Aden! I just have one question: How do I know if I’m getting meaningful results?
That’s a good question.
Actually, it’s an incredibly important question. To answer it, though, we need to look at 3 aspects of your data—variance, confidence and volume.
The first thing you need to understand about your data is how stable it is. In a nutshell, variance is how much your numbers will change—even when you don’t change anything.
The simple fact of the matter is that your conversion rates will naturally fluctuate over time. These fluctuations are due to factors outside of your controls such as changes in competition, economic factors or seasonal effects.
By running what is sometimes referred to as an A/A test, you can determine what how much natural variability you can expect from your conversion rate.
Understanding your variance allows you to identify which changes fall within the normal variability of your site and which changes represent meaningful differences in your conversion rate.
For example, if your conversion rate varies by ±4%, a 3% improvement in conversions probably doesn’t indicate a successful test. On the other hand, if your variance is ±2%, that same 3% improvement just might be a gold mine.
This becomes particularly important when that 3% improvement could be worth a lot of profit to your company.
In some cases, companies have had results that seemed like multimillion dollar wins, but were still within normal variance. Over time, the data asserted itself and all that extra profit went down the drain.
Now that you’re second-guessing the validity of your results, let’s talk about confidence.
In statistics, confidence is an estimate of the likelihood that your results were due to chance.
Sure, your new conversion rate didn’t fall within normal site variance, but isn’t it possible that you just happened to send a bunch of more-likely-to-convert-than-normal visitors to your page?
Typically, you want to be at least 90% confident that your results were not due to chance. You may also see this reported as ρ-value (a ρ-value < 0.1 is the same thing as > 90% confidence, a ρ-value of < 0.05 is the same as > 95% confidence and so on).
While I’m a firm believer in the value of statistical confidence, I’ve also seen companies spend a lot of time and money trying to reach 90% confidence on a change that was clearly working.
If your data is stable, meaningful and big enough to indicate that you’ve found a winner, it’s okay to move forward even if you have only reached 70% confidence.
In this situation, you typically end up in a 90+ percent confidence bracket anyways, so why wait and waste potential profit?
Fortunately, you don’t have to be a statistician to calculate confidence. Most testing programs will calculate that for you. What’s important is to know what confidence represents and where it comes from. Once you know that, you can confidently use confidence to determine the value of your existence tests.
So, how do you get low variance and high confidence? Lots and lots of traffic!
The bigger your traffic, the more stable your data tends to be.
For existence testing, the most important metric is usually number of conversions. In our experience, it usually works best if you have at least 100 conversions per testing arm.
For sites with a lot of traffic volume, you may be able to run multiple tests simultaneously. For smaller sites, it may take months to get reliable data.
If you do happen to be testing a smaller site, I’d recommend testing dramatic changes such as radically different landing pages or omitting entire steps in your conversion funnel.
The bigger the difference in your tests, the bigger the difference in your test results.
Big differences are much easier to separate from normal variance than small changes, so you can get results without having to wait for a huge number of conversions.
Traffic volume is something of a necessity if you want to optimize your site. Conversion rate optimization can make a huge difference to your site, but you have to have enough traffic to run your tests.
Putting It All Together
When done right, website testing is a phenomenal way to improve your website.
Whether you’re pulling apart your sales funnel or maximizing conversion rates from a page,website optimization shows you which elements of your conversion process are contributing or detracting from your success. Using data to eliminate the bad and maximize the good can do wonders for your company.
By the way, if you’d like me to take a look at your website and give you some testing recommendations, let me know here or in the comments. I’d love to help!
What about you? How do you feel about website optimization? Did this post spark any ideas?
Latest posts by Aden Andrus (see all)
- How Much Money is Your Crappy Website Design Wasting? - October 14, 2019
- Why Everyone Hates Pop-Up Ads (and What You Can Do About It) - October 9, 2019
- 5 Ways to Spend Less Time in Google Ads Manager - October 7, 2019