Entrepreneurs List

Google Web Analytics

Google Web Analytics and A/B Tests

In this section, we’ll be covering basic Google Web Analytics metrics, more advanced analytics for Adsense and Adwords like conversion tracking, and finally, A/B tests.

As you can imagine, it takes quite a bit of skill across each domain to put together an effective website. A/B tests are a huge part of any conversion based site and success or failure in A/B tests can affect the viability of Adwords and SEO campaigns.

So, enough chatter, let’s get started!

What are Google Web Analytics?

On nearly every page you visit online you not only download text and images but also download a little chunk of javascript that phones home about your visit.

On this phone call home it reports where you came from, the page you are looking at, your general geographical location, your operating system and your browser of choice.

It also reports any other pages you visit on that site, if you buy something from them and how long you’ve been there. In short, it sends a lot of information!

It’s this information that we’re most interested in because it can tell us a lot about our visitors. For example, it can compare how many people found your site from another site that linked to you and, more importantly, it can tell you how often that person buys something from you (which is called a conversion).

This is what analytics is. It’s the automatic gathering of information from your visitors that can help you understand their actions and will help you make better decisions.

Using numbers and statistics to make sound decisions

Before we get into the details about Google Web Analytics and A/B testing, just realize we’ll be dealing with a lot of numbers. And we’ll be dealing with some statistics. However, rest assured that you won’t need to know anything about regression (though, if you do, you will have a distinct upper hand) or confidence intervals (though, you’ll also have an upper hand if you know that kind of stuff too!).

That being said, you will need to know a little bit about sample size. Allow me to explain.

We all know quite a bit about percentages from our everyday lives. For example, 1 out of 2 is 50%. 2 out of 3 is 66.67%. And so on.

However, what is the difference between 1 out of 2 and 60 out of 120? They are both 50%, right? The most important difference here is sample size. If you just want to get to the nugget of the concept, read this line a few times: the larger a sample size, the more reliable its conclusions.

Read it a couple times for good measure.
The idea is this, if you only have a handful of samples to measure, the chances of those samples being a fluke are very high. For example, if 5 out of 6 people say they like your product, you might be happy to hear that 83% like your product.

But who knows, maybe you just got luck and picked your 5 biggest fans. What if you expanded your sample size to include 1,200 people and found out that 636 liked your product?

That’s only 53%. It’s the same product and same question, but which experiment is more reliable?

The same goes for any number based statistics in Google Web Analytics and A/B testing, don’t draw any conclusions until you have a nice and big sample size. This may be hard to do because if you are just getting started, you might only size a dozen visitors a day.

It will take you a month or two to gather the sample size you need in order to make wise decisions. So be patient. And never, ever underestimate the importance of sample size.

Now, we can move onto the good stuff.

Where can I get Google Web Analytics and how do I install it?

Nowadays, there are many dozens of analytic software providers, but the most common and powerful by far is Google’s analytics package called, big surprise, Google Analytics.

However, there are several other options like:
Google Analytics (http://www.google.com/analytics/) – free (hosted for you)
Clicky (http://getclicky.com/) – $30-$120/yr (hosted for you)
Mint (http://haveamint.com/) – $30 per site
Piwik (http://piwik.org/) – free

Unless you have very specific needs like IP address tracking or complete control over your data, you are probably going to use Google’s because, frankly, they are among the best out there and it ties in directly to Adsense and Adwords (if you use either product).

Installation is usually very simple. It involves copying and pasting the javascript into your page’s HTML source. If you use a CMS like WordPress, Joomla! or Drupal, this is often very simple to add a plugin to do this for you.

Otherwise, a quick edit of the correct template file will suffice.
If you decide to go for either Mint or Piwik, you will need to install a copy on your own server and host it yourself. Both Clicky and Google Web Analytics are hosted for you and do not require this extra step.

How to interpret common metrics

Once you get your Google Web Analytics installed and have gathered some data, it’s time to start digging around in the interface and discover what your users are up to.

The first and most obvious metric is visitor count. Visitor count is exactly what it sounds like; it is a measurement of how many visits you’ve had over the course of a time period (usually a day or week).

There is a distinction between visits, unique visitors and pageviews though. Let’s imagine Jerry from Maine visits your site in the morning and looks at 6 pages. Later on that afternoon, he comes back and looks at another 2 pages.

Number of visits from Jerry would be two (one in the morning, one in the afternoon, unique visitors would be one (Jerry is only counted once here) and pageviews would be 8 (6 in the morning and 2 in the afternoon).

Another common metric is bounce rate. This tells you what percent of visitors visited but left after only one measly pageview. This can means that the visitor wasn’t very interested in reading anymore and we might consider finding ways to capture his/her interest and look at another page.

Related posts plugins are a common method of decreasing bounce rate and increasing pageviews.

Other metrics to look at are things like sources of traffic, which can be search engine, referring sites or direct traffic. Search engine traffic can be further segmented down into keywords. Referring sites can be segmented into exact site URLs.

Direct traffic is simply type-into-the-address-bar traffic. These guys know they want to go to your site so they just go.

Using metrics to track trends

One of the most effective ways to use Google Web Analytics packages is to track trends. For example, you may be doing a lot of work to rank for a particular keyword.

A quick check on the traffic sources, drilling down to search engines and then even further on your particular keyword can show you if you’ve begun to earn more traffic from that keyword or not.

This gives you direct feedback and lets you make a decision: are my efforts to rank for this keyword paying off?

Additionally, consistent work can give a very obvious trend increase in traffic. Just for fun, try estimating your traffic into the future. Often times this is a straight line.

Just see how much you increase every month and forecast forward. For example, let’s say you had these raw traffic numbers:

So that means in February, we saw an increase of about 7.9% (2,433-2,254/2254), March at 4.4%, April 2.6%, May 4.1%, and June 7.8%. All we have to do is average these to get ((7.9%) + (4%) + (2.6%) + (4.1%) + (7.8%)) / 5 = 5.28% a month. Let’s forecast forward 12 months assuming we continue to grow at that rate: (1.0528^12) * 2,930 = 5,432.

But watch out for unnatural spikes in traffic. Just because you got picked up by a popular blog one month doesn’t mean that that increase is lasting. This is what we call an “outlier”. If you get an insanely high or low number for a month, just throw it out. Your results will be much more realistic.

Advanced Metric Tracking

Adsense

Once you have Adsense set up along with Analytics, you’ll need to tie the two services together by allowing Analytics to pull Adsense data from your account.

This is a simple process that requires you to edit each of your sites’ profile settings. Once you do this and data has been collected, get ready to do some digging to find out which keywords, traffic sources and pages are earning you the most money.

If your cup of tea for monetization is Adsense or you use Adwords for driving traffic to your website, rest assured that you should be using Google’s Analytics package because they integrate so closely into your Adwords campaigns and Adsense earnings.
Before we go any further, make sure you select a decent amount of time to analyze.

There is no use in analyzing only a day or two at a time. Try for the a month or at least a couple weeks.

eCPM

This measures your effective cost per thousand impressions. In short, this is how much you earn for every thousand ad impression (though this can get complicated if you have more than one ad on a page as one pageview will show up as multiple impressions). The higher you can get this, the better.

Tips to increase eCPM: try larger ads with better, more obvious placement. Other color schemes might be advised as well.

Revenue per 1000 visits

This is an extremely valuable metric. Consider this the #1 metric to increase. This tells you how much money you make per 1000 visits. So think of it this way, if you can get an extra 1000 visits today, how much money would you make? This is also a great way to do “what if” scenarios. So, let’s say I have an average of $6.92 revenue per 1000 visits.

What if I want to make $100 a day? I would need:
( $100 / $6.92 ) * 1000 = 14,451 visits a day

Tips to increase revenue per 1000 visits: same tips as increasing eCPM but try placing more ads on a single page.

CTR

This is a very simple metric that is correlated with eCPM and revenue per 1000 visits. It just tells you how often a user clicks on an ad. This click may or may not earn you a lot of money, but the click is the first step.
Tips to increase CTR: place ads in better positions with more obvious color schemes. Also, try larger ad sizes or alternate ad shapes.

Conversion Testing

If you don’t use Adsense or other ad networks to generate revenue, you are likely trying to sell a specific product, gather a lead, or collect an email among other things. It doesn’t matter what you want your visitor to do, when a visitor does it, we call that a “conversion”. If you have a solid understanding of conversions, skip ahead as we’ll just be talking about the concept as a whole.

The first step of setting up conversion tracking is figuring out exactly what you want your visitors to do once they visit. Like we said before, that could just about anything. Within most Google Web Analytics packages they are referred to as “goals”. Most of the times, the actual goal can be as simple as visiting a thank you page. The only way to get to that page is by doing whatever you want them to do (filling out a form, ordering a product, etc.). Additionally, enter a dollar value for the goal. This will help you figure out important break even points later on (especially if you use Adwords).
The second step is setting up the funnel. The funnel is actually quite simple, it is just the order of pages a visitor must go through to complete a goal. If your conversion is to collect an email address from which they will be redirected to the goal page which doubles as a thank you page, then your funnel will only be a single page deep. However, if you need to collect all kinds of information across several pages to complete a sale, then your funnel will be much deeper.
The third and final step is to wait and analyze the results. If done correctly, you will have a lot of data to look at. But we’ll cover the most important concept right now.

Conversion Rate

This metric is an all-important metric. This is the king of metrics. The metric you should pay the most attention to. Why? Because this tells you what percentage of your visitors are converting (and earning you money). And the best thing is you can view your conversion rate from pretty much any angle. For example, look at entrance pages and see which ones have the highest conversion rate. Or, try looking at referrers and focus on which websites send traffic that convert the best. And, even better, look at which search terms and keywords are converting best and only pay for those keywords or work on ranking for those terms.

Additionally, you want to do your best at increasing your existing conversion rate. The most popular and most used method is A/B testing. A/B testing is an extremely powerful tool that allows you to pit two different pages against each other and see which one has the best conversion rate. After your figure out the victor, you make that the base design and use it to make a slightly differing design. Rinse and repeat.

Tip: Use A/B testing! It can increase your conversion rate by leaps and bounds. We’ve seen increases from about 2.5% to over 15% and 20% over time. We’ll talk more about A/B testing later on!

Adwords

Most of the time, you aren’t relying on only SEO rankings or referring sites to drive you traffic and you’ll want to integrate your Adwords campaigns with Google Web Analytics. Luckily, Google makes this extremely simple to do. If you decide to use a different analytics package or different advertising mediums or providers, this may be slightly more complicated to configure, but the basic concepts covered here are still applicable.

In this section, we won’t be talking so much about the operation of Adwords (we have a whole chapter dedicated to PPC) but we will be talking about using your Analytics package to dig deep into your data and pull out some useful information. Adwords itself has basic metrics built in (like conversion rate) but lacks some of the full features and fine grain detail that Analytics can provide. So use both TOGETHER!

In order to fully utilize Adwords, you’ll need to make sure you have your goal conversions set up properly. Usually, Adwords requires you to place a little piece of code on your thank you page (or goal page) while Google Web Analytics will have you enter the URL. Make sure to get both of these methods set up the same.

A/B Testing

A/B testing is a long and labor intensive process. While the majority of your time will be spend waiting for results (more on this later), the process is never-ending. You can always improve! So, my advice to you is to learn a lot about A/B and conversion testing so you can reap the long term benefits. Just think about it this way: once you have a good product, there may be no need to continuously fix what isn’t broken. But with A/B testing, you aren’t fixing something but improving it in increments.

So, what is A/B testing? A/B testing can come in many different flavors and be used on all kinds of different mediums. At its most basic level, A/B testing is presenting two or more competing versions of a medium (maybe two different web page designs or layouts, or even two different ad designs or text) and comparing their performance. After some time has passed and you have some statistics about each version, you can choose the winning version to be the default.

Let’s have a quick example: let’s say you’ve updated the layout on your landing page. This is the page that the user first lands on and introduces your product. You have two versions, A and B, and both are displayed below. Let’s assume the content is about the same, but only the layout and copy changed.

Now, after a couple weeks, you have had many hundreds of visitors and over two dozen sales. When you dig into the stats, you notice that your overall conversion rate was 1.79%. That is, you had 2,175 visits and 39 sales.

However, you notice that version B had a conversion rate of 2.38% and version A only had a conversion rate of 1.19%. That is, version B had 26 sales and 1090 visits and version A had 13 sales and 1085 visits. So, we can tell that something in version B works better. Maybe it’s the green button, or the different header.

Or it could even be the 33% off tag that lets them know about the free shipping. Regardless, what you do know is that version B is better. So, you can end this test, make version B the default and design a new design that you think will be even better.
However, let’s say the next design you make loses to old design.

No problem, just eliminate the new design and start over again.
As you can see, you can A/B or split test almost endlessly. There are always improvements to make. Let’s talk about some of the most common things to split test.

Layout

Layout is rather intensive prospect as it requires some changes that are more difficult that simply changing an image or something text. However, it can provide some of the biggest boosts. Just imagine: a layout that looks obvious and simple to you may in fact confuse your target users. If the call to action isn’t above the fold or isn’t obvious, you can waste a lot of precious clicks.

This is why we tend to start with a couple basic, simple layouts. This gets us in the ballpark as far as which layout will be the most effective. We use the winner to be the base of our entire project and move forward from there. We almost never revisit major layout A/B testing during a campaign unless everything else has been completely exhausted.
Tip: for a quick and easy layout test, just try flipping the entire page. Left sidebars go right. Right aligned images for left. Etc…

Copy

This is by far the most common element to split test. I tend to try several different version of a key phrase or blurb. A great area to start is by telling them why they should purchase from you. 98% satisfaction. 33% off. Instant delivery. Money back guarantee.

Each of those stinger lines has an effect. Try mixing and matching these stinger lines across blurbs and headers. Eventually you’ll find a perfect match that increases your conversion rate significantly.

Tip: At a loss for copy to test? Try bolding and italicizing keywords and headings.

Images

Stock images are great for increasing conversions as well. Try a service like iStockPhoto.com and purchase several similar, generic images and split test each (as well as a version with no image at all).

You might be surprised.

Tip: Be sure to use test high quality imagery. Pixelated images usually have worse stats (usually…).

Colors

Another great idea is switching out colors and testing that effect on purchases. You’d be surprised how much color affects the mood of your visitors. An especially common item to split test colors on is the buy now button. A common choice is red or green.

Additionally, try to switch out font colors, especially in banners and headers, or even background colors for the same. Try complementary colors and clashing colors. You never know what will work, and most the times, it defies logic.

Tip: I suggest trying to switch out fonts as well, you’d be surprised what a font can do to incite mood. Something like http://code.google.com/webfonts/preview or http://www.fontsquirrel.com/fontface has embeddable HTML fonts so you don’t have to muck about with images.

Design (or the lack of design)

This goes back to the layout suggestion in the beginning. Sometimes you’d be surprised what changing the design will do for you. And this means general site design, usually that encompasses things like font choice, navigation design and background design, as well as overall color scheme.

Try some fancy designs and try some really basic and ugly designs. Again, you might be surprised.
The actually switching of designs should be done primarily with CSS, not HTML. Do a little research on CSS and HTML to learn the difference, or find someone who knows and ask them to help.

Tip: If you are testing the design of a page with forms, try removing or modifying the fields in the form or even try field validation with a green checkmark for each completed and valid field.

Price

This one is often overlooked, you have a product and you guess your price. That’s how it works right? Wrong! Create different price points and test which leads to pretty conversions and more importantly, greater profit margins. This can be a tricky one to implement because once you set a price, you need to follow through with it!

If you use a digital service to sell a digital product (like clickbank.com and PayPal) you might simply create numerous products with the same download but with different prices. On each page, you include the appropriate button for that price point.
After it’s all said and done, not only should you compare conversion rates, but you should also compare your revenue streams and see which variation maximizes profits and revenue.

For example, you might sell 102 copies at $10 a pop ($1,020 total revenue), but you might sell 72 copies for $18 ($1,296). Which is better for you? Unless you have an opportunity to upsell or otherwise monetize existing customers, I’d go with the $18 price point.

Incentives

In the same way of testing price, you can test incentives like offering an extra product for free, extended terms for free, or advertise a percentage off from “regular” prices. Remember, anything you can imagine to might increase sales or revenue, this is a perfect opportunity to test it, even if it isn’t commonly done.

Tip: Try offering varying coupon codes (both raw codes and offerings) if your product can support it!

Implementing Split Testing

There are several providers of split testing services; chief among them is Google’s Website Optimizer. In fact, this is the one I most recommend because of the ease of use and acceptance in the community.

Like we’ve mentioned before, they offer two types of conversion testing: A/B experiments and multivariate experiments. We’ll stick to discussing A/B experiments initially.

Setting up a test

Generally, the first thing you should do is prepare several different copies of your page to be tested with the changes you’d like to test. You should then figure out which page is considered the goal page. The goal, again, can be as simple as a thank you page, or even the first page of the order screen.

It depends on what you want to track.

Simply create a new A/B experiment, luckily, Google makes this rather simple, but we’ll talk about some of the finer points of setting up the experiment.

You’ll need to place javascript on each page that you are interested in, for example, on each different test page and the goal page. Google will double check to make sure the code exists and is working before you can finish setting up the test. If for some reason Google can’t reach the code (due to the way your website operates), then you might have some difficulties getting the test set up.

You might need to look at the way your website works and fix it temporarily.

Interpreting Results

While most A/B suites will interpret the results for you, it is still important to have an understanding of the numbers they will give you. A lot of the math and statistical models that dictate how A/B results are interpreted can be very tricky and confusing. However, we’re going to break it down into easier to digest pieces and understand each of the numbers we see.

Remember to review the short talk about sample size earlier in this section, as we’ll be referring to sample size again shortly.
Here’s a handy, real life screenshot of Google’s tool to give you an idea of what the interface looks like. Just review it quickly and pay attention to the numbers and chart and then read on to learn more.

As you’ll notice, we have two competing designs: Original and List View. As you can imagine, the second version implemented a type of list for showcasing features, while the original had a more traditional paragraph form of feature discussion. After running the test for about a month, we had 436 visitors and 26 conversions, giving us a conversion rate of about 5.96%. However, when you compare the two versions, you notice the original was doing markedly better. The original had a conversion rate of about 8.18% while the list view had a conversion rate of about 3.7%.

While we’re at it, let’s discuss what we know about the estimated conversion rate. After the initial number, you notice the +/- directly after that. The +/- gives us a range in which we estimate that the true conversion rate is within. For example, in the original, our conversion rate is 8.18% +/- 2.6%. That gives us a range of 5.58%-10.78%. This means, we can expect the true conversion rate to be within that range. Now, as sample size increases, our range will decrease because a higher sample size makes us more confident in our estimations. Looking at the list view we have a range of 1.8%-5.6%. I want you to pay close attention to how close the list view’s upper range is to the original’s lower range. In fact, the overlap is only 0.02%. As that overlap shrinks or even disappears, you’ll notice the CBO getting very large or very small. CBO? Allow me to explain…

Now, what I want you to pay very close attention to is the chance to beat original (CBO) column. This is where the magic happens. At the beginning of the test, the CBO will be at 50%. As the test proceeds and the competing designs do better or worse than the original, the CBO will either increase or decrease. For example, in this screenshot of an experiment, the list view is doing much worse than the original; therefore it has a much worse CBO of 2.67%. That means, if you ran this exact experiment 100 times, on average, the original have a better conversion rate about 97 times and the list view would have a better conversion rate the other 3 times.

Basically, if the test is close race wherein all variations have similar conversion rates, expect to wait for a very high sample size before you will have a CBO that is conclusive. I usually require a CBO of about 95% or better before I will end an experiment and make the winner the new default. On the other hand, I usually end an experiment if the CBO is less than 5% and keep the original as the default.

Keep in mind, if you have more than two versions competing, make sure to wait until all CBOs are within a safe threshold to end the test, and pick the variation with the highest conversion rate to be the new default (as long as it has a CBO of above 95%, of course!).

If you want to be even surer of your decisions, you can always have stricter requirements. For example, some people like to require a confirmation or rejection level of 99% and 1% for CBO. This is much more stringent and usually requires a much larger sample size (and more time). I find 95% and 5% to be satisfactory, but some people only go to 90% and 10%. There is a tradeoff between accuracy and time savings.

Additionally, if you notice the conversion rates are identical and the CBO remains very close to 50% while sample size is ballooning, you might consider ending the test early and trying something new. At this point it usually doesn’t matter which variation you chose to keep, the idea is to try something that can give you more results.

Multivariate Testing

Mulitvariate testing is an interesting variation on the basic idea behind A/B testing. Like the name implies, an A/B or split experiment has several complete designs that compete in an arena. However, multivariate experiments use a single page design and have many different sections that vary within the test.

For example, let’s say Bob is working on his Brewing Kit landing page. He could set the header to vary and enter 4 different variations on that. On the same test, you could also test 2 stock images in the body and 6 different “Buy Now” button designs. As a final kicker, you can play with 3 different lists of features during the same test.

As you begin to look at the results page, you’ll notice a massive list of combinations (there should be a unique combination for every single possible mix of variants). In the previous example for Bob’s Brewing Kit, you’d have 144 different variations (4 headers * 2 images * 6 buttons * 3 lists).

You can think of the results page showing all 144 variations as a supersized split test (with 144 splits!). However, the real magic is in the results page showing the sections separately. While there are 4 sections in Bob’s test, we have a screenshot to show you two:

On that page you’ll see a list of all the header, image, button and list variations you have. And you’ll see some numbers showing how they’ve fared against the other versions of the same section using many of the same numbers like CBO from the split test. For example, you might notice that out of the 2 images, the first variation is doing much, much better.

In fact, it is nearly conclusive (CBO almost less than 5%) that the Dark Brew Kit image is inferior to the Original image. And for the lists, the second variation (Grain First, Taste Second) is doing better than either of them although it isn’t conclusive yet.

You might imagine that the 6 buttons are definitely inconclusive (just think: there are a lot more variations to split the sample size between).

However, it is important to remember that multivariate testing can be rather tricky to interpret. So our recommendation here is to wait until the test completes and rely on the suite’s conclusions. You might have to wait for 10,000-20,000 pageviews before it will come to a natural end, but making the wrong decision is too easy to do in this situation.

You might end up surprising yourself when some of the lousier variants on their own end up creating an extremely powerful combination. For example, what if the Dark Brew Kit image combined with the Malt Flavor, No Taste list had some sort of super power when combined on the same page? You’ll never know if you end the test early!

More A/B Testing Tips

Don’t Assume Anything

I am constantly surprised (and shocked!) when I decide to test something odd or even what I might consider stupid or a waste of time and it wins. If people had a knack for designing the perfectly converting page based on best practices, there wouldn’t be any test suites available.

So, constantly play with the oddest things. If you’ve ever seen an odd sales pitch page and thought “Who in the world designed this hunk of junk?” just remember that it was probably designed through hundreds of iterations. A little red text there, a new button here, a funny picture there: all key ingredients to their continued success.

So my advice here: test everything, even the weird or strange ideas you have. If you have the luxury of traffic, try absurd ideas and see how they go. If not, play it safe for starters, but always remember, you are testing because you don’t know, and unfortunately (as I am reminded nearly every day), I don’t know jack!

When to Use A/B vs. Multivariate

As you will probably recognize, A/B and multivariate tests are extremely powerful, but they each do some things better. We tend to use A/B tests in the initial phase of deployment, that is, to test different layouts or designs.

The main reason behind this is that A/B tests allow for complete page control. You can do whatever you want for each split because it is effectively a unique URL with completely separate HTML.

When you start drilling into specific sectors of the page, you don’t need to replace large chunks of HTML (like the entire sidebar, or header) or the entire page. Instead, you want to surgically replace images or text copy in several different places. This is where multivariate testing excels. However, the more variations on sections that you test, the longer it will take to get to solid assumptions.

Optimize Multivariate Test Variant Numbers

If you decide a multivariate test is for you, try to choose a consistent number of variants for each section. For example, a test with 4 sections, where each has 3 variants except for one that has 9 variants will have a total number of 243 combinations (3*3*3*9).

However, as you wait for a sample size big enough to tell you which of the 9 variants in the last section is best (and gives you a satisfactory CBO), the other sections will have long ago responded with solid CBO’s.

Now you are just wasting time continuing to test those sections. For example, let’s say you have 18,000 pageviews. In the sections with 3 variants each, each variant is given a sample size of 6,000 (18,000/3). The section with 9 variants only gets 2,000 samples each (18,000/9).

Remember how important sample size is?

A wiser choice would be to keep all the sections at the same number of variants. As a counter example, let’s say 4 variants each. That would yield 256 combinations, only slightly larger than our previous example. However, at 18,000 views, you’d likely be done with the test because each variant would have had 4,500 samples.

Disabling Losing Combinations

This is especially useful if you are running a large split test or a multivariate test with uneven variant numbers across sections. The idea here is to stop wasting your precious views contributing to a sample size of something that you already have determined to be either a clear winner or a clear loser!

Let’s look at the screenshot. You’ll notice that the appearance is a little off, don’t worry, that’s just the old version of Google’s Website Optimizer. What I do want you to pay attention to is this: which (if any) of the combinations should we disable?

If you answered “None, unless you have extremely loose standards…” you are correct! These guys need a lot bigger sample size! Just look at their +/- (very big ranges) and CBO’s (not 95% or 5%)… However, let’s look at the worse combinations (at the bottom).

As you can see, combination 2 is getting pretty close to the point of being disabled. Once it hits 5% CBO, we usually disable a combination because it has a very slim chance of beating the original. There is no reason to waste samples on something you’ve already agreed is no good.

Break it up by Clicks

A common practice (and the one I recommended earlier) requires setting up a goal page and hoping the user completes all the necessary steps to getting on the goal page so you can record a conversion. An alternative method is to break each page apart into its own test and measure the clicking of the continue button!

This is a more advanced technique, but let me describe it this way through an example: you have three distinct pages in your funnel: a landing page (welcome!), a signup page (enter your email…), and a thank you page (please check your email now). Usually, you create an experiment on the landing page and placing the goal code on the thank you page like this:

Landing page experiment -> Signup page -> Thank you page / Goal!

This usually works pretty well. Instead, this tip suggests counting the click on a continue button on the landing page as a goal. Additionally, you can create a new and separate experiment on the signup page and make clicking the “Signup now!” button a goal as well!

This serves two purposes: 1) we’re making the experiment page stand alone as a complete experiment and 2) we’re inviting more granular experiments on each page in a funnel instead of the beginning and end of a funnel.

This requires a little javascript hacking, but once you get the feel for it, you’ll do well. First, you need to create the experiment pages (or page) just like normal for the A/B (or multivariate) test. Second, you’ll create a dummy goal page in which there is no way a user could get to it (like ex.com/dumgoal875.htm) and place the goal code on it. Third, you’ll need to add this to your experiment page…

<script type=”text/javascript”>
function doGoal(that) {
try {
var gwoTracker=_gat._getTracker(“UA-XXXXXXX-X”);
gwoTracker._trackPageview(“/YYYYYYYYYY/goal”);
setTimeout(‘document.location = “‘ + that.href + ‘”‘, 100)
}catch(err){}
}
</script>

…where you’ll need to replace UA-XXXXXXX-X and YYYYYYYYYY with the appropriate numbers from the actual conversion code. Fourth, and finally, on either the button or link, you’ll need to place onclick=’doGoal(this);return false;’ into it like this:

<a href=”http://example.com/buy/”>Buy Now!</a>
<a href=”http://example.com/buy/” onclick=”doGoal(this);return false;”>Buy Now!</a>

or…

<input type=”submit” value=”Signup now!”>
<input type=”submit” onclick=”doGoal(this);return false;” value=”Signup now!”>

At this point, each click will count as a conversion for that exact page. This should help you create self-contained experiments that span only one page. The beauty of this is you can string together many different experiments like this:

Landing page experiment -> Goal! -> Signup page experiment -> Goal! -> Thank you page.

Warning! If you are doing a split experiment with multiple URLs, don’t forget to place the modified goal code from about on each variant! A multivariate experiment only has one page, but make sure the modified goal code is on there as well!