Questions? (877) 643-7244
FutureNow Post
Wednesday, Jul. 9, 2008 at 1:00 pm

Video: How to Do A/B Split-Testing on Lower Traffic Sites

By John Quarto-vonTivadar
July 9th, 2008

Dr. Ralph Wilson of Web Marketing Today spent a few minutes interviewing FutureNow’sĀ  Bryan Eisenberg about testing on sites that have little traffic. You can view the video below. You may also be interested in reading more about the hierarchy of optimization when you are done viewing the video.

YouTube Preview Image

Bryan and I have co-authored a new book all about testing and helping you figure out what to test. It’s called Always Be Testing: The Complete Guide to Google Webiste Optimizer (published by Sysbex/Wiley) and we’re expecting it out next month; you can pre-order it now on Amazon.

Add Your Comments

Comments (6)

  1. Always nice to see Dr. Wilson. I’d say this is good advice even for 100 conversions/week. When a test run is going to take a minimum of 3-4 weeks, you can’t afford to test things at random, mess with low-impact changes, or try to run a huge, multivariate test.

    One big problem I have on low-traffic sites is the increased temptation to check results too early. As a research psychologist by training, I ought to know better, but I still find myself peeking in the first week and wanting to stop the test when my favorite version is ahead. It’s a bad, bad habit.

  2. Also, don’t forget that if a test runs for too long, the result might be affected by seasonal fluctuations in your specific business.

    Fx. running a test on a skiing trip landing page from November to February is bad, because the visitors are in totally different places in their buying cycle when comparing November to February, and thus they will produce different behavious and results.

  3. It’s important to remember that small traffic sites may not have a vast supply of customers to test with anyways. If we have a proven message that brings customers in, and if that’s only 1 customer a day for instance, testing a new strategy might lose that 1 customer for us.

    Still waiting for the perfect win-win.

  4. The main problem with low traffic websites are that they do not have much power to experiment or sometimes it an prove detrimental.

  5. these are two great issues to bring up! I would add that:

    1) low traffic sites do have the issue of sufficient traffic but that is far less than is commonly understood, esp for people who are testing with a tool like GWO that (unfortuantely) distributes the test evenly across all variations. There are well-known techniques that allow statistically good testing with far less traffic than, say, GWO, requires

    2) the fact that testing can be deterimental is certainly an issue but if no testing is being done then the only rationale prediction for the result of testing is “no change” — if you actually “knew” that a change would increase conversion (or whatever goal you measure success by) then you would simply implement. And if you “knew” the effect would be bad, you wouldn’t test at all, you’d jsut discard that idea. So one actually doesn’t know if improvement will occur or not, which suggests that a sub-set of traffic should be exposed to traffic (see point 1)

    in addition, testing “blind” is the biggest cause of “deterimental” testing — what you really should be doing is working with an expert who knows what tests to do, their relative importance, and can help you establish for yourself the opportunity cost of NOT testing.

  6. Always be testing for sure. Never heard of split testing though.

Add Your Comments


Print this Article

John is the co-author of the best-selling Always Be Testing and 3 other books. You can friend him on Facebook, though beware his wacky swing dancer friends, or contact him directly at

More articles from John Quarto-vonTivadar

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: