We all know that testing is a great tool to use when deciding between images, content and layout, but have you ever considered it to make business decisions? Recently, a client came to me with a predicament: UPS is raising their shipping costs by 5%. My client wanted to know if it would be best to swallow these costs or pass these charges onto their customers. Obviously, they didn’t want to jeopardize their customers; however, incurring greater shipping costs was something they weren’t crazy about either. Fortunately, our groundbreaking OnTarget strategies can address problems such as this. So, how do we do it? Test it, of course!
I worked with them to create the following research question for the test:
With UPS increasing our rates, we will encounter a 5% loss if we swallow this cost. However, before making a decision, we want to test what our potential loss may be if we pass this loss onto the customer. If the potential loss is higher than the cost of swallowing this 5%, it would be wise for us to either take the 5% loss or seek out other shipping options.
Whenever you want to create a test, first think about why you want to test it. This is true for design tests, copywriting tests, content tests, headline tests, image tests, and business strategy test such as the one we are illustrating here. In short: any test begins with a question. You should be able to formulate a hypothesis or at least a research question. If you can’t get this far, then it’s probably not the best test to run.
It’s impossible to draw conclusions from data you can’t trust. This means you not only need to know what each discrete piece of data you are collecting represents, but also be able to ensure that each piece of data is clean. Diligent collection and tracking of data is an essential component of any test. This is when being anal is a plus. Any other tests being conducted on the site should be paused while this test is running. You also want to make sure you have solid, consistent data collected prior to running the test so you can look at not only your winning variation but also compare to past data to ensure you’re making the right choice.
It also wouldn’t be wise to run this particular test during a time where you’re experiencing a large influx of late stage traffic. This means that holiday seasons, or times when there’s an annual sale would not be the right time to test. Yes, I remember my recent post about why the holidays are the perfect time to test, but there are caveats to that, and this is one of them. Holidays and sale times push visitors toward the late stage of buying more quickly, a moment when they may be more heavily influenced by price, so tests that influence pricing should be put on hold for another time. Be sure to give the key issues involved in your test, and how other business factors could skew or be skewed by those issues, some critical thought before you decide when to run a test. Each test is unique, and comes with its own constraints that need to be considered and worked into the testing solution. It’s part of ensuring clean data.
Testing is a valuable tool that can be used in a variety of situations, so don’t forget it when you’re making higher level business decisions bigger than what color button works best (but test that too)! We’d love to hear how you used testing to inform your business strategy, so share your business strategy testing story now.