Lesson 6 is basic scientific knowledge. You have to limit your variables. In user testing, it also helps to keep the subject “fresh” by testing a few tasks. If you test too many tasks the user may get bored, tired, etc. You may also get too much prior-learning coming into play.

The first mistake has been proven true by about half a century of consumer surveys wherein consumers wherein what consumers say they want IS NOT what they want. The most notable example of this would be Coke’s research prior to launching “New Coke” – people claimed they liked new coke better, but that wasn’t at all how it turned out in the marketplace.

The second rule has been proved by centuries of scientific research and law. In court, you’re not allowed to ask a leading question. And in psychology, your research is invalid if you tainted the subjects’ responses with leading questions. The same applies with user testing.

The third mistake is somewhat of an extension or combination of the first two: don’t take what they say they want at face value. As Henry Ford said, if he’d listened to the customer, he would have designed a better horse and buggy!

Mistakes 4 & 5 are really just encouragement or warnings against getting discouraged. Feel free to ignore them and get discouraged if you like ; )

Lesson 6 is basic scientific knowledge. You have to limit your variables. In user testing, it also helps to keep the subject “fresh” by testing a few tasks. If you test too many tasks the user may get bored, tired, etc. You may also get too much prior-learning coming into play.

Not sure if this is what qualifies as “market research” but the mistakes aren’t really presented as the distillation of some statistical analysis – they’re what one experienced User Testing professional passed onto her student and what he found to be most true and useful in her advice.

- Jeff