The first mistake has been proven true by about half a century of consumer surveys wherein consumers wherein what consumers say they want IS NOT what they want. The most notable example of this would be Coke’s research prior to launching “New Coke” – people claimed they liked new coke better, but that wasn’t at all how it turned out in the marketplace.

The second rule has been proved by centuries of scientific research and law. In court, you’re not allowed to ask a leading question. And in psychology, your research is invalid if you tainted the subjects’ responses with leading questions. The same applies with user testing.

The third mistake is somewhat of an extension or combination of the first two: don’t take what they say they want at face value. As Henry Ford said, if he’d listened to the customer, he would have designed a better horse and buggy!

Mistakes 4 & 5 are really just encouragement or warnings against getting discouraged. Feel free to ignore them and get discouraged if you like ; )

Lesson 6 is basic scientific knowledge. You have to limit your variables. In user testing, it also helps to keep the subject “fresh” by testing a few tasks. If you test too many tasks the user may get bored, tired, etc. You may also get too much prior-learning coming into play.

Not sure if this is what qualifies as “market research” but the mistakes aren’t really presented as the distillation of some statistical analysis – they’re what one experienced User Testing professional passed onto her student and what he found to be most true and useful in her advice.

- Jeff

Meaning sit down at your local Starbuck and offer a coffee in return for testing your website.

It is cheap – and the user is not prepared (meaning they are giving you a more honest answer).

Btw, I always ask — at the completion of the test — “Please rate that on a scale of 1 to 5, with 5 being the best.”

Good comment.

Of course, the primary focus of a usability study must be on task completion, but eliciting users’ subjective responses to key content or functional elements is important to clarify their perceptions and give the project team a chance to hear what their users like, value and need. As long as these issues are explored at the _conclusion_ of a task, they don’t interfere with the ability to get solid observational data.

Social media marketing and testing techniques top the trends in this week’s links. Check ‘em out.
……

Correction:

> If you’ve done 20 tests in a row and, let’s say, 5 out of 20 were failures…

I actually meant, “let’s say only the last 5 tests out of 20 were successful and the first 15 all sucked, but you were making iterative changes all along…well the last 5 successful tests are saying you’ve got a usable design.”

John (Site Doublers), I *LOVE* your “dummy task” idea! Will totally use that.