Still–some of your comments are quite correct. But in reality, an experienced market researcher will often recommend a combination of quant and qual methods (depending on goals and budget); the default in quant research is not a questionnaire built largely around Likert scales.

We could go on forever, but the main thing I’d like to add is the issue of sample and population selection. Almost all surveys these days seem to be carried out so unscientifically that it probably doesn’t matter what questions you ask, the results will be invalid. I saw a survey last week that claimed that over 50% of businesses were actively using Twitter for their business! For crying out loud, 50% of all businesses aren’t even online yet. They’d surveyed a mailing list subscribed to mostly by digital marketing agencies, and assumed the results would be good for the whole business world.

What I like about your approach is that it generates qualitative responses, which can be examined individually. I’m not sure the question format in your examples is the real problem, I’d say that was lack of expertise in surveying. But unless you are a well trained survey professional, qualitative is the way to go. It’s just a safer weapon in inexperienced hands.

Just to establish some context, grokdotcom is an online persuasion and marketing blog, speaking to owners of Internet businesses and other online/internet marketers and professionals. So my comments quite admittedly do NOT apply to the use of Likert-scaled surveys within various academic fields and specialties such as psychology, sociology, etc.

So, for the audience in question, to say that I’m only talking about the abuse of Likert scales may be technically true, but is misleading as most people think that abusive practices represent a minority, where as probably more than 95% of Likert scaled surveys used within marketing, PR, customer research, etc represent misuse/abuse of the tool.

I’ll put it this way, I’ve seen hundreds of various scaled surveys and I’ve never seen one that provided valuable insight. Almost every open-ended survey or even customer service transcripts I’ve come across have yielded gold. Even mediocre open-ended surveys typically have nuggets.

That either type of survey can be abused (or used effectively) misses the point. In actual use Likert-scaled surveys (without any open ended or written add-ons) are:

* vastly more likely to be poorly constructed
* far less tolerant of sloppy construction
* far more opaque in the answers they yield
* far more likely to have their numerical, and therefore seemingly “objective,” results represented as “fact”

Given the reality on the street when it comes to these surveys, I stand by my advice to favor open-ended questionnaires. For those with expertise in creating Likert-scaled surveys with “an understanding of their [questions] convergent, internal and external validity,” than I’d say, have at it. But that doesn’t even remotely describe 99.9% of marketers, customer research teams, etc.

- Jeff

In nearly every application you can do it poorly or do it well. I could point to hundreds of thousands of blogs that are abysmal, but I wouldn’t say that blogs aren’t worth doing.

Open ended questions are open to the same type of question bias and can be written in ways that will generate responses of a certain type.

Likert scales can be used to consistently track behavior and attitudes over time. I’d argue that open ended questions make this far more difficult.

Are you doing a keyword analysis of the responses? How do you interpret the open ended results? Do you interpret them the same from time period to time period? There’s a lot more subjectivity in open ended questions is there not?

You may take a person’s comment out of context, or that person may not have accurately communicated what they meant.

Using both Likert scales and open-ended questions is a better strategy. Throwing the baby out with the bath water just doesn’t make much sense to me.