Questions? (877) 643-7244
FutureNow Post
Tuesday, Aug. 26, 2008 at 2:52 pm

Does Your Testing Organization Have A Gag Reflex?

By John Quarto-vonTivadar
August 26th, 2008

gag reflex

Does your organization or company have a gag reflex when it comes to testing? Never mind the medicine to fix a problem, are they even prepared to acknowledge that the problem exists?

Ronny Kohavi from Microsoft’s Experimentation Platform had a short post the other day about Dr. Ign├íc Semmelweis (pronounced “Eeg-natz Shem-mel-vise”, those darn Hungarians!), a 19th century doctor, whose observations (and subsequent testing) of what only a generation later would be understood as cross-patient germ infection, reduced mortality 10-fold when applied. It also ran counter to established medical opinion and Semmelweis was subsequently run out of his hospital.

Today, an organization that does that is euphemistically said to have a Semmelweis Reflex, which Ronny quotes from Wikipedia as “a reflex-like rejection of new knowledge because it contradicts entrenched norms, beliefs, or paradigms”. A related adage for this same idea, one I’m sure you’ve all heard, is “don’t drink your own kool-aid” [Admittedly, even that phrase is three decades old now -- how many readers know the events that gave rise to that reference? And who even drinks kool-aid anymore in this Snapple-VentiChai-RedBull world?]

Now, the lesson is not that anyone with an opinion (and a bullhorn) should be considered a prophet and possibly tomorrow’s saint; no, there really are plenty of kooks in the world. The Semmelweis Reflex is not only reserved for groups who are, hmm, let us say, “ignorant of the facts”. Rather, the Semmelweis Reflex is just as prevalent and possibly more dangerous in a group that considers itself experts in its field — the ones who should know better than to reject new knowledge out of hand — just like those fine well-intentioned doctors at Semmelweis’ Vienna medical research institution.

Does your organization not give testing the weight it deserves because you “know” such-and-such is true? Is your conversion rate maxed out at 2% because “that’s the way it’s always been”? or “that’s what everyone else in my industry gets”? or “I really know what my customers want”? Perhaps you indeed do; if so, testing and re-confirming core assumptions is always healthy.

Ronny’s post (download the PDF he provides) is absolutely worth its quick read (and worth sharing with HiPPOs) and it just might start you thinking. And the next time time an expert — and (gulp!) even an expert at FutureNow — tells you that such and such is true, feel free to question it! Ask for evidence, or better yet — try a small experiment of your own to confirm that idea.

And make “because I say so…” a phrase best reserved for children’s bed-times.

Editors Note: John co-authored Always Be Testing with Bryan Eisenberg and is FutureNow’s Chief Scientist. John worked as an astrophysicist at NASA (on parts of the Hubble Telescope that work). So when John talks about the science of testing, we listen since when a mistake on a NASA multi-variate test occurs the impact can be a lot worse than that of an inefficient “Add-To-Cart” button.

Add Your Comments

Comments (4)

  1. Nice article! I couldnt agree more to the point that companies have to acknowledge the problem to fix it.



  2. … and want to fix it. It’s so easy (I know) for companies to just become complacent. What’s the difference between 20 and 22 signups a day, it’s only 2 more signups right. =)

  3. It’s a Nice article! I like very so much.

  4. Thank for sharing.

    It’s great information.

Add Your Comments


Print this Article

John is the co-author of the best-selling Always Be Testing and 3 other books. You can friend him on Facebook, though beware his wacky swing dancer friends, or contact him directly at

More articles from John Quarto-vonTivadar

Marketing Optimization Blog
FREE Newsletter Sign-Up
send it once every: