More DIY Disasters

I have written before about the pitfalls of DIY research. Online tools such as Survey Monkey and Zoomerang make people think that anyone can design and execute a survey. And this is true, anyone can. But many people who try DIY surveys wind up with problems. DIY surveys do have their place, but too many people try to use them when they should not. I think of them as the Draino of research. You reach for the Draino when you have a simple clogged drain, but when the problem is more systemic, you call a plumber.

We hear about many of these disasters because clients call us to see if we can help them salvage what they have done. We try our best to help them, and in most cases some valuable learning comes out of the exercise. But we came across a couple recently that caused real problems.

The first project appeared simple enough. The problem is that the first question has multiple choice answers about product use, and more than one choice is appropriate. In fact, most consumers would have more than one choice. But the client inadvertently programmed the survey to only accept one answer. This meant that when people checked a second or third answer, the software deleted their previous answer. And since the software did not allow for a rotation of the answer list, this meant the survey reported a disproportionate, suspect number of people using the product at the bottom of this list. To complicate this problem, answers to this question drove the logic for almost all remaining questions. We could salvage some of the information, but most of it was lost.

The second project was a DIY survey where the client intercepted respondents on their own website. They hoped to get 400 interviews. They promised all respondents who completely the survey a $10 Amazon gift certificate. One little problem – the DIY software lacked quota control, or at least lacked one the client could find. Within 24 hours, they had about 5,000 completed interviews. Suddenly they were liable for $50,000 in incentives instead of the $4,000 they had budgeted.

So the next time you have the need for marketing research, ask yourself if you should reach for the Draino or call a plumber.