Lousy Research

The San Francisco Chronicle recently gave most of a full page to the review of power toothbrushes. The story is at http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2009/02/25/DD481635MI.DTL&hw=take+five+toothbrushes&sn=001&sc=1000.

The fact that they showed almost life size pictures of the toothbrushes should have given me the hint that the story was mostly fluff. In essence, this was an IHUT, and their approach bothered me. First, they pitted a high end Sonicare (retail of $169.99) against four lower end brands (retail of $4.89 to $14.19). Then, they had different reporters test different toothbrushes. I know it was only meant as a review, but what an awful approach.

Your thoughts?

Cell Phone Only Households

I have been watching the trend toward cell phone only households for some time. In fact, an article about this appeared in our newsletter about a year ago. If you are interested, the link is http://www.cooper-roberts.com/newsletters/Q108_Newsletter.htm#1. This trend can have a significant impact on the representation of telephone surveys. It is one reason we frequently recommend an online survey instead of telephone.

Many people have felt that this phenomenon was primarily limited to certain areas, like the west coast and the northeast. But a recent estimate by the CDC (I know, it seems odd that the Center for Disease Control would be interested in cell phone usage) claims that the largest proportion of cell phone only households is in Oklahoma and Utah (both estimated at 26%). The next highest states are places I didn’t expect – Nebraska, Arkansas, Idaho, Iowa and Kentucky.

Who would have figured?

Holidays and Market Research

I wore green this Tuesday, so did 82% of other Americans. I had a corned beef dinner at home last night, 33% of other Americans had some type of Irish meal at home last night. I am not of Irish descent, nor are 88% of other Americans. Still, many of us celebrated St. Patrick’s Day. It fact, Americans were expected to spend $3.3 billion on St. Patrick’s Day related purchases and activities this year. Cinco De Mayo is growing to also be a major day of celebration in the U.S. Celebration and expenditures on these holidays pales compared to the big holidays like Thanksgiving, Christmas and New Year’s.

So what happens if I take a survey on St. Patrick’s Day, or on Cinco De Mayo? Will my answers be biased in any way? We only buy and cook corned beef once a year. What if I had been asked to complete a survey on recent meat purchases? Cabbage, potato and Irish beer consumption also indexed high this week. While the foods of St. Patrick’s Day probably don’t see a lot of research (except beer), the foods of Cinco De Mayo see a lot more research. Think salsa, guacamole, tortillas, and beer (again).

If we tried to avoid all holidays, there would be very few weeks that we could do surveys. So that is impossible. Naturally we do avoid the big holidays. Other then that, you have to use some common sense. Ask yourself if the results could have had any holiday bias. Guinness may or may not want to complete surveys near St. Patrick’s Day – depending on their objectives.

Have you experienced any survey bias associated with holidays?

Ethnography and the Recession: Part II

I just returned from an MRA (Marketing Research Association) board meeting in New York. This was my first trip to New York since last June. It was somewhat of a shock for me to discover how depressing the mood is in New York. Clearly the impact of the financial markets is having a huge impact on the psyche of NYC. This led me to add to my list of an ethnographer’s observations on the recession.

• There was only one person ahead of me in the line to catch a taxi at JFK – compared to 40-50 people on my last trip.
• One of my “usual” restaurants, which had always been packed, was about one-third full, and they were running a $35.00 prix fixe meal offer.
• The organic salad, carry-out place that usually had a line out the door for lunch was almost empty.
• Many restaurants were offering everyday 25% to 50% discounts on a bottle of wine.
• The busiest restaurant in Times Square was McDonalds.
• My hotel dropped the room rate twice between my initial booking and my arrival.
• Fellow MRA board members were sharing tips during our breaks on what their firms are doing to cut costs.
  

Now I wonder if these are temporary lifestyle changes, or are Americans permanently changing the fundamental way they do things. Your thoughts?

Let it snow!

The Northeast is being slammed by a late winter snow storm today. Major cities like Boston, New York, and Philadelphia are reporting substantial accumulation, delayed or canceled flights, and school closures. Even our clients in Baltimore could not get to their offices today. Many people stayed home from work today or are telecommuting.

What does this mean for marketing research? I have written before about the impact of sample deployment and the time of day. Without proper sample management, this can distort study results. A big Nor’easter, like the one hitting today, could also impact study results. Because of the storm, and many people staying home, are you likely to get more or less interviews from the Northeast today? Is their frame of mind different today? Will you get a different demographic profile today, perhaps more working moms than usual?

These are all things to think about when deploying sample. I’ve always said “I love a good snow storm, as long as I don’t have to travel in it”, but now I might also say “I love a good snow storm, as long as I don’t have to deploy sample”.