Are Focus Groups Scientific?
by, 04-23-2011 at 09:35 PM (1300 Views)
Once upon a time, I was a member of a focus group. They paid me $20 to answer questions about graham crackers. I was chosen because our household consumed a lot of graham crackers in those days; I had toddlers, what can I say?
I thought I would be asked questions about which was better, honey or cinnamon? Do I like crisp or chewy? What qualities are best for dunking in milk?
Nope. Hardly any of the questions were about the product. First they wanted to verify that we really did use at least a box a week. The other questions were about my shopping habits and about my media use— what magazines I read, which TV and radio shows I tuned in, did I attend ball games, and stuff like that. This experience left me with THREE REASONS to believe that FOCUS GROUPS ARE UNRELIABLE.
#1 Lies of Omission.
We also attended a church where the pastor was known to launch into diatribes about the filth on TV, of which the daytime bed-swapping was the worst. This church had a lot of stay-at-home moms and grandmas, and it had a very active women's Bible study group. During the chit-chat time over coffee, some women would talk about shows they watched on TBN (religious cablecast). Nearly every show they would discuss aired in the mornings. C'mon, it does not take Sherlock Holmes' deductive powers to notice that folks were oddly silent about afternoon viewing. No one in that group was going to talk about soaps, especially when the pastor's wife was in attendance.
At that time in my life, I had a radio that would pick up the audio of TV shows. I listened to OLTL nearly every day. So, technically I was not "watching" any soaps regularly. Between the young children and a part-time job, I felt fortunate to be able to arrange my day so that I could listen and clean at the same time. So back to the topic of the focus group, no, I did not tell them that I was hearing every commercial that aired!
Soaps have a sleazy, low-brow stereotype. People are going to hold back.
#2 Lies of Pride.
I have seen this at work in beauty shops, and it is so universal in human nature it is going to skew any focus group in a similar manner. People offer public judgments to make themselves look good, not because it is what they really think. Furthermore, the focus group was getting paid by people who had chosen us only because we bought graham crackers. Even when trying very hard to not be biased, there was a certain amount of peer pressure to give the "right" answers.
#3 It is a FOCUS group!
It is not a BIG PICTURE group. Look how much disagreement there is between people who actually like soaps! If I say, "Ford is not a rapist, only an opportunist," we are not going to have a consensus! If people are recruited for a focus group and know nothing about the character histories, when shown the prom night seduction scene in isolation, MOST would form their opinion based on seeing that a hot guy stopped the minute the hot chick freaked and asked him to stop.
Focus groups have a place, and that place is to do some fine-tuning within a concept. They will not be a reliable guide for flipping concepts.