Stop Flying Blind - A book by Michael Mace

8. Understanding market research

The three perspectives a company needs in order to map the future are competitive analysis, market research, and advanced technology analysis. This week we’ll start our discussion of market research. Considering how widely used market research is, it’s surprising how many people don’t understand how it’s done and how to use it. So we’ll start with the basics…

At its core, competitive analysis is about intuition, supplemented by objective information. Market research is the opposite; at its core it’s a science. Intuition plays only a secondary role, seeding the questions that a good market research study tests. Market researchers hate to guess. They take pride in the rigor of their studies, and like a good science experiment, good market research is repeatable — if you ran it over and over again, you’d get basically the same results every time.

Another important difference between market research and competitive analysis is that market research has a well defined, accepted role in most corporations. It’s hard to find a large company that doesn’t do at least some market research, and there are respected professional training programs for researchers.

Unfortunately, most of the companies I deal with use market research only in a limited, tactical way. Market research teams are usually chartered as service groups — they take specific questions from various parts of the company, deliver the answers, and move on to the next project. Usually those questions focus on measuring what people think — “will you vote for this presidential candidate?” or “what do you think of this brand of soap?”

It’s much less common for market research teams to be asked how customers think. What motivates them? What do they care about? How do they make decisions? This sort of strategic research takes a fair amount of time and uses up resources that business units would prefer to spend on immediate problems. But I think it’s incredibly valuable, because if you know how customers think, you can predict how they’ll react to changes in the marketplace. Like good competitive analysis, good market research doesn’t just help you know what’s happening today, it helps you predict the future.

I think the right role for a market research team is one that balances tactical services and strategic vision. This chapter is about how to make that happen.

The basics of market research

I think it’s very important for all managers to have a basic grounding in how market research works and how to tell good research from bad. Market research is an incredibly powerful tool. When it’s done right, it can be the foundation of a wildly successful corporate strategy. When done wrong, it can lead an entire company to march boldly over a cliff. You wouldn’t feed your body spoiled food, so don’t feed your mind spoiled ideas.

Unfortunately, unlike spoiled food, spoiled research often smells great and comes wrapped in a very attractive PowerPoint presentation. Here are some tips to help you look beyond the slides.

How market research is conducted. There are usually two important players in any market research study, the in-house manager and the supplier. The study’s in-house manager is an employee of your company, usually a professional researcher, who takes the company’s questions and figures out how to get them answered with research. The researcher will determine what kind of study to use, and will contract with an external research supplier to conduct the actual study.

The research supplier is an outside company that does nothing but conduct market research studies. Suppliers have specialized facilities and resources that most companies couldn’t afford to keep in-house. For example, they’ll maintain a large list of people who can be contacted for a research study. Usually suppliers specialize in particular types of studies. Companies that do focus groups (see below) will have offices with the proper rooms, and people trained to run the groups. Companies that do numerical surveys will have banks of phone operators trained to conduct surveys by phone or in person.

The in-house manager works very closely with the supplier at the beginning and end of the process. At the beginning, they work together to design the survey, including the exact wording of any questionnaire, and the rules for how people surveyed (the “respondents”) will be selected (their “qualifications”). At the end, they work together on the interpretation of the findings. The research supplier delivers a final report, which the researcher may pass along to the company as-is, or may replace it with a revised report tailored more to the internal needs of the company.

Quantitative and qualitative research. There are two basic types of market research, quantitative and qualitative. Quantitative research gives you hard numbers — it’s a scientifically-conducted survey that gives you statistical information on the market as a whole. Opinion polls are quantiative research.

Qualitative research is any market study that doesn’t give you reliable numbers. The most common qualitative research is a focus group, in which a small number of people spend several hours discussing a topic, while researchers behind a two-way mirror watch them. The number of people in the group is too small to give meaningful information about the market as a whole.

Uses of qualitative research. Focus groups and similar studies are often used as fodder for an advertising campaign. You’ll get a group of target customers in a room and study how they talk — what words do they use, what mannerisms, and so on. This helps the creative team develop ads that speak directly to the customer in his or her own terms. Often you’ll find a particular person in a group who really exemplifies the type of customer you’re trying to reach. You can show video of this person to the creative team and say, “give me an ad that appeals to him.” How often have you seen brochures, websites, or even ads written in jargon that’s difficult for anyone outside the company to understand? Often this happens when a company doesn’t have a clear understanding of the person they’re trying to communicate to. Focus groups can help with this.

You can also use focus groups to check for potential disasters with new brand names and logos, before you make a mistake that’ll take a lot of money to correct. A legendary example is the Chevy Nova, with “no va” meaning “doesn’t go” in Spanish. Unfortunately, that one turns out to be so legendary that it’s not true, but I’ve lived through some real cases. For example, a company I worked for once came very, very close to giving itself a name that sounds like “excrement” in Chinese. It was especially ironic because the company had a large number of employees in China.

In the computer industry, we learned years ago that our love of using numbers in a product name could get us in trouble overseas. Apple once produced a computer called the Macintosh Performa 4400. The character for 4 in Chinese resembles the word for Death, and the character for 0 resembles the word for Again. So in Chinese culture 4400 means something like “die die again again.” Needless to say, the product carried a different number when it went on sale in Asia.

But it isn’t just numbers and names that can get you in trouble. Almost any graphical element, or even color, can have unanticipated meanings in various cultures. For example, Macintosh computers once used an on-screen icon of an upraised hand to signify that a function had stopped working (the hand was like the raised hand of a policeman directing traffic). In the mobile phone industry, one of the major software companies once adopted a hand outline as its logo (which was apparently meant to indicate that the software powered devices you hold in your hand).

The problem with all of this is that the hand, held flat out, is a serious insult in some cultures, meaning more or less, “let me feed you some manure.” Getting that message from your PC or mobile phone isn’t very appealing.

This hand icon was displayed when a Macintosh computer reported to the user that a program had crashed (or “unexpectedly quit,” as Apple liked to put it). It replaced a bomb icon that was understood universally across all cultures, but communicated so clearly that it scared the bejeezus out of nontechnical users.

For a time, this was the logo for Symbian Epoc, an operating system for smart phones. If you haven’t heard of it, don’t feel bad — Symbian eventually dropped both the logo and the product name.

In another version of the Macintosh software, an animated image of a hand with fingers counting from one to five was used to indicate that the user should wait. It turns out that the various combinations of fingers meant insulting things in several countries, making this the first cross-culturally insulting icon.1

You can use focus groups to test this sort of thing, showing a group of local customers the proposed logo or name, and getting their reactions. Personally, though, I don’t think a full focus group is necessary. Just ask the locals who are on your staff, or if you are selling through a distributor, ask some local people who work for them. They can give you a faster reaction at virtually no cost.

Focus groups are also sometimes used to collect product requirements. You get a group of users together and talk with them about what they like about a current product, and what they’d like to see in the future. I think doing this is a big mistake. Because a focus group is not scientifically structured, the reactions you get from it aren’t projectable to the whole market. You might have a group of freaks who’ll mislead you into creating a product that’ll sell to only 1% of your target market.

But because focus groups are a lot cheaper than quantitative studies, it’s very tempting to try to use them as a substitute. For example, I’ve seen product plans with statements like “60% of the people in our focus groups said they liked the product.” While this is a factual statement, it’s also meaningless because the group wasn’t a scientific sample of customers.

The usual excuse for doing this is, “it’s better to use the focus group than not have any research at all.” That’s rubbish. The “data” you got from the focus groups is no better than a coin toss. You’d be much better off letting the smartest people in your company make an educated guess.

Things to look for in good qualitative research. The best focus groups are great conversations in which you get to eavesdrop, so you want to look for conditions that’ll produce a useful conversation. That means a good setting, good people, and a good moderator.

The location should be in a city that isn’t dominated by your industry. In your industry’s hometown there’s too much risk that you’ll get some insider know-it-all who’ll take over the conversation. So don’t do movie focus groups in West Hollywood, don’t do car focus groups in Detroit, and don’t do computer focus groups in San Jose.

A good focus group facility has a room with one or more walls of sound-proofed one-way glass, so you can recline in comfort and eat pizza and M&Ms while the subjects talk. Ideally, the camera filming the group should also be behind the glass, so the participants don’t fell self-conscious. There should be a good sound system, so you can easily hear what’s going on inside the room. And the whole building should be reasonably insulated against outside noise. I once sat through a focus group where the participants almost had to shout to be heard over the chants of a protest group marching in the street outside the building.

The moderator is the person who guides the focus group, asking questions and keeping the conversation on track. A good moderator is like a good television talk show host — very alert to people, and capable of drawing them out by asking good followup questions. But unlike many TV shows, the moderator can’t be the center of attention. Sometimes a moderator will have an agenda, a preconceived idea of what the group should say, and they’ll subtly impose that on the group. You don’t want that.

You also want a good sampling of participants. You should never fill a focus group with company employees, and I’m uncomfortable with even using friends of employees. There’s too much chance of bias. A good focus group firm will have a large stable of potential participants that it can screen for the attributes you’re looking for, recruiting a good cross-section of the customers you want.

Typically the cost of a good set of focus groups will be around $20,000, and can go a lot higher depending on how many cities you want to visit. If I was short on money, I’d try to do one good set of groups in a single city rather than doing a series of groups on the cheap in several places. You’re not doing numerical research anyway, so the quality of the conversation is more important than the number of conversations you have.

Finally, you should make sure you get videotape of the groups. Get actual digital tape, not just a DVD — DVDs are notoriously slow and difficult to transfer into editable form on a computer (I learned that one the hard way). I also like to control the editing of any summary created from the video. At a minimum, you should tell the person doing the editing exactly which excerpts you feel are most important. If you leave this up to the focus group company, you may end up with a summary that misses the things you felt were most important.


Please click here to rate this section (the link will open a one-screen anonymous survey, and you’ll get to see the results after you take it).

Next week: Quantitative research.

  1. Almost any gesture or hand position you can imagine turns out to be insulting in somebody’s culture. Check it out. [↩ back]

5 Responses to “8. Understanding market research”

  1. Avi Greengart said on June 5th, 2006 at 9:35 pm

    I’ve seen a lot of focus groups misused to try to discover latent market needs. That’s better done with social anthropologists or just by sending your product managers into the real world every once in a while (make it a goal tied to compensation, or they’ll never leave the building. There’s always some fire that needs to be doused somewhere). There’s no question that good ideas come out of a focus group – you’re asking random people to think about a problem you’re trying to solve. But don’t set up a focus group with this as the goal.

    I’ve also seen focus groups misused to test pricing models. That NEVER works – ask a bunch of people being paid $50 – 100 to sit around a table and talk what they’d pay for something and they’ll make up whatever number makes the moderator nod his/her head slightly. It’s like the Simpsons episode where Maggie was thought to be a genius because she was following clues Lisa was unconsciously giving her.

    Focus groups are best used for getting gut-check reactions to names, brands, and experiences. Some scenarios:

    Too often a company gets overly conservative with marketing because they fear customer backlash – a focus group can be used to safely explore possible reactions.

    Technology companies often assume far too much about the knowledge or environment a product will be used in. Get a bunch of real people together and discuss home networking, for example, and you’ll discover that asking them to open a port on the router to allow external access presumes that they even know they have a router in the first place.

    Considering buying a company for its brand? You might want to see what associations people have with that brand. I was actually part of a focus group convened to screen ads for Zenith plasma HDTVs right after LG bought the company. They got what they wanted in terms of direction on which ad to run, but what struck me as sadly hilarious was that the whole effort was misguided. Zenith was an early HDTV pioneer and had a fair number of patents on the technological underpinnings. The LG people knew this and apparently assumed that everyone else did, too. At the time “LG” had no brand equity whatsoever, while it was clear from the focus group that “Zenith” actually had negative brand equity: several of my focus groupies thought the company had died about 50 years ago. Word associations included “large console radios,” “my grandfather’s TV,” and “old, outdated, but solid.” The moderator asked, “would you buy a Zenith HDTV?” This elicited: “No.” “No.” “Why would somebody use the Zenith name on a Japanese HDTV?” and, after the moderator looked pleadingly towards the glass wall, someone meekly offered “well, if it was a really good TV, sure.”


  2. Mike Mace said on June 6th, 2006 at 7:36 pm

    Great comments, Avi. Thanks!

    >>sending your product managers into the real world every once in a while (make it a goal tied to compensation, or they’ll never leave the building

    Agreed. Even more than making it a part of compensation, the leaders of the product management team need to set an example by doing some customer visits themselves. Eventually it becomes an accepted norm within the group.

    I’ve also seen focus groups misused to test pricing models

    Yikes! A coercive atmosphere AND an invalid sample.

    I agree with your suggestions on proper uses of focus groups. Good stuff.

  3. Marketing Research said on November 2nd, 2009 at 12:25 am

    This is a very good and informative post.

  4. Burn Software For Mac | The Blu-Ray Way said on February 7th, 2013 at 3:30 am

    [...] any data loss. Good luck! For more iPod support, see http://apple.comPowered by Yahoo! AnswersMichael asks…Is Scrivener a trustworthy and best tool to write a novel?I'm thinking of downloading…Scrivener a trustworthy and best tool to write a novel?I'm thinking of downloading it or buying it, [...]

  5. rosstaylor said on March 7th, 2013 at 4:11 am

    This is very nice Blog post , specially i like the description about Uses of qualitative research, Quantitative and qualitative research.
    Thanks for the Post on Market Research Report.

Leave a Reply