Last time I talked about the need to segment the market if you’re designing a new type of product. If you design a product to please everyone, chances are you’ll end up with inoffensive pablum that excites no one. That works pretty well in politics, where voters have only a couple of choices. But in new product design, where consumers can choose from an almost infinite range of new products, unexciting is usually deadly. So you should optimize the product to make one segment of customers deleriously happy, and not worry about the rest.
Unfortunately, segmenting the market for a new type of product is a lot harder than you might expect.

There’s a huge amount of accepted wisdom on how you’re supposed to use research to identify market segments. But most of it is designed to refine the segments in an existing market — for example, what’s the under-age-12 market for tennis shoes like? When you apply those same processes to defining the market for a new type of product, something nasty happens: you can’t find any segments. The reality is that market segments for a new category of product don’t exist until that product is delivered. Segments gradually coalesce from a feedback loop between the desires of customers and the products that companies offer to them.

The process is a little like the way that astronomers say the solar system was formed. You start with a big cloud of gas and dust. Small lumps and thick areas in the cloud slowly draw together under the influence of gravity. Wait long enough, and stars and planets will eventually emerge.

When you do research on potential new markets, you’re searching around in the cloud for thick spots. The evidence will be vague and contradictory, and you can easily miss it if you’re not careful. The trick is to look not for segments themselves, but for groups of people who share desires or other characteristics that you can mold into a new segment.

For example, there was no real market for sports utility vehicles in the United States until some clever folks at the auto companies called it into being. Early civilian jeeps were sold as farm tools, believe it or not. But after the disappearance of the station wagon, there was a need for cars with a lot of carrying capacity and with a less domesticated image than minivans. Virtually no car buyers would have thought to ask for an SUV, but when offered a car that could haul a lot of stuff and also had a buff image, people jumped all over it.

.

How to find the lumps

As I mentioned in Chapter Nine, a lot of research companies are happy to sell you ready-made market segmentation schemes that they have derived from demographic data. These segmentations are built around age, income, and other basic characteristics of the population, and usually split a country into about a dozen groups, each with around 6-12% of the population.

In some cases these segmentations can be useful, especially if you’re selling a product that shoots for very large generic markets, or is closely tied to age or income (TV shows and blockbuster movies come to mind). But for new product categories, especially in high tech, I’ve found that generic segmentations are close to useless. They’re backward-looking, telling you how people have behaved in the past, not what they’re going to do in the future. And the segments aren’t cut the right way.

For example, a generic segmentation will tell you that young people tend to be early adopters, something that’s often true but is also meaningless if you happen to be designing a product for older people. The best customers for the RIM Blackberry mobile e-mail device have been business professionals over age 40 working on Wall Street. A generic national segmentation wouldn’t have ever pointed you at that segment, because it’s too small to show up in a mass national survey. And most generic segmentations point new tech products not at over-40 adults, but at 20-something technology lovers. There are some companies that have tried to make e-mail devices for the under-20 crowd, most notably the Danger Hiptop. But they have achieved only a tiny fraction of RIM’s success.

So you need to do your own segmentation. But even then things are still tricky. Traditional research methodologies were designed in the consumer goods industry, in which it’s pretty easy to tell who your customers will be. If you’re making a new laundry soap, you talk to people who do laundry. The traditional research approach says you start with some very general focus groups with those laundry-users. Those groups help you think of some guesses — hypotheses, if you want to sound professional — about what the customers might want (”I know, let’s make it lemon scented!”). You then use quantitative studies to test those guesses and gather precise information on how the market will react (”women under 40 prefer citrus cents, but those over 40 prefer floral scents”). The process culminates when you have enough information that you can tell the product development team exactly what to create. They build prototype products (”Blammo, the zesty new laundry soap for a new generation”), you test them, and when you’re ready you launch the product.

That process falls apart rapidly when defining a market for a dramatically different product. The first problem is that in order to recruit people for the focus group, you have to know what your market is. If the market’s not yet defined, you are almost 100% certain to recruit the wrong people for the groups. The focus groups are the foundation of the whole research process, so if you start with the wrong people, it will invalidate everything else you do. As we say in the computer industry: Garbage in, garbage out.

There’s also a practical reason for not starting with focus groups. I’ve also found that it’s almost impossible to make an engineering team wait for you to complete the three-step research plan. Because of long product development lead times, they need to get started on their work very early. So they’ll come to the focus groups and start development based on whatever they happen to hear there. In most cases, they’ll lock onto whichever customer comments match their preconceptions, and ignore the rest. By the time you get the quantitative research done, they’ll be halfway finished building the product.

When you’re defining a new market I think you need to turn the traditional research process on its head. Your goal isn’t to gradually build up a fine understanding of the market, it’s to get a general idea of the opportunities as quickly as possible so the engineers can start work. Then you refine your understanding of the market as they refine their product.

Do the quantitative study first. That means the first step isn’t focus groups, it’s quantitative research to try to get a feel for the structure of the market. One of the best ways I’ve found to do this is with a feature survey. Start with a broad cross-section of the public. Give them a very general description of the type of product you’re working on (for example, a mobile phone with advanced features). Then give them a long list of features, and get them to rate each feature for its relative importance if they were buying that product. Again using the phone example, you could list things like long battery life, low weight, color screen, ability to play MP3 music files, and so on. You don’t have to list every possible feature, but make sure you’re covering all the possible categories — entertainment-related features, communication-related features, etc. Compiling this list is a great place to have the engineers help you brainstorm.

You should also capture demographic information on everyone in the study — income, education, job, age, etc.

When you get the data, your temptation will be to look up which features “scored the best.” Go ahead and look if you want, but afterwards you should ignore that information. What you want to look for is not high scores, but lumps in the cloud. By that I mean features that tend to cluster together for some groups of customers — for example, if someone favors long battery life, do they also favor color screens? There’s a type of research called discrete-choice modeling that’s great for this sort of work. A good research company can help you do this analysis. The clusters you identify are your potential market segments. For each of them, dig into their demographics and other information until you understand what makes those users different from the rest.

The segments will seem depressingly vague when you look at the data. The researchers may tell you that correlations between features are weak, or there may be only a few features that form a particular cluster. That’s fine. Remember, you’re looking for tendencies that you can turn into a segment, not full-blown pre-existing segments. If the market were fully formed, there would already be 12 companies selling to it. Even a major market like SUVs didn’t just materialize in one year, it developed gradually as car makers noticed an interest in more rugged transportation, and consumers responded to the first such cars targeted at that idea.

Next, do a quick and dirty product concept test. Once you’ve identified the clusters, the next step is to have the engineers create some product concepts for them. These do not have to be full prototypes. In fact, few high tech companies have time to test-market working prototype products these days. It’s enough to have a nice sketch, along with a one or two paragraph description of the product, and basic price, size, and weight estimates. This is where you want to get the most creative thinking from your product development people. You have to make sure the concepts are well enough described that people can understand them and picture how they’d be used. Then do a quantitative test of the concepts, contacting the segments you identified in step one (something you can do since you gathered good demographics on them), and seeing how they react to the product descriptions. Do they like the ideas? How interested are they in buying? How much would they pay? This is the time to gather as much information as you can on preferred buying channels, price points, and so on.

You should test all the product concepts on all the segments, even if you think that one product will really only appeal to only one group. So, for example, if you were testing phone ideas, you’d ask the communication-focused users to rate the entertainment-featured phone, even though you think they won’t like it. If you’re correct about the segments, the right people will want the right products. If you’re wrong about the segments, your results will be all over the map and you’ll need to rethink the market.

Once again, the correlations here may be frighteningly mild. For example, 60% of your target market might want the product designed for them, while 40% want something they weren’t supposed to want. That’s fine; people are always more complex in reality than they are in a segmentation model. As long as you have a little angle to work with, good marketing and product design can solidify it into a full-blown segment.

Focus groups come last. The concept test should give you the information you need to complete product development. The final step is to do focus groups with the target customers, when the product is nearly finished. You don’t do the groups to gather information; you do them to gather ammunition. Record video of the groups, and edit the video down to about ten minutes of the customers describing themselves and reacting enthusiastically to the product. If you’re in an established company, you can use the tape in-house to help explain your product and get people to support its launch. If you’re in a startup, you use the tape to help you raise money for the launch. And in either case, the tape helps you educate the press and analysts about your product.

Researchers are sometimes uncomfortable with using focus groups this way. They feel that research should always be a search for the objective truth, and that selectively editing the focus group findings is a sort of crime against nature. Don’t let them throw a guilt trip on you, baby. Focus groups aren’t statistically valid anyway. Besides, you’re not pursuing abstract truth, you are building a new market out of (almost) thin air. You need ammunition to bring that market to life, and the focus groups are your source of ammunition. Use them unashamedly.

.

To summarize, traditional research on a market works like this:

1. Focus groups to brainstorm. Do focus groups to get a feel for the customers, get some ideas about what they want, and create some hypotheses of user segments to test.

2. Quantitative research to test. Do a quantitative study to validate the hypotheses you formed in step one, and size the segments.

3. Product concepts to refine. Conduct product concept tests to validate the designs produced by your engineers.

.

The new market definition process works like this:

1. Quantitative research to find lumps. Do rapid quantitative research testing many feature possibilities. Analyze results to identify customer clusters (potential market segments).

2. Quick product idea test to latent segments. Conduct quantitative product concept tests, with very sketchy product descriptions, to validate the segments and give the engineers enough information to go to work.

3. Focus groups for marketing ammunition. At the end of the process, conduct focus groups with people reacting to nearly finished products, to collect video verification of the segments and help you prepare for launch.

.

Next time I’ll talk about a totally different type of market research — advertising proof studies.

“Markets that don’t exist can’t be analyzed….The only thing we may know for sure when we read experts’ forecasts about how large emerging markets will become is that they are wrong.” –Clayton Christensen, The Innovator’s Dilemma

One of the most difficult tasks in market research is guiding product development. The tech industry’s bipolar view of the future dominates its handling of new product research. The visionary companies tend to reject all product-related market research. They rely on their own ideas and instincts.

On the other hand, reactive companies try to make all their product decisions through research. When you do this it’s very easy to use the wrong sort of research. As I’ve mentioned before, if you use a focus group to make product decisions, you might as well flip a coin, because there’s no way to know if the group represents customers as a whole.

It’s better to use quantitative research – at least then you’ll know you have a representative sample of customers. But there are still two major drawbacks to this sort of research, which I call the possibility gap and the blender.

The possibility gap. The visionaries are right on this point, customers usually don’t know what they want until they see it. If you ask an existing user for product ideas, they’ll take what’s wrong with the current product and dress that up as ideas for the future. For example, for years I looked at research on PC users, and they always asked for computers that are cheaper, have more memory, and run faster. Why? Because those are the barriers the users run up against most often.

In 1995, almost no customers in PC research studies were asking for high-speed network connections and photo-realistic 3D graphics, yet those turned out to be probably the most important new PC features in the following decade. To catch those opportunities, you would have needed a much deeper understanding of user psychology and of technology trends.

Hold that thought.

The blender. In most product feature studies, people are given a long list of features to evaluate, and the features that get the highest average score are the ones selected for the product. The problem with this is that it turns the customers into a single average, as if they had been dropped into a blender. The only features that score highly will be the lowest common denominator ones that affect everyone — things like weight, size, and ease of use. If you have a feature that’s beloved by some customers but hated by others, the two groups will cancel one-another out.

There’s a good example of this in the mobile phone world. If you survey mobile phone users about feature desires, the issues that rise to the top are smaller size, lower cost, and longer battery life. Those are the things that irritate almost all phone users. More advanced features, like built-in e-mail, end up close to the bottom of the list.

Despite this, two of the hottest advanced phones in the US today are Research in Motion’s BlackBerry and Palm’s Treo, both of which combine phones and e-mail features. They’re not attractive at all to most mobile phone users, but are beloved by the10% of mobile phone users who are so obsessive about communication that they want their e-mail with them all the time.

Very often, at least in technology products, the biggest opportunities are products that some people love but others hate. So what you want to look for in feature research isn’t the blended average, it’s the lumps that are in the mix before you turn the blender on. What feature requests cluster together? Do the people asking for those features have personalities or demographics in common? What problem do they share that drives them toward wanting those features?

.
The right way to guide products with research

I think the way to get past the blender and the possibility gap is not to try to design the actual products through research. Instead, focus on understanding the needs and psychology of the customers, so you can anticipate the way they’d react to new features. How do they live their lives? What do they care about? What are they trying to accomplish? What challenges do they face that you might be able to help with? Once your product engineers absorb these issues, they’ll start to more or less automatically design the right products.

Let’s take mobile phones again as an example. When you look in depth at the motivations of mobile phone buyers in the US and Europe, it pretty quickly becomes clear that a majority of them — about two thirds, actually — care only about basic voice and maybe text messaging. In the US, they buy the cheapest service plan they can, and take the free phone that comes along with it. In Europe, they’re usually on a very low-cost pay as you go plan (where they add money to the phone account as they go, rather than paying a flat monthly fee), and they often turn the phone off in order to hold down their bills.

These people have no interest in any advanced features or services. If you’re doing a study on advanced phones and you keep them in the research mix, their sheer numbers will make you conclude that there’s little hope for any sort of advanced phone. And, in fact, that just what some mobile companies have concluded.

But if you exclude those basic users from your study, you find that about one-third of mobile phone buyers actually are interested in advanced features of various sorts. One-third may sound like a small number, but keep in mind that about 700 million mobile phones were sold worldwide in 2005. A third of that is about 230 million phones a year, enough to attract almost any company’s attention.

The problem with these advanced users is that they don’t all want the same thing. If you apply the blender principle and mix them together as a group, you’ll find that on average they are moderately interested in almost every feature imaginable. This has led a lot of companies to create “smart phones” that are basically kitchen-sink bundles of features lumped together. These products usually don’t sell very well, because in the process of trying to be everything to everyone they become too big, too expensive, and too complex for anyone to love.

The products remind me of politicians trying to assemble the largest possible coalition of voters by not offending anyone. That sometimes works in politics because the voters have only a handful of parties to choose from, and a politician has to assemble a majority vote. But a product has an unlimited number of competitors, and 10% share might be a huge win. Better to please some people intensely and piss off everyone else than to get a lukewarm reaction from everyone.

Instead of trying to attack the engaged users all at once, you need to look for segments within them. Are there groups of people who want certain features in particular?

When you do that with advanced phone buyers, three groups emerge. One group gives high ratings to all communication-related features — e-mail, instant messaging, built-in fax, etc. Basically, they’re communication junkies, and they’ll pay extra for a communication-enhanced phone. These are the people buying RIM Blackberries and Palm Treos today.

The second group gives high ratings to information-related features — large memory, document display, databases, etc. These are people in information-intense jobs who need a mobile memory supplement. Think of a doctor looking up drug dosage information on the go, or a lawyer trying to find a case reference in court.

The third group responds best to entertainment-related features: music, video, games, and other ways to have fun. These entertainment-focused users tend to be younger than the others, and don’t want to give up their electronic lifestyle even as they enter the job market.

Segmenting the market isn’t a new idea; the auto industry has been doing it for more than 70 years (think sports utility vehicles and sports cars). But although the idea of segmentation is straight from Marketing 101, and is heavily used in established industries, it’s very hard to do in a new industry or product category. Market segments are only obvious after they have been proved by a successful product. Until someone builds that first e-mail phone or SUV, the natural human tendency is to either dismiss the existence of the market, or to lump the customers together and try to hit a home run with all of them at once.

You need to resist that temptation. Products designed to please all segments almost always fail, and if you wait until someone else validates the market, you’ll be fighting with 20 other companies to dislodge a competitor rather than running ahead of the pack.

Next time I’ll talk about how to segment the market for a new product.

This weblog is an experiment in developing a book online. I let it rest for a little while because I wanted to think about the feedback I was getting. A number of people seemed confused by some of the chapters — they felt the chapters were incomplete, or they weren’t sure what the point was.

I realized the problem was my fault. To adapt the book content to a weblog, I was taking the draft chapters and splitting them into digestible chunks, posting one chunk a week. Unfortunately, web posts are typically much shorter than book chapters, and have a different structure. In a book, a chapter is usually a fairly long essay. It constructs an argument in segments, like a gourmet five-course meal that builds from appetizers to soup to salad and so on.

A good weblog post is more like eating bon-bons. Instead of building a structured argument, it makes a single point concisely and then gets out of the way. By chopping the chapters into bits, I was giving you the worst of both worlds – you didn’t get the full argument you’d expect from a book chapter, but you also didn’t get the single clear point of a good blog post. Too many of the online chapters felt like fragments – because that’s what they were.

So I’m going to structure the writing a little differently in the future. Instead of trying to replicate book chapters, I’ll just write about the ideas that I want to cover in the book, roughly one idea per week. I think this will make the weblog a little easier to read, and I hope it’ll also encourage more discussion.

As always, I’m very interested in your comments and suggestions. Please don’t be shy!

Working in the high tech industry, I have become very jaded about claims that the Internet is going to revolutionize something. Grocery shopping, book-reading, even going to the bathroom were all supposed to be transformed by the Web.¹ So I hesitate to say this, but I really believe it: The Internet is producing a revolution in quantitative market research.

Web-based companies are driving incredible reductions in the cost and time needed to collect quantitative information. It’s now becoming possible for even a small company to create the sort of studies that previously were available only to the largest companies and political organizations.

To explain how dramatic the change is, I first have to describe the steps you take to conduct a traditional quantitative research study. First, you work with a researcher to create and review the questionnaire. This alone can take weeks. The questionnaire has to be specially coded and formatted so that phone attendants can understand it. Then the researcher identifies a list of people to call, the call screeners have to be briefed, the phone calls have to be made (and often re-made if no one’s home), the results have to be tabulated and analyzed, and then the researchers create a presentation of the results. The process generally takes 2-3 months and costs at least $30,000 in the US — sometimes a lot more depending on what you want to learn.

Several companies on the Web have recently automated this process. Examples include SurveyMonkey and Zoomerang, and I know there are a lot of others. In all of them, the process generally works the same way: you create your own survey online, send e-mails to recruit the respondents, they fill out the survey on the Web, and then you download your results. I did my first survey this way about a year ago, and the process took less than a week:

On Monday afternoon I wrote the survey. At 8 pm Monday we sent the e-mails inviting respondents. At 11 pm that night, I checked the results from home just before I went to bed. We already had 800 responses, and I could check the tabulated results for each question. There were several interesting trends that I decided to watch closely. I let the survey run until Thursday, and we got a total of 5,400 responses, 60% of them from people outside the US. On Friday morning I showed the results to our CEO, and he picked out four questions that he wanted made into charts for a speech the next week. I downloaded the results into Excel, and sent him the charts before I went home that night.

Total time elapsed: Five business days. Total cost to the company: About $800.

Oh and by the way, that $800 included a year’s subscription to the survey service, which will let me do follow-up studies for free.

Total reduction in time: About 92%. Total reduction in cost: At least 95%.

Now, this is not an invitation to go cut your company’s market research budget by 95%. There are some significant limitations to online research. Here’s are the most important, plus some ideas on how to work around them.

Drawback #1: It’s limited to web-users. Conventional market research uses phone calls or the good old postal service to contact people. Although this can be slow and expensive, it reaches almost everyone. Online market research reaches only people who use the web. Although in 2006 that was about 70%-77% of adults in the US, and a rising percentage elsewhere, it’s not everyone.²
Web usage is generally lower outside the US. Below are some Web usage rates for some prominent countries:

Percent of population who have access to the Internet³

The penetration limit may not matter if you’re selling a high tech product in the developed countries — almost all of your target customers will probably be online anyway. But it’s a big deal to a company selling, for example, discount eyeglasses for elderly people (only about 40% of people over age 65 in the US are online).

Even if you are selling a technology product, you shouldn’t make the mistake of projecting to the whole adult population from online results. For example, if 10% of people in an online study say they like your product, you can’t conclude that 10% of adults in the country like your stuff, only that 10% of the web users like it.

There are a couple of workarounds for this.

If you’re trying to decide whether there’s a big enough market for your product, try the math assuming that the only people you’ll sell to are those who are online. For example, if in an online study you find that 10% of the people surveyed want a product, your estimated available market is 10% of the online population (in the US, that would be 10% of about 210 million online users, or 21 million people). You know the real market will be bigger than that, but at least you have a floor. If that’s enough people to make your product profitable, you have the info you need to launch it.

(Even in some low-penetration countries such as China and India, the actual number of Internet users is large enough that it adds up to a substantial potential market. If you believe the statistics, China is second only to the US in number of Internet users, and India’s Internet population is slightly larger than the UK’s. I’d read these numbers with a bit of caution, though — in some countries Internet access is through Internet cafes rather than a PC that’s in the home or office. That means usage patterns, and your ability to market to these people online may be very different.)

Another tactic you can try is supplementing your online research with other studies. For example, you could conduct one conventional market research study side by side with an online one. By comparing the results, you’d get an idea of how you need to adjust the online results to account for the full population. Then you could probably get away with just online studies for a year or 18 months before you’d need another conventional study to recalibrate.

You should also ask about online access whenever you collect information from your customers. For example, rather than just asking age and education on a product registration card, also ask your customers if they use the web. That will tell you what percent of your users you’re reaching with online studies.

Drawback #2: You need a good list. To conduct an effective online survey, you have to send e-mail invitations to a random list of web users. If there’s a bias in the list of people you contact, your results will be biased. In my experience, this is the biggest source of errors in online studies. For example, I’ve seen online surveys made of people who register to post messages on a particular website. People who post actively online are far different from the average web user, and the results you get from them will not reflect “normal” people.

I’ve also seen industry analyst companies trumpet the results of surveys of their own subscribers, as if those people represented average customers. Remarkably, those surveys seem to always reflect back whatever messages the analyst firm has been preaching. This happens both because people repeat the messages they’ve been told, and because people tend to subscribe to industry analysis services that they agree with in the first place.

But the worst way to get a list of customers is to post a message to an online forum inviting people to come respond to your survey. When respondents can select themselves, you inevitably get flooded with fanatics and cranks. That isn’t a survey, it’s a popularity contest.

Finding a good list is hard. There are companies that specialize in collecting lists of people willing to fill out online surveys. You can rent access to these lists (although it’s often expensive). I use this technique a lot in my own work, and the results seem quite solid, but I do sometimes worry that people who go out of their way to volunteer for surveys may not be a great proxy for the public as a whole.

If you have friends in research at other companies, you may be able to share access to customer lists with companies that don’t compete with you. For example, if you were looking to evaluate the market for a new video camera, a list of television owners might be a good starting point. Unfortunately, there are legal restrictions on how companies can share e-mail lists; make sure you don’t violate the law.

Over time, it’s best to compile your own customer list. That won’t help you size the market for a completely new product, but it will let you track your current ones, and evaluate the market for derivative products. Chances are your company’s marketing department is already compiling a customer list. They probably regard it as a marketing tool, something they can milk like a cow to get additional revenue. But the list is also a valuable source of information, and can save you a lot of money on research. Reward your customers for sharing their e-mail addresses with you, and make sure the marketing department doesn’t drive those people away by spamming them with too many “special” offers.

Drawback #3: You need to know how to design and analyze surveys. This is trickier than it looks. It’s easy to accidentally bias a survey by asking a question in the wrong way. For example, if you ask people to rate something on a numerical scale (i.e., “one to six, with six being best”), it’s a good idea to give an even number of choices. If you give an odd number, a lot of people will cop out by choosing the middle, neutral, option.

If you haven’t had some training, it’s best to get help from someone who knows how to construct a survey. (This is the point where I should probably do a shameless promotion for the consulting company I work for.)

.

Fun with online research

Now that I’ve listed the challenges with online research, let’s talk about the opportunities. The first opportunity is frequency. You’ve seen the “tracking polls” that professional politicians use in election campaigns. You can now do your own tracking studies. If you have a good list of e-mail addresses, you can easily survey a subset of them every week, watching for changes in attitudes and tracking the effect of things like ad and PR campaigns. Do this right, and you should never be caught by surprise by a market trend again.

It’s very helpful if you can collect demographic information about the people on your customer list. Once you have this, you can use it to aim targeted surveys at particular segments — for example, people with a certain income level, or in a particular age group. This will let you learn much more detailed and subtle information on the market than you could have collected in traditional studies. Be sure you understand the limits on information collection in the regions where you operate, though — some countries are adopting restrictive rules on the use of customer information, and the rules change frequently.

The other opportunity, I think, is driving the use of research more deeply into your company. In a situation where market research is virtually free and almost instant, your use of it can change dramatically. Trying to make a tough decision on whether or not to build in a feature? You can have data in a week. Thinking about a price change? You can get customer feedback almost instantly. I’m not saying you should let research make your business decisions for you. As I’ve said elsewhere in this book, I think the best decisions come from a mix of data and gut instinct. But now there’s no excuse for basing a decision only on guesswork. The data’s almost free. Why would you possibly want to operate without it?

Here’s your chance to check out SurveyMonkey. Please click here to rate this post. The link will open a one-screen survey hosted by SurveyMonkey, and you’ll get to see the results after you take it.

Next week: Using market research to plan product strategy.

1. In 2004, there was allegedly a project at Microsoft to create an Internet-equipped public restroom, so people could browse while otherwise occupied. There’s some disagreement over whether it was a serious project, or just a joke. Everyone agrees that if there was such a project, it was killed soon after it was first disclosed to the public. Meanwhile, those of us in the mobile industry know that a significant number of people use smartphones to browse and send text messages while they’re using the facilities. So maybe this is an area where the Internet is breeding a revolution after all. [↩ back]
2. Harris Interactive measured US penetration at about 77% in May 2006. The other US statistics in this section also come from Harris. You can check out a summary of their study here. Other sources put the US figure at about 69%. The difference may be due to variations in the way different studies define Internet penetration. [↩ back]
3. These are generally 2005 figures, as collected by internetworldstats.com. [↩ back]

I’m sorry it has been a couple of weeks since I posted. I was involved in a very time-consuming protest against a developer’s plan for my neighborhood, and had to cut back on other activities.

This week we continue our look at market research, with thoughts on what to look for in a market researcher, and how to present findings.

.
What to look for in a market researcher

Obviously, the most important aspect of a good researcher is professional competence. You need someone who’s well trained, and has experience in a wide variety of different methodologies. The popular stereotype is that numerical analysts aren’t good at dealing with people, but the best researchers often have an interesting mix of people skills and quantitative analysis skills. You can still be a successful researcher even if you’re an introvert. But I’ve never seen a successful researcher who was bad with numbers.

The other characteristic to look for is the ability and willingness to think beyond the numbers. This is a hard thing to find in market researchers; many of them are very methodical, and reluctant to draw any implications from the facts they’ve discovered. For example, they might report that a particular customer segment has a high percentage of elderly people, but they’ll be very uncomfortable speculating on why there are so many old people in the group.

This is a significant problem because there’s never enough money and time to research everything you want to know. At some point you have to stop gathering data and fill in the gaps with your best guesses and extrapolations. This is profoundly uncomfortable to many researchers because it runs contrary to their training. The whole point of market research is not to guess. And many market researchers aren’t very good at it, either.

To find researchers who are good at drawing implications, talk with them about their previous studies. Ask what they learned, what conclusions they drew, and what actions they recommended. The more insightful and non-obvious their conclusions, the better.

The other thing to watch out for is people who are comfortable forming implications from their research, but form bad ones. Sometimes this will just be because they’re not very insightful. It’s best to avoid hiring these people. But sometimes it happens just because they’re naïve about your industry. It’s going to be almost impossible to find researchers who are both very skilled in their craft and deeply informed about your industry. To get around this problem, have the researcher work with a competitive analyst when thinking about implications. The competitive analyst may be able to give some of the industry background that the researcher lacks.

.
How to communicate market research findings

There’s no substitute for a good presentation when delivering market research. Graphs of research results and video of customers both work very well in a presentation format, and people always have a lot of questions that can best be dealt with in a live setting. A future chapter will give general guidelines on how to present. What I want to focus on here is something you shouldn’t do.

Don’t let an outside research supplier present the findings. As part of their standard service, the company that conducted the research for you will prepare a slide presentation of the findings. They’re proud of this work, and they’ll want to come in and present the slides to your whole management team.

That’s usually a bad idea.

First, remember that your goal is not to present raw data to the company, it’s to make sure the company takes appropriate action on your findings. That means the implications of your study are more important than the actual data, and they need to be tailored to the internal vocabulary and politics of your company. Most outside suppliers can’t understand this; they simply don’t have the context. Most of them will just present raw data — or worse, any implications they draw may not be appropriate to your company, or may be phrased in ways that people in your company will misunderstand.

For example, I’ve had outside researchers recommend my company adopt strategies that were already tried, and failed, years before. Or they have given advice that undercut exactly what we were trying to get the company to do. Once this has happened, it’s almost impossible for you to correct their messages, since you’re the person who chose the research supplier in the first place. At best you’ll look incompetent.

Second, almost every research company I’ve ever dealt with creates terrible presentations. And by using the word terrible, I’m probably understating the problem. Most of the supplier presentations I’ve seen are either ugly or incomprehensible, with a little bit of repetitiveness thrown in for good measure. I don’t know why this is. Maybe the presentations are an outlet for the repressed artistic yearnings of researchers (who, after all, spend most of their time doing very dry numerical work). Or more likely, the researchers are so in love with their data that they try to cram it all into one slide. Whatever the reason, most of the supplier presentations I’ve seen are so complex and poorly structured that you can’t understand them unless you already know everything about the study.

Here are two real-world slides from supplier presentations. Company confidential information has been obscured, and the supplier names have been removed. Both of these slides came from wonderful studies on which the suppliers did great work — they just didn’t do a good job of presenting it.

Here’s a good example of a researcher who couldn’t bear to leave out any data. Even though I remember this study vividly, I had to spend several minutes studying this particular slide before I could figure out what it was trying to say. The actual research finding was, “companies are planning to buy more of our product next year than they did this year,” but good luck getting that from the slide even if you could read it (which no one could do unless they were sitting in the front row of the room).

Some research firms just aren’t very good at communicating visually. This slide is from the final report of one of the most innovative, influential research studies I’ve ever been involved in. It was prepared by one of the best research companies in the country, but you’d never know it from the chart, which I’m still not sure I understand. The supplier’s report had another 111 slides just like this. If I had allowed this supplier to present the findings to my company, no one would have understood the research, let alone acted on it.

Unless the research supplier is unusually good at presenting, and well attuned to your business, you and your company will be much, much, much better off if you rework the supplier’s presentation into words and graphics tailored to communicate clearly to your corporation.

Written documents. These have a role to play, but I think it’s more for reference than as the primary deliverable. If you’ve done an especially large or data-heavy study — for example, a major survey of your company’s installed base — it’s a good idea to give people a reference document on the findings. For an installed base study, the document could include tables on all the key demographics of your users, things like age, geographic distribution, education level, income, how long they’ve owned, satisfaction level, and on and on. This will let people in the company look up any specific information they need, rather than coming to you.

This is also a great document to post on your company’s intranet.

E-mail is a good supplement to presenting your research. Create a message summarizing the most important implications and most surprising findings of the study and send it to key stakeholders (if you are paired with a competitive analysis team, their competitive info mailing list is a great place to distribute this information).

But I don’t recommend that you try to communicate all of a study’s findings in an e-mail. There’s just too much to explain. Use the message more as an advertisement for your presentation sessions, and as a supplement to get your most important messages to people who don’t have time to come to the presentations.

__________

Please click here to rate this section (the link will open a one-screen anonymous survey, and you’ll get to see the results after you take it).

Next week: The online revolution in market research.

This week we continue our look at market research, with a discussion of how to work with market researchers. The typical market researcher has a very specialized skill set that’s not fully understood, or necessarily valued, by the company as a whole. If there’s an MR team in your organization, you need to spend some time learning how they work and what makes a good research study.

In the tech industry there’s an informal rule that if you want to get along with hardware engineers, you have to learn how to appreciate their block diagrams. A block diagram is a drawing that shows how the various components of a circuit or computing device work together. If you can understand the basics of an engineer’s block diagram, his or her respect for you will go way up, and you might even be treated like a sentient being.

This is a block diagram of the Data Translation DT9840, a “low-cost real-time data acquisition USB module with an embedded DSP for high-accuracy noise and vibration testing.” Check out the dual 24-bit analog inputs. Sweet!

The equivalent of block diagrams for a market researcher is something called a crosstab. Crosstabs are documents the size of a regional phone book, listing every question asked in a quantitative survey and every response, cut by a myriad of different statistical groupings — age, income, and so on. Reading crosstabs can feel about like, well, reading a phone book. But there’s a hidden beauty to them. As you look through the questions and answers, you’ll start to pick up subtle patterns and get a feel for how the customers actually think. Here’s a simplified example of something you might see in a crosstab:

This is a little excerpt from a study that looked at Internet usage in the US. In this question, people were asked if they had browsed the web in the last three months. The vertical columns across the top divide the results by the age of the respondents and their sex. The horizontal row labeled “Total” shows the total number of people surveyed in each category. For example, the survey talked to 433 people aged 65 and older. The row labeled “Have browsed web” shows the number of people who answered yes to the question, “have you browsed the Web in the last three months?” So, 75 out of 433 people aged 65 or older said yes, or 17% of the sample.

To me, there are two important findings in this sample, one of them a surprise and one not. The thing that didn’t surprise me is that elderly people are less likely to use the Web. The surprising finding was that the rate of Web usage was very flat for people under age 54. For most technology products, young people are more enthusiastic adopters.

You can’t get this sort of intimate familiarity with the data in a study just by reading a summary; you have to look at the crosstabs. Unfortunately, it’s a lot of work to read them. A typical crosstab document for a major research study could easily have more than 400 pages, each of them looking about like this:

This is a typical crosstab page. Don’t worry that it’s unreadable; the original Word document is in about five point type.

Chances are you won’t be reading crosstabs every day. But you should do enough of it that you get comfortable with the formatting and can pick out important information. If nothing else, it’ll help you ask the researchers much more informed questions.

A caution about crosstabs. Because they’re the mother lode of information about a study, crosstabs can be dangerous in the wrong hands. If someone misreads a crosstab, they might completely misinterpret a research study. Because of this, some researchers don’t like to show anyone their crosstabs, and I think you should be very reluctant to circulate them freely in your company. If a researcher is kind enough to share their crosstabs with you, keep in mind that it’s an act of trust. Be sure to check with them if you form any conclusions about the data, and don’t give the crosstabs to anyone else without telling them.

Become a methodology groupie. This is the other key to getting along with market researchers. Methodology means the way the study was conducted — how the people in the survey were chosen, how the questions were asked, and how the results were tabulated. I gave you a start on understanding methodology in the first part of this chapter, but if you’ll be working with researchers regularly you should do a little more study on your own. If you’re not a born researcher, methodology is about as interesting as double-entry bookkeeping, but it’s hideously important. If it’s done wrong, it can completely skew the results of a study, so researchers spend a huge amount of time agonizing about it. If you want to understand their world, you should know enough about methodology so you can at least tell the difference between a reasonably well structured study and one that belongs in a circus.

.
How to organize a market research team

Reporting structure. A market research team can vary in size tremendously, depending on the size of the company it serves. In a very small company, you can get away with having no full-time researchers at all. In this case you contract out your research to an external expert who manages the projects for you and delivers the findings. I don’t like this model because researchers pick up a lot of information and insight along the way that never makes it into a formal report at the end. If the researcher lives outside your company, that insight will be lost.

In a multi-division company with several business units, you’ll need several researchers. The first question is whether to have those people report to a central team, or to distribute them into the business units. If there’s any business synergy at all between the BUs, I think it’s best to have the team located centrally. This has several advantages:

–First, it’s more efficient. Depending on how much work there is, a single researcher can often handle the needs of two or more business units. If you parcel out the researchers to each BU, you’ll have to hire more people.

–Second, if the team is centralized, there’s a growth path for the researchers. Market research is a specialized skill, and it’s very hard for a researcher to “graduate” from that role to something else. Most of them don’t want to do anything else anyway. But they would like to have the opportunity for promotion, something they can get in a central team. It’s also very helpful to have researchers supervised by someone who’s a professional researcher themselves. There’s a huge amount of expertise needed in market research, and it’s very hard for a non-expert to evaluate the quality of a researcher’s work and give them meaningful feedback on their projects. Having a non-researcher lead a market research team is like having a non-doctor lead a medical research team.

–Third, if the researchers work together, comparing notes and talking about their work, they’ll be able to spot trends and information that crosses multiple studies. Often these serendipitous discoveries are the most useful.

The drawback of a central market research team is that the business units tend to view it as distant and not focused on their needs. This is a genuine risk. One way to make the central team more acceptable is to have the researchers report “dotted line” into the business units. The researchers sit in on the BU staff meetings, so they feel like a part of the team and are responsive to its needs. But their formal reporting structure still runs back through the central MR team.

Allow only one source of customer truth in the company. As I mentioned above, strategic market research that focuses on understanding how customers think can be the most valuable output of a market research team. But you should not focus all of your group’s efforts on that sort of research. In fact, it’s very important to make sure that your group is also the exclusive source of tactical market research services for the company. If someone needs a study on sales of a particular product, or customer attitudes in a particular company, you should never turn away that request.

If you don’t have enough people in your team to do all the research the company wants, you should pre-qualify a couple of outside vendors who can take on the extra work. Make sure they understand the projects you’re doing, and the main themes that you’re trying to educate the company about. You should also supervise loosely the work they do for your company. In particular, you should take a look at their conclusions before they deliver a research report to the company. Be ruthless about enforcing this rule.

This is a contrast to what I said a competitive analysis team should do. For competitive analysis, one of the biggest challenges is not getting consumed by trivial support requests from the company. For a market research team, one of the biggest challenges is making sure there’s only one unified source of customer “truth” for the company. In my experience, if you let parts of the company start doing their own market research without supervision, you’ll quickly end up with competing versions of the “truth” floating around the firm. If you leave a business unit to its own devices, inevitably it will contract with a low-cost researcher who produces poor findings, or who tells them what they want to hear. This mangled research will conflict with some of the things you’ve found about the market, so you’ll end up arguing against the BU’s research. This can get very ugly. The average employee at your company doesn’t have the knowledge to tell the difference between a good study and a bad one, so your argument can quickly degenerate into a mud-slinging match about who has the biggest methodology. Even if you win the argument, you’ll make enemies.

Far better to prevent the problem from happening in the first place by making sure all research comes through you and is professionally conducted.

Work style. Many market researchers are extroverts. That’s not surprising, since their profession focuses on understanding people. But much of the actual work of market research — designing a study and analyzing the results — is pretty solitary, involving a researcher wrestling one-on-one with a computer and sometimes hundreds of pages of tabulated numbers.

Unlike competitive analysts, a market research team shouldn’t be pushed into collaborative work all the time. Researchers need a balance between opportunities to work alone and interaction with their team. The interaction is mostly at the start and end of a study. At the start, a study design and questionnaire should be reviewed by others in the group. If you’re doing focus groups, it’s good to have several people from the research team attend some of the groups, just to get a feel for what customers are saying. And at the end of a study, the researcher’s conclusions and presentation of findings should always be previewed, and defended, in front of the entire group.

It’s also pretty common in any large study to have a few strange results that seem out of place or are hard to explain. For example, I’ve seen cases where people gave very different answers when small changes were made in the wording of a question. If you have any cases like that, it’s very important to discuss them with the group, and figure out what they mean, before the findings go to anyone outside. Some people in your company will be basically skeptical about market research, and will jump on any error or ambiguity as an excuse to dismiss the entire study. Protect your team’s credibility by reviewing studies carefully before they’re delivered.

Don’t call them analysts. As an example of how specialized the market research world can be, it’s important to be careful with the job titles you give to market researchers. I once made the mistake of referring to them as “analysts.” I thought that would be a compliment, because analysis is a more active and valuable activity than just running a research study. It turned out to be an insult. In the market research world, an analyst is a junior trainee who massages data after a more senior researcher has conducted a study. Calling a researcher an analyst is like calling a corporate vice-president an administrator.

This is another example of why it’s good to have a professional researcher manage an MR team.

__________

Please click here to rate this section (the link will open a one-screen anonymous survey, and you’ll get to see the results after you take it).

Next week: Picking researchers and presenting findings.

The three perspectives a company needs in order to map the future are competitive analysis, market research, and advanced technology analysis. This week we continue our look at market research with a discussion of quantitative research – surveys and other studies that give you statistically reliable numbers. Or that would, if they were conducted properly…

.
Uses of quantitative research

Because quantitative research gives you accurate numbers, it can be used to keep score for your business. How many people are aware of your products? What do they like and dislike about them? Do they like your products better than the competition’s? Do they plan to buy from you in the next six months?

All of these things are relatively easy to measure, but make sure the survey is very well constructed. These statistics are basically report cards on the work being done by the company’s marketing and product teams. If you find bad news, you need to be sure it’s completely accurate. Besides screwing up the company’s decisions, producing flawed research can easily get you fired.

One of the most important statistics your company will want to track is purchase intent. If you can gaze into the future and estimate how many people will buy your products in the next quarter or year, those figures can be driven straight into business plans and sales goals. But purchase intent is also one of the toughest numbers to interpret. There’s a long path from someone thinking about buying a product to actually purchasing it, and any interruption along that process can throw off your findings. I’ve seen studies that showed rising purchase intent even though actual sales were dropping. It’s best to use this sort of research to check for potential warning signs of trouble, but don’t let good results lull you into a false sense of security, and be very careful about building these figures into the business plan.

It’s also commonplace to use surveys to test things like reactions to new products and new pricing. Like purchase intent, this research can be very tricky research to interpret, because conditions almost always change from the time you conduct a study until the time you take action on it. For example, in the computer hardware industry we usually set the pricing for fall’s products in the spring. The research has to start even sooner, so you’ll have time to collect the results and study them. Pretty soon you’re surveying people in February for a decision that won’t be implemented until October. People might tell you they love a price in Spring, but by the time the product ships in fall, there are three new competitors at lower prices, one of the competitors has launched an aggressive new promotional campaign, and economic conditions have changed.

You can of course try to anticipate all these things in your research study, but pretty quickly you have to make so many future assumptions that you’re conducting an academic exercise rather than testing something in the real world.

When I was at Apple, I spent some time as the head of marketing for the home and education business unit. I tried using a quantitative survey to forecast that fall’s sales and set pricing, and the exercise turned out to be a waste of time – the results were out of date by the time we could act on them. Today, Internet-based surveying might let a company move faster.

The other challenge to keep in mind in price research is that people almost always overstate how much they’re willing to pay. Think about it — in the very process of conducting the survey, you have to describe the product in some detail, focusing the subject’s attention on it much more than would normally happen. This is almost certain to make them more interested than they would be in the real world, where your messages will be lost in a flood of other things being communicated to them.

People also just plain tend to get cheaper as they go further into the buying process. They might say $299 was an ideal price when surveyed, but when they go to actually buy the product that money feels a lot more important to them. Maybe there’s some other item across the store that they might like to buy instead, or maybe they want to go out to dinner and a show this weekend.

This doesn’t mean it’s pointless to do any research on pricing, but I think it’s better to try to research price bands — what range of prices are people willing to pay for certain classes of product — rather than trying to set the exact price of a single product. And if the research does indicate that a certain price is optimal, treat that as the upper limit on your pricing rather than the midpoint.

.
Things to look for in quantitative research

It’s very easy to screw up a quantitative research study. Even small errors in methodology can make the results meaningless, so it’s best to work with someone who knows research. There are more potential pitfalls than I can list here, but a couple of prominent things to watch out for include:

–Make sure you’re surveying enough people so you can be reasonably sure that the results represent the population as a whole. In research terms, you want a large enough sample so that your findings will be statistically significant. Preferably, the margin of error in the study should be plus or minus five percentage points at the 95% confidence level. That means that if you see a five percentage point difference in a question (52% say yes, 47% say no), there’s a 95% chance that the majority of people actually would say yes if you surveyed everyone in the country.

For consumer tech products in the US, that usually means you need to survey a couple of thousand people minimum. For a corporate product, about 200-300 people may be sufficient, since the world of corporate buyers is a lot smaller than the world of consumers.

–You need to be sure the list of potential respondents (the people you’re surveying) doesn’t have any biases. If you’re surveying a pool of people who are inclined toward a particular answer, it’s sure to skew your results. You see this all the time when, for example, magazines survey their own readers and then report the results as if they represent the country as a whole.

–Work with the researcher closely on crafting the implications of the study. A great market researcher has to be meticulous about methodology, but that same focus on detail can make them reluctant to draw conclusions that reach beyond the basics of the data. This is especially likely to happen when you use outside research consultants, who won’t understand your industry as well as you do. They’ll tend to give you implications that are straight-line projections of their findings, without much context.

For example, if people have less than positive opinions of your product, the researchers might report “you need to improve impressions of this product” or even “the product is a failure.” But you might have other information — perhaps there was a product recall that temporarily hurt opinions of the product; or maybe your company launched the product as a stop-gap, knowing there would be problems. It can be tremendously demoralizing to have an outside researcher come into your company and beat up a product without the right context on what its goals were and what else is happening in the market.

–Beware of buried assumptions. Sometimes a researcher’s unstated assumptions about the market will slip into their selection of what to emphasize and how to phrase it. For example, suppose you did a survey showing that 19% of the population liked your product. A researcher could report that fact with either of the following sentences: “Unfortunately, only 19% of adults want the product” or “Fortunately, nearly one in five adults want the product.” One sentence makes the finding sound bad, the other makes it sound good. Sometimes an outside researcher will make assumptions about what your company’s goals are, and editorial comments like this will slip into their report without them even realizing it.

Before the launch of the original Palm Pilot, Palm commissioned a survey to determine how many people would want the product. The survey showed that two percent of US adults were extremely interested. Many researchers would interpret that as a terrible result; 98% of adults weren’t entranced by the product. But Palm looked at those numbers and decided they were good news — two percent of US adults is about five million handhelds, a very attractive figure for a small hardware company. So the launch went ahead and the rest was history.

–Be very cautious of “off the shelf” customer segmentations. Several market research companies have conducted very large studies on people around the industrialized world. Based on these studies, they have divided the population into segments — usually about a dozen of them — with various demographic and interest profiles. The segments are usually given offbeat, vaguely disturbing titles like “Sultry Seniors” and “TechnoTweens.”

These companies specialize in mapping your products to their demographic segmentations, and using that information to tell you what to do. There’s nothing wrong with this, and sometimes the segments can be useful.

But very often the segmentation of your customers will be specific to the products you make. For example, one of the hottest trends in advanced phones today is building in e-mail capability. The people who want this feature don’t fit into a particular lifestyle category, they’re just people who obsess about communication. The target market for an e-mail phone won’t show up in most standardized segmentations. If you can afford it, you’re much better off doing a segmentation study that’s specific to your industry.

–Price is the biggest drawback of quantitative research. I’ve rarely seen a consumer study in the US that cost less than $30,000, and they can easily go over $100,000 if you want to get specific detail on multiple market segments. Expanding a survey into Europe typically costs about $30,000 per country, and Asia is even more expensive than the US.

You can sometimes find companies that will charge you less, but usually they’re taking some hidden shortcut that will reduce the value of the research. Typically they’re identifying respondents on the cheap, in ways that could bias the findings.

__________

Please click here to rate this section (the link will open a one-screen anonymous survey, and you’ll get to see the results after you take it).

Next week: How to get along with market researchers.

The three perspectives a company needs in order to map the future are competitive analysis, market research, and advanced technology analysis. This week we’ll start our discussion of market research. Considering how widely used market research is, it’s surprising how many people don’t understand how it’s done and how to use it. So we’ll start with the basics…

.
At its core, competitive analysis is about intuition, supplemented by objective information. Market research is the opposite; at its core it’s a science. Intuition plays only a secondary role, seeding the questions that a good market research study tests. Market researchers hate to guess. They take pride in the rigor of their studies, and like a good science experiment, good market research is repeatable — if you ran it over and over again, you’d get basically the same results every time.

Another important difference between market research and competitive analysis is that market research has a well defined, accepted role in most corporations. It’s hard to find a large company that doesn’t do at least some market research, and there are respected professional training programs for researchers.

Unfortunately, most of the companies I deal with use market research only in a limited, tactical way. Market research teams are usually chartered as service groups — they take specific questions from various parts of the company, deliver the answers, and move on to the next project. Usually those questions focus on measuring what people think — “will you vote for this presidential candidate?” or “what do you think of this brand of soap?”

It’s much less common for market research teams to be asked how customers think. What motivates them? What do they care about? How do they make decisions? This sort of strategic research takes a fair amount of time and uses up resources that business units would prefer to spend on immediate problems. But I think it’s incredibly valuable, because if you know how customers think, you can predict how they’ll react to changes in the marketplace. Like good competitive analysis, good market research doesn’t just help you know what’s happening today, it helps you predict the future.

I think the right role for a market research team is one that balances tactical services and strategic vision. This chapter is about how to make that happen.

.
The basics of market research

I think it’s very important for all managers to have a basic grounding in how market research works and how to tell good research from bad. Market research is an incredibly powerful tool. When it’s done right, it can be the foundation of a wildly successful corporate strategy. When done wrong, it can lead an entire company to march boldly over a cliff. You wouldn’t feed your body spoiled food, so don’t feed your mind spoiled ideas.

Unfortunately, unlike spoiled food, spoiled research often smells great and comes wrapped in a very attractive PowerPoint presentation. Here are some tips to help you look beyond the slides.

.
How market research is conducted. There are usually two important players in any market research study, the in-house manager and the supplier. The study’s in-house manager is an employee of your company, usually a professional researcher, who takes the company’s questions and figures out how to get them answered with research. The researcher will determine what kind of study to use, and will contract with an external research supplier to conduct the actual study.

The research supplier is an outside company that does nothing but conduct market research studies. Suppliers have specialized facilities and resources that most companies couldn’t afford to keep in-house. For example, they’ll maintain a large list of people who can be contacted for a research study. Usually suppliers specialize in particular types of studies. Companies that do focus groups (see below) will have offices with the proper rooms, and people trained to run the groups. Companies that do numerical surveys will have banks of phone operators trained to conduct surveys by phone or in person.

The in-house manager works very closely with the supplier at the beginning and end of the process. At the beginning, they work together to design the survey, including the exact wording of any questionnaire, and the rules for how people surveyed (the “respondents”) will be selected (their “qualifications”). At the end, they work together on the interpretation of the findings. The research supplier delivers a final report, which the researcher may pass along to the company as-is, or may replace it with a revised report tailored more to the internal needs of the company.

.
Quantitative and qualitative research. There are two basic types of market research, quantitative and qualitative. Quantitative research gives you hard numbers — it’s a scientifically-conducted survey that gives you statistical information on the market as a whole. Opinion polls are quantiative research.

Qualitative research is any market study that doesn’t give you reliable numbers. The most common qualitative research is a focus group, in which a small number of people spend several hours discussing a topic, while researchers behind a two-way mirror watch them. The number of people in the group is too small to give meaningful information about the market as a whole.

.
Uses of qualitative research. Focus groups and similar studies are often used as fodder for an advertising campaign. You’ll get a group of target customers in a room and study how they talk — what words do they use, what mannerisms, and so on. This helps the creative team develop ads that speak directly to the customer in his or her own terms. Often you’ll find a particular person in a group who really exemplifies the type of customer you’re trying to reach. You can show video of this person to the creative team and say, “give me an ad that appeals to him.” How often have you seen brochures, websites, or even ads written in jargon that’s difficult for anyone outside the company to understand? Often this happens when a company doesn’t have a clear understanding of the person they’re trying to communicate to. Focus groups can help with this.

You can also use focus groups to check for potential disasters with new brand names and logos, before you make a mistake that’ll take a lot of money to correct. A legendary example is the Chevy Nova, with “no va” meaning “doesn’t go” in Spanish. Unfortunately, that one turns out to be so legendary that it’s not true, but I’ve lived through some real cases. For example, a company I worked for once came very, very close to giving itself a name that sounds like “excrement” in Chinese. It was especially ironic because the company had a large number of employees in China.

In the computer industry, we learned years ago that our love of using numbers in a product name could get us in trouble overseas. Apple once produced a computer called the Macintosh Performa 4400. The character for 4 in Chinese resembles the word for Death, and the character for 0 resembles the word for Again. So in Chinese culture 4400 means something like “die die again again.” Needless to say, the product carried a different number when it went on sale in Asia.

But it isn’t just numbers and names that can get you in trouble. Almost any graphical element, or even color, can have unanticipated meanings in various cultures. For example, Macintosh computers once used an on-screen icon of an upraised hand to signify that a function had stopped working (the hand was like the raised hand of a policeman directing traffic). In the mobile phone industry, one of the major software companies once adopted a hand outline as its logo (which was apparently meant to indicate that the software powered devices you hold in your hand).

The problem with all of this is that the hand, held flat out, is a serious insult in some cultures, meaning more or less, “let me feed you some manure.” Getting that message from your PC or mobile phone isn’t very appealing.

This hand icon was displayed when a Macintosh computer reported to the user that a program had crashed (or “unexpectedly quit,” as Apple liked to put it). It replaced a bomb icon that was understood universally across all cultures, but communicated so clearly that it scared the bejeezus out of nontechnical users.

For a time, this was the logo for Symbian Epoc, an operating system for smart phones. If you haven’t heard of it, don’t feel bad — Symbian eventually dropped both the logo and the product name.

In another version of the Macintosh software, an animated image of a hand with fingers counting from one to five was used to indicate that the user should wait. It turns out that the various combinations of fingers meant insulting things in several countries, making this the first cross-culturally insulting icon.¹

You can use focus groups to test this sort of thing, showing a group of local customers the proposed logo or name, and getting their reactions. Personally, though, I don’t think a full focus group is necessary. Just ask the locals who are on your staff, or if you are selling through a distributor, ask some local people who work for them. They can give you a faster reaction at virtually no cost.

Focus groups are also sometimes used to collect product requirements. You get a group of users together and talk with them about what they like about a current product, and what they’d like to see in the future. I think doing this is a big mistake. Because a focus group is not scientifically structured, the reactions you get from it aren’t projectable to the whole market. You might have a group of freaks who’ll mislead you into creating a product that’ll sell to only 1% of your target market.

But because focus groups are a lot cheaper than quantitative studies, it’s very tempting to try to use them as a substitute. For example, I’ve seen product plans with statements like “60% of the people in our focus groups said they liked the product.” While this is a factual statement, it’s also meaningless because the group wasn’t a scientific sample of customers.

The usual excuse for doing this is, “it’s better to use the focus group than not have any research at all.” That’s rubbish. The “data” you got from the focus groups is no better than a coin toss. You’d be much better off letting the smartest people in your company make an educated guess.

.
Things to look for in good qualitative research. The best focus groups are great conversations in which you get to eavesdrop, so you want to look for conditions that’ll produce a useful conversation. That means a good setting, good people, and a good moderator.

The location should be in a city that isn’t dominated by your industry. In your industry’s hometown there’s too much risk that you’ll get some insider know-it-all who’ll take over the conversation. So don’t do movie focus groups in West Hollywood, don’t do car focus groups in Detroit, and don’t do computer focus groups in San Jose.

A good focus group facility has a room with one or more walls of sound-proofed one-way glass, so you can recline in comfort and eat pizza and M&Ms while the subjects talk. Ideally, the camera filming the group should also be behind the glass, so the participants don’t fell self-conscious. There should be a good sound system, so you can easily hear what’s going on inside the room. And the whole building should be reasonably insulated against outside noise. I once sat through a focus group where the participants almost had to shout to be heard over the chants of a protest group marching in the street outside the building.

The moderator is the person who guides the focus group, asking questions and keeping the conversation on track. A good moderator is like a good television talk show host — very alert to people, and capable of drawing them out by asking good followup questions. But unlike many TV shows, the moderator can’t be the center of attention. Sometimes a moderator will have an agenda, a preconceived idea of what the group should say, and they’ll subtly impose that on the group. You don’t want that.

You also want a good sampling of participants. You should never fill a focus group with company employees, and I’m uncomfortable with even using friends of employees. There’s too much chance of bias. A good focus group firm will have a large stable of potential participants that it can screen for the attributes you’re looking for, recruiting a good cross-section of the customers you want.

Typically the cost of a good set of focus groups will be around $20,000, and can go a lot higher depending on how many cities you want to visit. If I was short on money, I’d try to do one good set of groups in a single city rather than doing a series of groups on the cheap in several places. You’re not doing numerical research anyway, so the quality of the conversation is more important than the number of conversations you have.

Finally, you should make sure you get videotape of the groups. Get actual digital tape, not just a DVD — DVDs are notoriously slow and difficult to transfer into editable form on a computer (I learned that one the hard way). I also like to control the editing of any summary created from the video. At a minimum, you should tell the person doing the editing exactly which excerpts you feel are most important. If you leave this up to the focus group company, you may end up with a summary that misses the things you felt were most important.

__________

Please click here to rate this section (the link will open a one-screen anonymous survey, and you’ll get to see the results after you take it).

Next week: Quantitative research.

1. Almost any gesture or hand position you can imagine turns out to be insulting in somebody’s culture. Check it out. [↩ back]

The three perspectives a company needs in order to map the future are competitive analysis, market research, and advanced technology analysis. This week concludes our deep dive on competitive analysis done right, looking at the deliverables that should be created by a competitive analysis team.

There are four basic types of deliverable produced by a competitive analysis team: services, maps, spears, and news.

Services are things the rest of the company asks you to do, to help them with their jobs. For example, the product planning folks will want information to fill the competitive sections of their product plans. Finance may want stats on competitors, so they can do some benchmarking for an earnings release or an internal analysis. An executive may want some competitive tidbits for a speech.

It’s essential to keep these requests under control. Depending on the size of your company, and of your team, you may be overwhelmed by people who want help. You could spend all of your time just responding to these requests, leaving no time available for the proactive work that yields the most value to the company. The proper role of a competitive team is not to be the sole source of all competitive information; it’s to get a unique perspective on the competition by studying it full time. That lets you understand the competition, help predict the future, and produce the most devastating competitive sales tools you can. (That’s one reason why I dislike the term “competitive intelligence” — it sets the wrong expectations.) If you spend all your time helping people do their own jobs, you won’t have time to do yours, your company will suffer, and you may eventually be laid off because the company views you as “very helpful” rather than “essential.”

But you can’t just blow off the requests, if for no other reason than corporate politics. But I try to aggressively sort them into categories. Requests from powerful executives must be fulfilled, for obvious reasons. If you get too many of these, take up the issue with your management. It’s their job to either resolve the problem or get you more resources. If they don’t do either, it’s a good sign that you should start looking for another employer.

Requests from others in the company should be made as self-serve as possible. I’ll run with a request if it relates to information we needed to research anyway, or if it’s on a critical subject for the company as a whole. Otherwise, I refer people to the industry analysis services we subscribe to. Sometimes I’ll help people do the lookup if they’re new to the process. If they know what they’re doing, I just point them at the services and say, “All we know on the subject is going to be there. Let me know if you get stuck.”

I’ll give more information on how to manage industry analysis services in a future chapter.

Maps are long-term marketplace forecasts. They are the most strategic work done by a competitive team, and they take the most effort. Since they require input from the market research and advanced technology functions, I’ll discuss them in a future chapter.

Spears are competitive information that helps your company close sales or score points in marketing (they’re information you can throw at the competition, like a spear). Some companies call these “sales knockoffs.” Making spears is one of the most enjoyable parts of competitive analysis, and if you do it right you’ll be a hero in the company forever. More on spears below.

News is quick-hit information on something that just happened in the marketplace. You don’t want to turn yourself into a headline-clipping service; that’s the sort of non-value-added activity that gets a group laid off. But you should send around information when you can add value to it. If a competitive announcement will raise a lot of questions from customers, you should circulate information on what to say about it. If an announcement has important implications to the company, you should make sure the company knows that, quickly. Basically, you should turn the news you distribute into mini-maps and spears.

.
The joy of spear-making

Why make spears? If a competitive team operates perfectly, the company will never get in competitive trouble — you’ll steer the company away from potential problems before they happen, and you’ll focus the company on its biggest opportunities. But in the real world, competitive problems will happen. A competitor will come out with a superior product or an unanticipated tactic that puts your company in a bad situation. Or you may benchmark a new product your company has made, and discover that it doesn’t live up to its promises.

That means you’ll sometimes have to deliver bad news to the company. The consequences of that news may be devastating to an executive or even a whole business unit. That executive might be very powerful in the corporation, or you might have friends in the business unit. You will be tempted to try to soften the blow, by moderating your language or listing a lot of caveats to your conclusions.

For example, you might be tempted to say something like, “while the situation appears challenging, it’s possible that more aggressive marketing combined with price actions would be able to stabilize the company’s share position” when what you’re really thinking is, “there’s no hope of saving this business.”

It’s very important that you not soften the news. When people get bad news, they always look for outs that would let them dismiss the consequences and go back to life as usual. If you give your company an opportunity to develop false hope, you’ll prevent it from taking the action it needs to take.

But repeated brutal honesty can give you a very unpleasant lifestyle in the company, not to mention a short one. To balance the bad news, and to give you the political credits you’ll need to survive delivering it, it’s important that you also play a visible and positive role in helping the company win. That’s where spear-making comes in.

What are spears? Spears are information that can be used directly in your company’s sales and marketing. They help the company close sales, win industry debates, and impress press and analysts. They position you as a partner in winning, not just an ivory-tower analyst.

A good competitive team should be spending at least a third of its time making spears, maybe up to 50%.

Here are some examples of spears:

–Your team compiles a list of the five most important flaws of each of the competition’s products. You print these flaws on reference cards and supply them to the salesforce.

–You create a presentation on why your products are better than the competition’s, and make yourself available to go along on some customer visits with the salesforce. It’s important not to turn yourself into a full-time sales support team, but going along on some sales calls will give you important front-line information on what’s happening in the market, and will win you the gratitude of the sales organization.

–You hire a third party analyst to document how much cheaper it is to own your products as compared to the competition. You give the resulting whitepaper to the marketing team. They can reprint it as collateral, or quote from it in advertisements.

–You create a monthly audio recording summarizing recent competitive developments, explaining what they mean and what to say about them. You distribute the recording to sales, PR, and marketing.

This sort of recording is often beloved by salespeople and others who have a lot of dead time behind the wheel of a car. I’ve had some people tell me that they listen to every recording several times, sometimes re-running an old one that’s relevant to a customer meeting they’re about to attend. The recordings help them appear well-informed and answer questions from customers and other outsiders.

In the past I’ve distributed these recordings on cassette tape. There was usually some sort of mailing going out to the salesforce every month, and we could get the tape slipped into that, which saved us money on postage.

Today some new cars don’t have tape players, so I’d be looking to burn CDs. Or, better yet, you could distribute the recordings as podcasts (electronic files playable on an MP3 music player). The nice thing about the podcast approach is that you could store all the recordings on a company website, enabling people to download whichever ones they need at any time.

Sometimes spear-making work can become very elaborate. When I was at Apple, we were engaged in a long-running battle to show that the Macintosh computer was superior to those based on Microsoft Windows. At one point we paid an analyst firm to conduct a very elaborate customer test based on the old “Pepsi Challenge” marketing campaign.

(The Pepsi Challenge was a famous ad campaign for Pepsi Cola in which people blind-tasted both Coke and Pepsi, and generally preferred Pepsi.)

In our challenge, randomly-selected people were asked to perform a series of tasks on both a Macintosh computer and one based on Windows. The tasks were things like saving a document, or hooking up a printer and printing a page of text. The researchers kept track of how long it took to complete the task, what percent of people finished the task correctly, and how people felt about the task after they completed it.

It was an exhausting process for our team — we had to make sure the instructions for the tasks matched exactly the instructions given by Microsoft and Apple, the computers had to be restored to their starting condition after every test, and the researchers had to do everything very carefully so they didn’t favor one computer over the other. From start to finish, the project took most of an analyst’s time for several months and cost many tens of thousands of dollars.

But in the end, the results were great. For the first time, we could objectively quantify our product’s advantages in terms of speed, quality, and customer satisfaction. The marketing team created a broadly-distributed whitepaper describing the results, and the test became the foundation for an ad campaign.

This sort of project is called a “proof” study because it gives independent proof of something that you already knew was true. You’re not doing the study to find new information, you’re doing it to verify a claim. Corporate lawyers often require independent proof like that before they’ll allow your company to make a claim in an ad, so if you do a good proof study it’ll be loved by the marketing team.

Making spears is fun because it makes the company happy and gives your team a great sense of accomplishment. Some spear projects, such as the challenge study I described above, are also an opportunity to build teamwork between the competitive and market research teams.

Drawbacks of spear-making. Sometimes competitive analysts will object to making spears. Some of them view themselves as academics, and feel that creating marketing fodder sullies their independence. It’s important to help them understand two things. First, you’re not asking them to lie, just to help the company communicate its advantages. The process is like getting ready for a date — you may dress and comb your hair better than you would most days, but you don’t try to flat-out lie about your personality or background (anyway, most of us don’t).

Second, the analysts need to understand that they’re not academics; they’re working in a company that needs to make money, and sometimes they have to get their hands dirty helping to bring the money in. Besides, helping out for a while in the front lines gives an analyst a much better perspective on what’s happening in the market.

It’s important not to get carried away by spear-making, though. A competitive team’s most important asset is trust. People must trust that you’ll be honest with them at all times. That means your spears must be completely true, and you must never give anyone in the company an argument that can get them into trouble in a conversation. For example, if you tell a salesperson that the competition’s Product X causes toenail cancer in lab rats, that had darned well better be unimpeachably true. If in fact the toenail cancer study was discredited six months ago, some customer is going to know that and will call your sales rep on it, causing acute embarrassment, maybe losing a sale, and permanently ruining your credibility with that sales rep and anyone else they talk to. And you know how salespeople talk.

How to communicate competitive information

What’s the best way to deliver information? Go back to your fundamental goal — you’re trying to be sure the company wins. Therefore, communicate whichever way will get their attention. If it works best to tap-dance in front of corporate HQ carrying a banner, take dancing lessons. If skywriting works best, get a pilot’s license. Fortunately, in most companies you can use a mix of more traditional media.

E-mail is the quickest and easiest way I’ve found to communicate competitive information. One good approach is to set up a list server (an e-mail program that sends messages to everyone on a mailing list). Every time there’s a significant competitive announcement, or when your team issues a new report, send a message to the list server. Keep the messages short (no more than two pages printed unless something amazing happens), and always include an analysis of the implications of the event and what the company should say about it.

Remember, your role is to be an analyst, not just a news reporter. They can get news feeds off the Internet; your added value is that you say what it means, and you help the company’s spokespeople look informed at all times.

I’ll give tips on how to write for e-mail in a future chapter.

Ideally, this sort of message should go out the same day that the competitive announcement is made. If you send out information three days after the fact, everyone will have already read about it on the Internet and they probably won’t even look at your message.

Sending out messages on the day things happen has an important implication for managing a competitive analysis team — you can’t pre-screen your team’s messages before they’re sent. I know that’ll make a lot of managers uncomfortable, and fifteen years ago it would have given me hives too. But the accelerating pace of communication means that you need to focus on pre-educating people and then trust them to follow the rules, rather than using review cycles to enforce compliance. If you force everyone in your team to go through reviews on all their messages, you won’t be able to comment on events the day they happen, and people will simply tune you out.

This doesn’t mean you should turn an employee loose on the mail list the first day they start. While they’re getting up to speed you should definitely be reviewing everything they write before it’s sent out. This is for their own protection as much as yours — if they give a bad impression of themselves early on, it’ll be very hard to fix that later. But once they’re up to speed and producing reliable work, you should turn them loose.

Receiving messages from the competitive mail list should be voluntary. Don’t sign people up automatically when they’re hired. If a manager asks you to sign up his or her entire team, don’t do it — instead, send each of them an invitation to subscribe. If you’re in an active industry, you could be sending a message to the list almost every day, and some people just don’t want to deal with this. If you force all that e-mail into their mailbox, they won’t read it — but they will start to hate you.

The purpose of the mail list is to get information to the people within the company who want to be deeply informed. Very often these are the opinion-setters anyway, so they’ll take care of spreading the word to everyone else. But some people, especially senior decision-makers, won’t have time to read everything you write. For these people, you should create a weekly e-mail summary of announcements and events. For each new item, create a one-paragraph summary that says what happened and gives the implications. Your goal with this summary is to get across the basics and entice them to read the whole report. So keep your summary very short and include a web link at the end that lets them download your full analysis.

Even if you do a great job on the weekly summaries, some people don’t respond well to written information. So you also need to talk with them face to face.

Presentations are the other main way you’ll be communicating competitive information. Your team should develop a general presentation on the company’s competitive situation and advantages, something that can be delivered in about an hour. You use this whenever you’re asked to give an overview presentation to a department in the company (if you’re in a large company, you may get a lot of these requests as your team’s team’s reputation grows). You and all the senior analysts in the team should be able to deliver this talk. Don’t script the whole thing word for word, but make sure you all have a good understanding of the key points to make.

The analysts on the team should also create presentations on key competitive issues that matter to the company. This could be something like an analysis of a competitor, or an examination of an important competitive issue (for example, if you were working at a mobile phone company you might create an analysis of the various competitive products for accessing e-mail on a phone). Again, an hour is the ideal length for a presentation like this, although you can go to 90 minutes if you have to. Anything longer than that is a special occasion, and you’ll need to schedule a comfortable room and a bathroom break for the people you inflict it on.

It’s possible to just schedule a presentation time and invite people from the company to attend, but I usually try to get a regularly scheduled presentation slot at the staff meetings of the key groups in the company. In a high tech company, those would be marketing, product management (the people who create product plans), sales, engineering, and the executive staff. At least once a month, you should be giving them a presentation of the most important recent competitive developments and findings, and you can also slot in presentations from your staffers. All of this helps you reach the people who don’t read e-mail. Remember that the presentations, like the e-mails, should include implications, not just news reporting. And put the implications up front, not at the back.

Staff meetings can also be a good time to do quick demos of new competitive products, and let people know of important upcoming events (for example, if the rumor mill says a new competitive product is about to launch).

In addition to presenting, you and members of your team should have regular one-on-one meetings with key people around the company, so you’ll know what they are up to and what problems they have. You’ll be able to pretty quickly identify which people have the most influence and the best ideas, and they’re the ones you should focus on. Talking with them regularly will help you identify what important decisions the company is about to make (so you can influence them), and will help you figure out which pieces of information would be most useful to the company.

The latter is very important because a lot of competitive information will be coming across your desk, and most of it you’ll just ignore because you don’t want to overload the company. If you know what issues people are working on, you’ll be able to pick out tidbits that are relevant to them and route them over. This helps the company, and also wins you a friend. If you don’t do that outreach, you’ll only know the needs of the people who are pushy enough to come to you, and they may not represent the most important needs of the company. In fact, they usually don’t.

You should adapt your methods of communication to the culture of your company. I don’t do paper memos any more because people at my last several employers didn’t read them. I’m sure there are still a lot of other companies where people do. I have worked with some companies that use voice mail more than e-mail. This sounds strange if you haven’t seen it in action, but these companies usually have a founder or CEO who’s more comfortable communicating verbally rather than in writing, so the company culture comes to depend heavily on voice mails, forwarded extensively from person to person. In those companies, presentations can be a lot more effective than e-mails, although they’re a lot more time-consuming to deliver.

Other ideas for communicating:

Open up the competitive e-mail list server so that all employees can post information to it (or, if you’re in a large company, select designated participants