To differentiate between hype and reality, and better understand if Tesco could build a business from scratch in the US and take market share from the likes of Wal-Mart, we decided to ask customers what they thought. So, with Tesco’s permission, Execution commissioned an independent market research firm to interview customers as they left Fresh & Easy stores. We interviewed 680 customers over 6 days, both during the week and at the weekend (Wed 26 — Mon 31 March), in shifts ranging from 10am to 9pm. We interviewed at nine different F&E stores: three in Los Angeles, California, three in Las Vegas, Nevada, and finally three stores in Phoenix, Arizona. The oldest store in our group had been open for nearly 5 months (since 14 November) and the newest store had opened just 7 weeks earlier on the 7 February. Finally, our sample included converted stores and a purpose built store.
Yet upon analysis, the methodology can’t support the claims ultimately made for the research. Here are some of the things that make it problematic:
1. The study is of Fresh & Easy customers only — not a total market analysis.
There is nothing wrong with studying the customers of one particular chain. In fact, it is the only thing to do if the research question you are asking is related to whether, for example, Kroger customers might prefer a Prime Beef Program or a Sushi bar. The thing to do would be to survey Kroger customers and ask them.
However, you cannot deduce the opinions or behavior of non-customers by surveying customers. In other words, it may be that the people who are attracted to the Fresh & Easy concept shop there and those not attracted to the concept shop elsewhere.
Execution is aware of this research problem and tries to head it off:
Critics of F&E may already be dismissing the usefulness of our study precisely because it is a study of F&E customers and not the whole US population. Perhaps they are already writing F&E off as a niche business with no widespread consumer appeal. We disagree; we found that F&E is attracting a wide variety of consumers. While 70% of our sample were women, all age groups over 25 were well represented, as were singles (27%), couples without children (34%) and families with children (35%). The average gross household income of all shoppers was quite high at $82,500 (versus $70k US average), but it ranged from $68k to $100k depending on a store’s location and matched each store’s local demographic. If there is a group that is not yet particularly shopping at F&E, it would potentially be 18-24 year olds on lower incomes, and possibly very large families on lower incomes. Otherwise we think it’s fair to say F&E has a broad appeal.
We don’t think this is an issue of praising or being critical of Fresh & Easy. We see this as a matter of reflecting on the limits of different research methodologies. The bottom line is that if you don’t survey the people in range of the stores who are NOT customers, you can’t say anything about them. Maybe they have tried it and hated it. Maybe they heard about it but found the concept unappealing. You just can’t say.
Sometimes the researchers make the mistake of assuming that demographic differences — sex, age, marital status, child status and income — are all there is. They assume, without any supporting research, that because some Fresh & Easy shoppers are, say, female, this means that a broader female shopping base can be attracted.
This may be so. But it may not be — and the research does not support or deny the supposition.
Other times, the researchers just ignore the clear results of Execution’s own survey. As we mentioned here, their own data clearly indicates that the consumers whose household income is below the US median — that is half the households in America — don’t shop the concept. Yet the researchers keep concluding that Fresh & Easy has ‘broad appeal.’ This is very odd.
If the real distinction between a Fresh & Easy shopper and a conventional US supermarket shopper is not sex, income, marital or child status, but rather, say, an attitude toward fresh deli that requires or does not require a service deli counter, this study will not and cannot — because it does not study non-shoppers — capture the problem.
In effect, the researchers simply assume that whatever they have surveyed are the key variables. There is simply no support for this assessment. Very often, it is a psychographic trait — say, the way one values a fresh, cooked-on-premises hot rotisserie chicken vs. a cold cooked-yesterday-in-commissary rotisserie chicken that can determine shopping patterns. But because Execution did not survey anyone who was not a current Fresh & Easy shopper, it is in no position to say whether these types of consumer attitudes are or are not going to serve as obstacles to expanding the Fresh & Easy customer base.
In order for this research to be useful at making comparisons between shoppers and non-shoppers, you need to survey both. The failure to do so makes all the “conclusions” that relate to non-Fresh & Easy shoppers simply the opinions of the authors of the study. These specific conclusions have no grounding in the research study conducted.
2. Doing this type of research at the store biases respondents.
There is no problem with doing research at retail stores. If a manufacturer, for example, is interested in knowing why some consumers put its product in the basket, then it can watch shoppers and interview them as they leave the store.
But interviewing shoppers about the shopping venue when the shopping venue is the focus of the study biases the results. Consumers tend to want to please the interviewer and give the “correct” answer — or the answer the interviewer would prefer to hear. In addition, interviews such as these, conducted after a shopper finished shopping, intercept the shoppers with frozen foods, prepared foods and cool foods rapidly melting, spoiling and warming, respectively, sitting in their baskets. Consumers often ‘yes the researcher to death’ to get finished and in their cars.
That is why it is important that the interviews be done on “neutral ground” without groceries spoiling and that the questioner ask sufficient questions so that the respondent cannot know what the real focus of interest is in the survey. In other words, even though all we may care about is, say, Fresh & Easy, we have to ask lots of questions about Wal-Mart, Vons, Ralphs, etc.
The consumers were interviewed on Fresh & Easy turf, and it would seem ungraceful of the shoppers to criticize the store excessively — it would be like criticizing people in their own homes. Second, even if Tesco executives did nothing to distort the research, certainly the store manager and his employees will be aware of this activity. They will have every incentive to do everything in their power to get good survey marks. That can range from providing great service to calling Cousin Lucy to come down and try and be selected for interviewing.
So the location of the interviews and the fact that this implies endorsement of the survey by Fresh & Easy biases the results in two ways: It makes consumers speak more favorably of their host and it encourages the employees to change the reality of the store and the customer base.
3. Telling Tesco About The Survey Can Corrupt the Process
Getting permission from a retailer to do a study is fine if a retailer doesn’t care about the results. So a frozen lasagna manufacturer might get permission to survey customers about purchase behavior around its lasagna.
In this case, though, Tesco has deep interest in the study results. They know that, at very least, these results will go to important investors who could influence the Tesco stock price. In all probability, the results will also be leaked to the press and become public knowledge.
So letting the subject — Tesco — know it was being studied opens the possibility to an interruption in the integrity of the project in ways great and small.
A retailer can spruce up the store on the day the interviews will take place, make sure there are no out-of-stocks, and beef up the staffing. The store can mark down things liberally, do extra sampling and give out plentiful discount coupons so that the consumers are very happy that day.
All this can be done without moving into the range of clearly unethical behavior such as having employees or spouses pose as satisfied consumers, which can happen as well.
Now, let us be clear: We have no reason to believe Tesco executives did any of these things. We also have no knowledge that they did not.
The best research design precludes the possibility that these types of things could happen by conducting the research in a neutral venue that precludes the necessity of telling the retailer it is being studied.
