Marion Nestle wrote a piece titled, Jim Prevor on Organics, Crop Yields and Food Politics. We were, of course, flattered to be quoted so extensively by such a thoughtful and eminent scholar.
Yet when we read the piece, we were reminded of how often what seems like differences of opinion are actually differences in regard to facts.
Professor Nestle closed her piece by mentioning a “fact” — here is what she said:
According to the Organic Trade Association, organic sales totaled nearly $27 billion in 2010, and constituted 11% of produce sales. Is this “tiny and insignificant”? I don’t think so.
Now one can hardly fault Professor Nestle for these statistics — she is quoting the Organic Trade Association — a reputable organization. The relevant excerpt from the OTA piece Professor Nestle links to is as follows:
Organic food and beverage sales represented approximately 4 percent of overall food and beverage sales in 2010. Leading were organic fruits and vegetables, now representing over 11 percent of all U.S. fruit and vegetable sales.
Source: Organic Trade Association’s 2011 Organic Industry Survey
We’ll leave total organic sales to someone else to deconstruct, but we took one look at the produce statistic — “11 percent of all U.S. fruit and vegetable sales” and said that was way off. After all, less than 1% of all US Farmland is certified organic, so where would this 11% come from? In other parts of the report, OTA says that 92.5% of organic produce sales are fresh — 2.8% canned, 3.2% frozen, 1.5% dried — so this almost all is supposedly fresh product.
First off, one should be clear that we are talking dollar sales. The Nielsen Perishables Group has found that organic fresh prices average 80% higher than conventional, so if 11% of sales were organic, that would mean that organic would account for 6.45% of the volume of produce.
Now we’ve seen sales statistics from lots of mainstream supermarkets and few are near that number in volume or dollars. We’ve seen numbers in foodservice distributors and major chain restaurants and those numbers are even less. Even Farmer’s Markets and CSA’s don’t typically reach this number if one is talking about certified organic product as the OTA presumably is. We know the market share for natural food retailers and know that even if their organic percentages were much higher, they wouldn’t add up.
It is not even that the number is wrong; half the number would be way off.
So we called the Organic Trade Association and asked how they got that number. They explained that Nutrition Business Journal, a private publisher that produces various publications, trade shows, etc., that serve the organic and natural communities, did an industry survey on behalf of OTA. This survey was managed and conducted by Carla Ooyen, Director of Market Research at Nutrition Business Journal, with the help of the NBJ staff. They operated under the project direction of Laura Batcha, OTA’s Chief of Policy and External Relations, and Angela Jagiello, Associate Director, OTA’s Conference and Product Development.
In the end, the research product as it related to fresh produce was so unlikely to produce anything approaching accurate numbers that the OTA should be embarrassed for publishing such numbers. New numbers based on the same methodology are due out soon for the 2012 study, and OTA should put a kibosh on them before they publish.
Here, from the OTA’s 2011 Organic Industry Survey, is the “Data Collection” portion of the methodology:
1.2.1 Data Collection
A primary objective of this project was to collect and compile data from the manufacturing community in the organic industry. To maximize the amount of information collected, NBI structured a survey similar to previous OTA surveys to facilitate quantitative assessment and comparison and to assess qualitative issues impacting the organic industry. NBJ distributed the survey and collected the responses through an online survey network. Further e-mails and telephone calls were used to improve response rate and clarify survey responses.
In the survey, NBJ assessed the “universe” of manufacturers in organic products so that survey data could be most effectively modeled into total industry statistics. In other words, traditional statistical modeling usually involves random sampling of defined subcategories (i.e., organic manufacturers between $5-10 million in sales) and conversion to total based on the knowledge of the survey universe. In this case, we used a somewhat hybrid approach, where respondents were not randomly selected, but rather ‘self-selected’ by their responses and our efforts to make the entire industry aware of the survey. Distinct efforts were made to obtain responses (or as a last resort revenue, product breakdown and sales channel estimates) from the top 100 or so manufacturers and private label brands to maximize the “revenue capture” of the survey database and industry model.
Of course, the bias slips in right at the start. If the “primary objective” is to “Compile data from the manufacturing community in the organic industry,” then the survey may provide useful information about organic manufacturers, but the survey will not be useful to compare organic and non-organic market share because the survey will not generally include non-organic manufacturers or at least the non-organic manufacturers are highly likely to be under-represented.
We will assume the word “manufacturers” is broad enough to include farmers and packers. Even so, to generate market share statistics, one needs to have a picture of all manufacturers — organic and non-organic. This survey wouldn’t do that.
Ok, so who did they actually survey?
“…respondents were not randomly selected, but rather “self-selected” by their responses and our efforts to make the entire industry aware of the survey.”
OK, in this case “entire industry” must mean the organic industry, not the fresh produce industry. We know they didn’t advertise the survey on the Perishable Pundit, nor PerishableNews.com’s produce channel. They didn’t run any ads in PRODUCE BUSINESS magazine, nor did they exhibit at The New York Produce Show and Conference — so whatever “efforts” were made in produce were limited.
More broadly, self-selected surveys are pretty close to worthless. One needs to make sure that survey respondents correspond to the general population one is looking to survey. It is especially easy to throw the numbers off if one only markets in a particular community.
Think about doing a survey where the “primary objective is to compile data from the Jewish community”… to promote responses, you advertise in publications geared toward Jewish people, get press releases published there, hand out flyers in synagogues, etc.
Now you might gain some useful information about Jewish people — although you would need to be very careful as certain groups, say younger Jews, may not read the Jewish newspapers or affiliate with synagogues in proportion to their numbers.
What is very clear is you couldn’t possibly use such a survey to compare the percentage of Jews in the population with non-Jews.
So what might be a reasonable estimate of organic market share?
Well the Nielsen Perishables Group maintains a database from scanner data that represents about 62 percent of US food retailers. Steve Lutz is the Executive Vice President at the Nielsen Perishables Group, and he tells us that their data shows that organic accounted for 5.5% of produce department sales for the 52 weeks ending July 30, 2011. This is pulled from scanner data so is much more accurate than survey results.
Once again, this is a dollar figure and the Nielsen Perishables Group has found that organics, on average, are priced 80% above conventional produce. So if 5.5% of sales is organic in dollars, that would translate to 3.1% in volume.
Yet this number is probably way too high as a percentage of produce sales. The Nielsen Perishables Group database is an incredible resource, but it does not include Wal-Mart, and Wal-Mart skews significantly lower on organic sales than mainstream supermarkets, such as Safeway and Kroger. This alone could drop a point or more off the percentage that organic represents.
Then we know that at retail, where consumers can see the organic product and elect to pay more for it, organic has much higher penetration than in foodservice. Sure white tablecloth restaurants use organic sometimes – but that is 1% of the business. Organic is hard to find at McDonald’s.
It is difficult to figure consumer prices in foodservice as the product is not sold independently from the ambiance, service, etc., but if we figure that retail dollar sales are 4.5% — the Nielsen Perishables Group number less a percentage point for the Wal-Mart effect — then with organics costing 80% more, the volume of organic at retail would be 2.6% of the volume.
It is reasonable to think that foodservice is half that percentage. In other words about 1.3% of the produce used in food service is organic. If foodservice accounts for 35% of produce usage, that would mean organics account for 2.15% of the total volume of produce sold in America.
We always like to give people the benefit of the doubt, so we assume that this is all an innocent mistake caused by OTA choosing to contract with a company not really deeply involved in fresh produce to do its research in this area. Though many, less charitable than us, would note that OTA has an interest in exaggerating the importance of the organic industry and that it may have purposely contracted its research out to an organization that produces the Natural Products Expo and similar products and thus has a similar interest rather than to a neutral party.
Surely OTA wants its research to be above reproach. It should consider working with a reputable university-based researcher. Perhaps we should give our friends at the Organic Trade Association the contact info for Ed McLaughlin, the Robert G. Tobin Professor of Marketing at Cornell University. He and his staff have done dozens of these studies for the fresh produce industry and other facets of the food industry. If the OTA is really interested in producing credible research, a staffer can contact Ed right here.
