A Closer Look At
Finished Product Testing

Few issues are more contentious… or more important to the industry than finished product testing. In our piece, Church Brothers/True Leaf Recalls, Then ‘Unrecalls’ Spring Mix/Arugula After Testing Mishap, we urged the industry to cease its common practice of “test and release” and switch to a “test and hold” system. Here is how we put it:

…this business of testing finished product and releasing the product before the test result is back is a disaster waiting to happen. There are new 12-hour salmonella tests being required by McDonald’s and already in use by companies such as Earthbound Farms. They should become the industry standard, and the practice should be test and hold — release only after the results are in. Most product doesn’t ship out the instant it comes off the line, so this 12-hour test is not going to pose many problems.

Almost immediately, we received a note from a processor urging caution on the 12-hour tests. We ran that piece under the title, Hold The Train…12-Hour Test May Not Be Best Answer, and it included the following excerpt from the processor:

After reading the Pundit on True Leaf and the false positives, I couldn’t help but think there may be an issue regarding the effectiveness of the 12-hour PCR based tests on the market. Many in the industry, us included, are using it based on pressure from accounts and not necessarily based on scientific accuracy. The PCR is a presence / absence test that looks for genetic material of e-coli and salmonella. It does not tell you if the cells are dead or alive, if it was one or one million cells, or if the genetic material was from a current or past event (i.e., a bird strike on the ground 12 months ago).

The developer stands behind the results and is very reputable so there is no issue from that standpoint. However, no one seems to have the scientific data (at least it’s not shared) that helps us determine the true accuracy. We all know painfully well the rate of false positives with the conventional testing methods.There was also a symposium held May 31, in Chicago, in which FDA and CA Dept of Health participated. One concern raised was the accuracy of the PCR test based on the very limited data complied and the speed at which the industry is ramping it up.

A well known food safety expert wrote us shortly thereafter and we ran his thoughts in a piece entitled No Quick Fix With 12-Hour Test:

A visiting scientist from the UK commented on how flabbergasted he was by the entire concept and the lack of critical commentary from the food safety community associated with the fresh produce industry.

We let him know that many of us agree with his comments but have been advised to keep our opinions to our selves.

The primary advocate for this program calls for a massive level of testing.When the dust settles, I don’t believe this will be viewed as one of the fresh produce industry’s brighter moments.

It is taking a validation tool and turning it into the center of a food safety program. That should, in and of itself, raise questions.

There are many problems with the program but the largest is the over selling of the effectiveness of the sampling effectiveness in identifying sporadic contamination events.

In reality much of the testing is primarily used to test artificially reduced lot sizes to minimize economic damage. This will ultimately be challenged by the FDA.Only time will tell if the program accomplishes this goal. These are not dollars spent to address the problems.

A few issues the industry might consider before we join a bandwagon for this test:

1. The test is not AOAC approved. Advocates have been saying it will be AOAC approved in 4 to 6 weeks. Unfortunately they have been saying that for nine months! AOAC is the third-party arbiter, devoted to analytical excellence, which gives the recognized “stamp of approval” to these types of tests.
2. The test is not widely used by professional microbiologists. It is limited to a single firm and not part of standard industry practice.
3. The industry seems to be attempting to “test in” quality or safety, Unfortunately, though testing can be useful for verification, substantive improvements in process are necessary for significantly enhanced safety. Resources spent on testing programs are typically not available for more substantive efforts.
4. Testing is a feel-good approach for buyers who prefer a simple test rather than having to wrestle with the difficult task of prevention.
5. Resources are being pulled away from the implementation of preventative and more sustainable solutions.
6. The testing program provides a false sense of security. Testing — if you are testing for that thing — may be a good way to identify certain types of things such as a chemical incorrectly applied to a whole field. However, testing for bacteria cannot reliably find things such as E. coli 0157:H7, which may be due to sporadic animal entry in one tiny part of the field.
7. The sampling from a fresh produce matrix (in contrast to homogenized liquids) may not deliver the level of assurance claimed. We don’t actually know enough about this test yet, to leap to the conclusion that it can have any place much less the determinative place in our industry food safety programs.
8. The math on testing is daunting. Has anyone attempted to project the increased testing in the fresh produce industry required to accomplish gains similar to the improvement in beef? For example, if the beef industry has succeeded in lowering a 1.5% level to say 0.6%, what level of testing might it require in produce if our contamination level starts at, say, 0.015% or even lower?

We then ran prevention Should Be Higher Priority Than Testing/Auditing, which included a letter from a recognized food safety expert that included this salient point:

The current issue with ginger is a classic example of the limits of any testing program. After spending literally millions of dollars testing for microbes, which is what is being spent by one firm, the industry is once again blind-sided by a pesticide issue. And all of the microbial testing in the world wouldn’t have addressed the ginger issue.

The industry, including the buyers, needs to stay true to the basic concepts of hazard analysis and make prevention from physical, chemical and microbial contamination their highest priority.

Neglecting or ignoring any of these points in order to deal with the crisis of the day is short-sighted.

After the Metz Fresh recall, which we discussed both here and here, the issue faced the industry front-and-center. Clearly any recalls because of presumptive positives are a real problem for the industry, creating doubts about the trade’s ability to protect consumers from contaminated products.

Yet many food safety experts and many companies known for stringent food safety protocols are concerned about testing.

To learn more about the subject, we turned to Mansour Samadpour. A microbiologist and founder of a food safety company, he became well known to many in the produce trade when, in the aftermath of the Fall 2006 spinach crisis, Natural Selection Foods turned to him for assistance in revising its food safety program. Here is how the Los Angeles Times described the meeting:

Days after the tragedy unfolded in mid-September, the company hired food safety microbiologist Mansour Samadpour. Known for his no-nonsense approach, Samadpour sat down with Daniels and Earthbound Farm President Charles Sweat and warned them that they were delusional if they thought it wouldn’t happen again.

“Another bullet is coming your way,” he warned. “Are pathogens in raw product reasonably likely to occur? Some say yes, some say no. The group saying no is delusional,” Samadpour said. “The question you have to answer is, will the processing eliminate the hazard? The answer for this industry is no. You can reduce; you cannot eliminate.”

Under the scientist’s guidance, Earthbound rapidly put in place the most aggressive testing and safety program in the industry. All its greens are now checked for pathogens, from seed to sale. Each lot is tested twice — upon arrival from a farm, and again when packaged products roll off processing line

The testing has confirmed what Samadpour already suspected: Inevitably, some crops are still contaminated with disease-causing bacteria. The challenge for the company is to make sure none reaches consumers. Hunting down pathogens in produce has become a personal crusade at Earthbound Farm. In the year since the E. coli outbreak, other companies have mounted costly safety efforts, but no one else tests all greens.

Mira Slott, Pundit Investigator and Special Projects Editor, spoke with Dr. Samadpour:

Dr. Mansour Samadpour Founder, Principal IEH Laboratories & Consulting Group Lake Forest Park, Washington

Q: To start, could you tell us about your background and experience, as well as an overview of what IEH does?

A: I had microbiologist training at University of Washington. After getting my PhD, I did post-doctorate work and joined the faculty of the School of Public Health, Department of Environmental Health, involved in early detection of foodborne outbreaks and progressing into preventing outbreaks from happening.

I have done work related to almost all major foodborne outbreaks for the last 20 years or so. Most of the serious ones happened in the beef and the ready-to-eat industries. My involvement included areas of managing the outbreaks and the recalls, and gathering information to identify contaminated lots and making sure recalls were comprehensive. I also looked into causes of the outbreaks, and coming up with measures to prevent them from happening again.

I started IEH about six years ago and at this point have about 27 offices with more than 45 PhD scientists on staff. I tried to create an environment with the necessary expertise to cover all the consequences of recalls and for getting clients to have preventative measures against outbreaks.

Q: What is the evolutionary history behind the 12-hour food safety test? What is it exactly? How and why did it start to be administered in the produce industry? Could you discuss the testing methods and procedures out on the market?

A: The first point I need to draw attention to is the fact a lot of rapid methods are available. A number of tests allow detecting pathogens in food in approximately 10 to 12 hours. Twenty years ago maybe it was distinctive, but certainly not now.

Q: Why is the produce industry so late in adapting it then?

A: There is not a lot of familiarity with this microbiological access. The industry is accustomed to using services with those specializing in the industry and never had the need to move to rapid methods.

A major need arose when the beef industry went to test-and-hold. A similar profile applies to the produce industry since the spinach E. coli crisis. Product with a 17- to 20-day shelf life can’t afford to lose two or three days for testing. For some time, the beef industry testing process was 24 to 48 hours, and then as needs dictated, E. coliH:O157 tests were cut to 18 hours and then 12 hours.. There is a misconception in the produce industry that this is something ultimately new and what we do is unique. We’re not the only game in town.

Q: How does the rapid test work? What does it show? What are its limitations? For example, some industry executives say this is a presence/absence test but won’t determine if it’s one cell or 1 million cells or whether it’s dead or alive or from a current or past event.

A: First let me reiterate that there are many rapid screening tests in the market. The IEH method is only one of the several methods that are being used for the produce industry. They are generally divided into two broad categories of immunochemistry-based or genetic-based.

The immunochemistry-based methods are mostly lateral flow devices. They resemble pregnancy test devices, which are sold in pharmacies. The genetic-based ones in the market are usually PCR-based. PCR stands for polymerase chain reaction. As a general rule in food microbiology, we always have to enrich the samples. This means adding appropriate nutrients to the sample and incubating the samples at 35-42 degree Centigrade for a period of time (usually eight hours). This allows very low levels of pathogens in the sample to grow to high enough numbers that they can be detected by the detection method (lateral flow devices or the genetic-based methods usually a PCR method).

Although both methods in theory can detect dead cells, in reality there are never enough dead cells (or live cells) in a sample to give a positive signal with any of these test methods. That is why we need to enrich the samples. In an eight hour enrichment, the pathogens will go through 16-20 multiplication cycle (one cell going to two, four, eight….1000…….1000,000 cells) . It is only after the enrichment process that the pathogens can be detected by the available methods.

The assertion that the methods detect dead cells in samples is simply wrong. As long as live cells are present in the sample that is being tested, the issue of when it was added is irrelevant.

Q: How accurate are these rapid tests?

A: The accuracy of the test results is a function of the sensitivity and specificity of the test method, and the skills/proficiency/competency of the laboratory personnel who are conducting the tests. The second factor is often overlooked. The best available methods used by incompetent technicians will result in a large percentage of false positive or false negative results.

For a method to be used, it has to have been validated for the type of samples for which it is used to test. For a laboratory to use a method, they also need to do their own internal validation studies to be conforming with ISO 17025 standards. The labs should also receive check samples.

Your readers can go to the American Proficiency Institute website to see examples of proficiency test results for various methods/labs.

Q: Can you discuss the reasons behind the false positives in recent, highly-publicized food safety incidents?

A: Let’s talk about the issue of false-positive results. I’m aware of two situations; one was last year’s false-positive green onion sample. In all fairness, we should say it was a presumptive positive, which was due to cross reactive antigens (a common problem with lateral flow devices). Another false positive was the recent incident with finished product testing that showed a positive Salmonella and precipitated a recall with True Leaf Farms’ spring mix and arugula products.

[Editor’s note: True Leaf Farms later received notice from Silliker Inc., the Northern California laboratory that previously reported the PCR positive result for Salmonella, that it was probably erroneous and admitted that the test had been tainted by laboratory error. The type of Salmonella isolated was identical with the lab-positive control strain indicating lab contamination was the most likely cause for the positive result. The lab-positive control strain is an extremely rare strain not normally found in foods, further reflecting that laboratory error led to the positive finding. New test results on the same product batches from another independent lab showed no indication of contamination. All tests were negative.] [ Editor’s note: IEH Laboratories was not involved in the above-mentioned false positives].

There are two main reasons why there are false positives. One is the test is cross reacting with a different bacteria. For example, other bacteria with the same DNA sequences or antigens can cross react with the detection system and give false positive results.

The second mechanism is that a sample will get contaminated (through technician error) by the positive-control organisms, which microbiology labs use as a part of their quality-control program, or byorganisms from a positive sample (a sample that contains the target organism), which is being run in the proximity of the sample (in time or place).

Q: How often does that occur? What you describe sounds like a lesson that should be learned in science lab 101.

A: This happens more than you think! When you grow bacteria, a tiny droplet could have more than 100 million cells in it. It can result in massive contamination.

In the beef industry, during a daily testing of the production, a company had 22 lots of beef trim test positive for E. coli O157 using a rapid screening test. The samples were flown to IEH Labs in Seattle. In four hours, we confirmed that the samples were positive for E. coli O157 (using molecular confirmation). Two days later, we had isolated E. coli O157 from the samples. When we subjected the isolates to genetic fingerprinting, we found that the O157 isolates has identical fingerprints as the test lab’s positive control strain. This clearly showed that the samples were contaminated by the lab’s positive control.

Q: What steps can be taken to alleviate these problems?

A: We have taken drastic measures to eliminate the cross contamination issue in our labs. This includes the unprecedented step of using a combination of non-pathogenic bacteria and DNA to replace pathogenic control strains. We simply don’t grow pathogens in our labs. Different types of samples and different runs are bracketed by negative control samples.

Another unique way in which we have addressed the false-positive issue is by using a large number of signals to detect a given pathogen. Almost all test kits in the market rely on using a single signal to detect a given pathogen.In our initial screening of a sample, we use five signals to detect E. coli O157.

Some of the five signals have to be simultaneously present to give a pathogen profile for E. coli O157.While using other test-rapid screening methods, other labs have to pronounce a sample as presumptive positive at this stage (remember the green onion debacle). Using five signals, if we see a pathogen profile for E. coli O157, we call the sample “Initial Reactive” (IR). It is only after going through “Molecular Confirmation” that we call a sample “presumptive positive”.

Q. What is Molecular Confirmation?

A: When testing for fresh food industries, no one has 3-5 days to wait for a sample to be subjected to culture confirmation. Culture-confirmation protocols have two problems; first they are slow, second, by their nature they are not as sensitive as molecular methods, and a truly positive sample with high background of non-target organisms can test negative in culture confirmation. For this reason, most beef companies take action on the basis of rapid-screening methods.

In order to bypass both of these deficiencies, we have developed “Molecular Confirmation”. Each IR is examined using 17 signals to determine if it is E. coli O157. This is accomplished in a few hours. If a sample is positive for E. coli O157 using Molecular Confirmation, we will report the sample presumptive positive. At this point we rely on the client’s instructions as to whether or not to perform culture confirmation.

Q: Has your Molecular Confirmation method ever been published in a peer-reviewed journal and/or other places for independent assessment?

A: I haven’t published a paper on the methodology. Sometimes in published works, the methodology is described. We do have several papers in which we have used our methodology with this particular method of detection. When we are doing an investigation, samples are tested and processed using molecular confirmation and we get conclusions and we publish a paper. I’ve never published a paper on the development of testing methods for E. coli.

Q. What about false positives?

A: From the stand point of public health and safety of consumers, we consider false negatives to be a far more significant problem. Inability of a method/lab to detect contaminated samples can result in release of contaminated products into the market. Having said that you also have to realize that for a lab to be able to detect a positive sample, the sample has to be positive above the lower detection limit of the test method.

There is also the issue of whether the sample represents the lot from which it was taken and the statistical confidence behind negative results. For instance, in a lot-acceptance testing program, when a sample is negative for a given pathogen and the lot is accepted, the probability of having accepted a defective lot is a function of the number of samples that were taken from the lot. We also believe that when the contamination is unevenly distributed, increase in the size of the test lot results in a decrease in the confidence in negative results.

Q. Some industry executives say there are naturally occurring good bacteria in produce that could throw the test and give a lot of false positive results creating additional challenges. Could you discuss?

A: What these industry executives have heard is correct. However the problem is not unique to the produce industry. It all goes to proper design of a test method. The problem with rapid-screening tests (affectionately known to your industry as 12-hour tests), is that most tests have been simplified to a level that have left a door open for non-target organisms to give false positive signals.

Even more dangerous than the false positives are the false negatives generated by some of these methods. Our way of eliminating this problem has been to take the simple out of these simple tests. Our methods of detection are very complicated and above the competency.

Q: We’re hearing from industry executives that about six percent of product is testing presumptive positive by PCR methods but that about 85 percent of the PCR tests are not confirmed positive. With all the additional measures you take, what percentage of false positives are you getting on your tests?

A: We’re getting zero false positives. In doing these rapid tests, you get a tremendous mixture of bacterias that can give you false positives. Everyone else calls this presumptive positives. We call this stage initial reactive. We tease out this bacteria through molecular confirmation. This examination is so comprehensive that when you call it a positive, it is a positive, no maybes, ifs or buts.

The process of growing presumptive positive samples out traditionally takes three to five days. We have invented this method of molecular confirmation to speed the process to about four hours. We are the only one that does this molecular confirmation.

Q: Can a company buy this molecular confirmation method from you?

A: No one has approached us to buy this test. When a company has a presumptive positive, it can send the sample to us to do a confirmation. We have this service. In an emergency, we will do this free of charge. When there is a serious situation and the possibility of an outbreak, for God’s sake, send the samples to us and we’ll run the molecular confirmation test at no charge. In emergency recalls, people in the industry know what we do and they could bring in the samples and we wouldn’t charge them a dime.

Q: If this test eliminates the chance of false positives, I don’t understand why industry executives aren’t clamoring to buy this molecular confirmation test from you. If after a testing lab or produce company reads this interview, and they come to you to purchase the kit, will you sell it them?

A: The problem is that the test is complex and requires a high level of competency to use it. If you have a fantastic rifle designed for sharp shooters, you do not want it in the hands of people without training. These are issues we’re struggling with. I could have sold so many test kits. Testing produce is much more challenging then testing raw beef. A gram of leafy greens can have anywhere between 1,000 and 100 million or more bacterias. It makes testing a difficult, challenging process and takes a lot of expertise to do it properly.

Q: Why don’t you just control distribution to reputable companies with expert skills?

A: We could have partnerships; we have acted as a reference lab for years. There are many companies that send their samples to our forensic analysis lab. It has never been our intention to be a test manufacturer and it’s not our business model.

It takes much expertise to run these samples, and our molecular confirmation method is not designed like a regular test kit that has been simplified for general use. I want to reiterate that in a crisis situation, we are willing to do confirmation from any location for any company at no charge and are willing to give them answers in a few hours.

For me personally, although people talk about test methods as the issue, once you use a reputable, ISO-certified lab, the biggest challenge and problem that we have as an industry is the issue of proper design and execution of sampling plans. An improperly designed sampling plan will generate negative results with very little confidence, and once it finds a positive sample, even if the test lot is under control, may result in massive recalls. We have to make sure that a sampling plan will deliver what it was intended, and will not have unintended consequences.

Q: What do you mean exactly?

A: For instance, when a company designs and implements a testing program, the sampling plan should have statistical justification to ensure an appropriate confidence in negative test results. It must also be prepared to deal with all consequences of having a positive test result. If the testing program is poorly designed, the company could end up doing a massive unnecessary recall.

For instance, Customer A requires finished goods testing.If all you do is test finished product, you may test positive. Let’s say there has been at least one confirmed finished goods positive test by another laboratory. It then can implicate everything you co-produced, plus everything harvested from the same fields/ranches (which were not tested through a field or a raw material testing program).

The implicated fields may implicate other processors (if they don’t have a defensible testing program). It can mushroom and get out of hand really fast. People have to be really careful.

My advice is that if you are not going to have a properly designed and comprehensive verification/testing program, you shouldn’t have one at all. If you are testing final product, you have to test raw product coming into the processing plant and the fields from where you harvest.

Q: There is talk in the industry that batch testing by lots using numerical formulas and statistics is an artificial way to test.For example, say each lot consists of 400 boxes, you test one lot and get a positive so throw out those 400 boxes. Then on the next 400 boxes you get a negative result, so you don’t need to throw it out. But what if that next lot came from the same field with the same characteristics? Wouldn’t it be better to set up testing by characteristics, such as where the product came from, when it was sanitized, etc.

A: When doing a testing program, you don’t go willy nilly taking two samples from this line, and two from this one. You assess products through the same line and joined from the next line, sharing the same raw material, etc. This is a science based on flow of production, how product moves through the processes, as well as statistics defining sample sizes and lots and controls backward and forward.

You can’t grab one sample and say it represents the entire production of one shift of 50,000 pounds. The sample you take has to represent the lots from which it’s being taken. There are international standards of the number of samples you take for E. coli O157 testing salmonella or lysteria, etc. This is lot -acceptance testing. This standard was written on consensus of scientists and on severity of outcome based on the pathogen.There’s an international organization called ICMSF, where you can read more about this.

General indicator samples may be five, lysteria samples may be 30. The smaller the size of the lot, the more confident you’ll be in your data. Say every two hours, every line. We take a lot of samples. From a single line, you may take 15 lots, 60 samples each. This goes into the design of the sampling plan. Then you have to have a hold program. Everything implicated is on hold and going backwards you’re also safe.

These are lot-acceptance programs. You set up standards for accepting this lot. Before you send this raw lot, it has to pass through your microbiological criteria for no salmonella, E. coli O157, or lysteria. Anything that doesn’t pass is rejected. You’ll have a lot of negatives. All you’re saying is they met your criteria, not that they are free of pathogens. Now you accept the raw material and put it through the wash system, and now you are testing final product with microbiological criteria, using the sensitivity of your test, it must meet your criteria or it’s rejected.

There are people who say guilt by association; this field is 100 acres, we should never touch that field again. There isn’t much science behind that assertion. Within the limits of the detection of the test, these are safeguards in the system.

Q: Do you have a recommendation of what percentage of product needs to be tested for there to be meaningful results? Could you give us a statistical perspective? If some 99 percent of produce is safe, what amount of testing would need to be done to catch that 1 percent? Do you believe, for example, if Natural Selections quadrupled the number of tests it conducted, we would see that many more positives?

A: In the beginning when designing the program, you have a food safety objective you are trying to achieve. The purpose of testing is not a magic bullet to make everything safe. It’s a component of a food safety program. If you go to a university, you prepare before taking the tests. It’s a verification step. The test is a means to establish if the student has learned anything.

In food safety testing, you see if the food has met the standard of microbiological acceptance criteria you’ve established. It’s a verification and validation of everything else you’ve done. If the company has great GAP programs, it won’t have a problem.There are two groups of testing philosophies: the first, despite the fact we have a strong food safety program in place, in order to verify it and establish it works, we’re going to test raw material and finished product, and another group saying, we don’t need to test because we have good practices in place.

Q: With the relatively small percentage of product actually being tested out of the millions of products being produced, could this be like finding a needle in a haystack, in essence identifying sporadic contamination events?

A: Again the key is to have a statistically defensible sampling plan. The needle in a haystack argument is useless here. The fact is that we are testing and we are finding pathogens in leafy greens. Raw materials that test positive are diverted from processing and are destroyed. Finished products that test positive are diverted from the market and are destroyed.

The second part of your question is actually correct. In most instances we are dealing with sporadic events. So far we have seen one instance where the problem has been systematic and not sporadic. Having said that, when the consequence of a sporadic event can be a massive outbreak, we all have a moral duty to minimize the possibility of a sporadic contamination event mushrooming into an outbreak.

Q: Hypothetically, a company has an extra $3 million a year to spend on food safety. Should they spend it on additional testing and/or improving soil, field and processing food safety systems?

A: Look at your budget and proportion it out. You have to continuously improve, validate and verify systems. It’s silly to put all of it in A and never measure whether A is useful.

Q: Some in the industry are concerned that testing could create a false sense of safety; a danger of taking a validation tool and turning it into the center of a food safety program. Some argue that while there is nothing wrong with testing, it could pull valuable resources away from the implementation of preventative and more sustainable solutions. What are the costs involved with testing? Could you address these concerns?

A: At the end of the day, each company has its own risk management, and it is not my place to make a comment on that. Suffice to say, for anybody that produces food in this country, it’s a matter of law that food is safe. It’s a felony to sell unsafe food. Now if you believe that through GAPS you can control bacteria and guarantee raw materials are not contaminated, and you can’t possibly have E. coli and salmonella, or washing will eliminate the pathogens, then that’s the level of risk you’re comfortable with.

Another company decides not to take any shortcuts, makes sure all preventative measures and interventions are established, but on top of that is also going to use testing to validate and insure all of these things are working. There are deviations to the norm in perishables, and contaminations can occur. Testing is another part of the food safety program; it enhances food safety. You can’t control what you are not measuring. It’s another component.

Just like your industry, at one time, the majority of decision makers in the beef industry were against having an E. coli O157 testing program. Outbreak after outbreak and recall after recall became the impetus behind the testing program, alongside massive efforts in interventions and improving the process. It came to a point that with a single directive in 2002 USDA-FSIS directed the industry to comprehensive testing programs for components of ground beef.

Each year more than 100 million pounds of beef trim and ground beef test positive for E. coli O157, and they are diverted from the fresh market. In the absence of the testing program, even with a comprehensive array of interventions, the beef industry would have unknowingly shipped more than a 100 million pounds of highly contaminated products to grocery stores and restaurants. This is a prime example of the positive role of testing as a verification step in a food safety program.

Q: But in the best scenario, wouldn’t the percentage of contaminated product discovered through testing in the produce industry be a very small fraction of that found in the beef industry based on the nature of the product?

A: We don’t know that yet. After a few years of testing, we will have a much better handle on the statistics (that is if the companies stop using food safety as a competitive advantage and start sharing data and protocols). But even if the percentage of the finished goods that test positive are low, given the fact that unlike ground beef, few people cook their salads, and any level of contamination with frank pathogens can result in a public health impact, are we comfortable with having our processes go without verification and not doing our best to find them, and letting the contaminated lots get into the chain of commerce?

Q: Could you discuss the issue of mounting recalls in the produce industry?

A: No one in a sampling program should have recalls because the essence of this testing program is that everything is on hold, and everything tested is also controlled. We put a lot of emphasis in developing proper programs. Presumptive positives come in different rates and in different ways. My opinion is that the importance of designing a proper program is 90 percent and choosing the method is 10 percent.

A lot of people are testing with no plan of action. And that results in recalls. There is no reason we should have had false positives in a lab or presumptive positives that resulted in recalls. In the absence of properly designed testing programs, companies are going to continue having recalls that will hurt the entire industry. These types of recalls are crises that could have been avoided and are truly self-inflicted wounds.

Q: What overall advice could you share with the industry on the best strategies to improve food safety?

A: Food safety is a constant struggle, with the constant need to measure and assess what’s working and what’s not. There are many, many challenges. No one at any point should get proud enough that they say they don’t have a problem.

Food safety is not an event that should be used as a comparative advantage. The industry needs to work together, share information and strategies and ways to begin to improve. We have many hurdles to overcome. This type of attitude that someone is using testing and not using anything else is just wrong.At the end of the day anything you do in a food safety program has a cost and introduces a level of complication. Testing helps take away some of the recalls and outbreaks and provides a chance for the industry to grow.

Whether it’s adding a new wash system or introducing different chemicals and strategies for reducing microbials, it should be encouraged. The last thing the industry should be doing is compiling reasons for why something to improve food safety shouldn’t be done.

This is a lengthy interview with a lengthy introduction, but on this issue much depends. Routine testing of raw material and finished product will cost the industry tens of millions each year just for fresh-cut leafy greens. If we were to extend testing to other types of produce, we could add another zero.

Yet, if it works, it could save the industry countless dollars by avoiding recalls and public outbreaks.

And, of course, this is a priceless benefit: If it works it could save lives and prevent serious illness.

We appreciate Dr. Samadpour speaking with us. He has been around a long time, going back to the famous Jack–in-the-Box outbreak. He has a long-established reputation among many of the food safety experts who have worked in beef, and he is frequently called on by the media for comment.

We confess that we want to believe. Although testing is expensive, it seems to us much, much less expensive than food safety outbreaks. So if we can really test everything in less than 12 hours and then release it confident in its safety, it would be worth the price.

Yet even after reading Dr. Samadpour’s words and recognizing him as both intelligent and knowledgeable, we find ourselves still asking questions:

1) PEER REVIEW

When people develop new ways of doing scientific work, one expects these new methods to be tested against the old methods. So, if a new test is developed, one would expect a trial to be done in which the old standard and the new test are done side by side. The results of these two tests would be written up in a Journal article, submitted for peer review, then published.

When Dr. Samadpour tells us about his test and procedures we, of course, get excited about the implications for the industry. But without a peer-reviewed assessment, we really don’t know if the things that Dr. Samadpour cites are accurate or not.

This has nothing to do with Dr. Samadpour’s ethics. We have no reason to think he is anything but scrupulously honest. However, the peer review process is designed not just to catch dishonesty but flaws in study design, insufficiency of data and other problems.

2) SEMANTICS

When we read a comment such as this, “We’re getting zero false positives. In doing these rapid tests, you get a tremendous mixture of bacterias that can give you false positives. Everyone else calls this presumptive positives. We call this stage initial reactive,” we wonder if there isn’t a touch of gilding the lily here.

Now if the product is being held, and that is what Dr. Samadpour recommends, then it doesn’t matter what it is called. But in terms of the test’s ability to avoid “presumptive positives,” we are not sure this is a solution as much as a redefinition of the problem.

3) STATISTICS

One of the really big questions on the efficacy of testing is the question of whether the testing program can legitimately limit the scope of recalls. In the LA Times article, here is the way they describe the Natural Selection Foods program:

A worker named Sonia, wearing sanitized gloves, a hard hat and a face mask, rolls a cart to the towering stacks of pallets. With forceps, she randomly pulls out three to five arugula leaves at a time. For each 1,600-pound lot of greens, Sonia plucks 60 of these “grab” samples, about 3 ounces total, then drops the leaves into a plastic bag labeled with a bar code identifying the farm and the date harvested.

Since the outbreak, Sonia and other Earthbound employees have extracted several million samples from the 40 truckloads that arrive daily, carrying enough salad greens to fill more than half a million bags every day. Each sealed bag is delivered to a trailer, where microbiologists are waiting to look for two strains of E. coli — including O157:H7, the virulent one that caused last year’s illnesses — and salmonella.

A lab worker adds nutrients and heats the leaves to 109 degrees for eight hours. Then he inserts a pink stick — a lateral flow device that resembles a drugstore pregnancy test — while another technician extracts DNA.

If those dual tests are positive, the lot is held four hours for two more sophisticated genetic tests that confirm the results.

Once cleared, greens are moved onto conveyor belts, where they are trimmed and triple-washed in chlorinated rinses. (After the outbreak, Earthbound added a third wash.) Then they are zapped with lasers to detect foreign objects. At the end of the line, packaged greens are held 12 hours and tested again.

Once a week, on average, a load of raw greens from a farm fails the tests.

Since the test-and-hold program began last October, 58 out of about 76,000 lots entering Earthbound’s plants in San Juan Bautista and Yuma, Ariz., have tested positive for pathogens, a rate of 0.0008%. That amounts to about 93,000 pounds of greens destroyed out of about 122 million pounds that growers sent to Earthbound in the last 10 1/2 months. They came from many farms — most in the Salinas area, but also in Arizona and the Coachella Valley. Investigators were dispatched to the fields the next day, but the sources remain a mystery.

Tests for finished products were added in February, and so far no packaged greens have failed. But odds are, pathogens will soon be found after processing. Chlorinated washes can reduce bacteria counts but not kill them all

Samadpour predicts that four of Earthbound’s finished lots, nearly 4 tons, will test positive every year, most often in summer. If — or when, Samadpour says — that happens, Earthbound will dump it all in the trash, shut down the line and re-sanitize the plant.

The question is this: If a positive is found on finished product, what product, precisely, should a company “dump in the trash”? Just the 1,600 pounds in the lot?

Many food safety experts say this would be arbitrary. That the fact that test on the previous 1,600 pounds might have come back negative proves nothing as the testing is not frequent enough to guarantee the absence of a pathogen.

Several experts told us they think the FDA will ultimately prohibit this type of effort as an artificial attempt to limit the impact of a pathogen — that the FDA will insist on dumping all of the product since the plant was last sanitized. Others mentioned looking for “relevant factors,” such as other product from the same field.

The resolution of this issue is crucial because even under a “test and hold” program, once a lot is “cleared,” it gets shipped. If that lot will still get recalled because another lot comes back positive, then “test and hold” won’t prevent recalls.

In other words, the testing program doesn’t prove things are safe. As Dr. Samadpour states:

“These are lot acceptance programs. You set up standards for accepting this lot. Before you send this raw lot, it has to pass through your microbiological criteria for no salmonella, E. coli O157, or lysteria. Anything that doesn’t pass is rejected. You’ll have a lot of negatives. All you’re saying is they met your criteria, not that they are free of pathogens. Now you accept the raw material and put it through the wash system, and now you are testing final product with microbiological criteria, using the sensitivity of your test, it must meet your criteria or it’s rejected.”

So a company can elect to set any criteria it chooses but once a food safety problem is identified, the solution, typically a recall, has to be acceptable to the FDA. We need some guidance here on how the FDA will view these testing programs.

If the FDA’s opinion is that if a company tests every 400 boxes then that is an acceptable lot to recall or not release, that will significantly increase the argument for testing. If the FDA says that it doesn’t respect such limits and that a positive means the processor should recall everything from the previous sanitation of the plant, this will lessen the argument for testing.

4) CONFLICT OF INTEREST

Part of the problem in resolving this issue is that, although in media reports Dr. Samadpour is often referred to as a consultant and articles will say things implying he is acting in a consulting capacity — for example, the LA Times article explains that “Samadpour advised Sweat and Daniels to immediately begin checking all greens for pathogens, which no one in the industry was doing” — our discussion with Dr. Samadpour indicates that he has made his company, consultant, test developer, laboratory and more all in one.

Although this doesn’t mean his advice is incorrect, there is something sort of disconcerting about a situation like this. One would prefer that one’s consultant didn’t have a financial interest in one outcome over another. If Dr. Samadpour recommends spending $3 million dollars a year leasing land to increase buffer zones, he gets nothing. If someone uses his company to run a testing program, he could have a large continuing income.

We don’t want to make too much of this. We have no reason to believe he hasn’t always given his best advice, but it strikes us as a consideration for many people. We suspect his advice to the trade would be seen as more influential if people did not see him as having a financial stake in the outcome.

******

To some extent the advocates of testing attack a straw man in attacking those who “oppose testing.” In speaking with food safety experts and industry leaders, we can’t find anyone who “opposes” testing. If a company wants to test, raw materials or finished product, more power to it.

The problem really comes in marketing the testing program.

Inevitably in going out to the industry and promoting testing, one is implying that the tests “prove” something. Much as to use Dr. Samadpour’s analogy, we assume that someone who passes a French test at a University actually learned some French.

The fear, and it strikes us as a reasonable one, is that the promotion of testing will bias industry efforts in favor of testing — at the expensive of other efforts.

In other words, if Processor One doubles its buffer zones, triples its water testing, sanitizes the plant twice as frequently, has its growers put traps in twice as often and Processor Two does none of this but tests, the market, composed of layman produce buyers, not microbiologists, will assume the testing provides a better security on food safety and go that route.

Dr. Samadpour makes a strong argument when he is asked whether an extra $3 million should be spent on testing or other food safety systems, he replies: “Look at your budget and proportion it out. You have to continuously improve, validate and verify systems. It’s silly to put all of it in A and never measure whether A is useful.”

We are still learning a lot here and testing is not only desirable but, in fact, required in order to evaluate the effectiveness of various food safety measures. If a processor were to require its growers to double their buffer zones, what effect would that have on pathogens in produce? Only by testing can we see if we are doing effective things.

We also take to heart Dr. Samadpour’s admonition about retail-driven testing. Too many processors that do not have comprehensive testing programs have implemented testing on some finished product to suit a particular customer, such as Jack-in-the-Box or Costco. This seems likely to cause problems down the road. As Dr. Samadpour explains:

For me personally, although people talk about test methods as the issue, once you use a reputable, ISO-certified lab, the biggest challenge and problem that we have as an industry is the issue of proper design and execution of sampling plans. An improperly designed sampling plan will generate negative results with very little confidence, and once it finds a positive sample, even if the test lot is under control, may result in massive recalls.We have to make sure that a sampling plan will deliver what it was intended, and will not have unintended consequences.

Q: What do you mean exactly?

A: For instance, when a company designs and implements a testing program, the sampling plan should have statistical justification to ensure an appropriate confidence in negative test results. It must also be prepared to deal with all consequences of having a positive test result. If the testing program is poorly designed, the company could end up doing a massive unnecessary recall.

For instance, Customer A requires finished goods testing.If all you do is test finished product, you may test positive. Let’s say there has been at least one confirmed finished goods positive test by another laboratory. It then can implicate everything you co-produced, plus everything harvested from the same fields/ranches (which were not tested through a field or a raw material testing program). The implicated fields may implicate other processors (if they don’t have a defensible testing program. It can mushroom and get out of hand really fast. People have to be really careful.

My advice is that if you are not going to have a properly designed and comprehensive verification/testing program, you shouldn’t have one at all. If you are testing final product, you have to test raw product coming into the processing plant, and the fields from where you harvest.

Put another way, random testing done just because it gets a customer is going to cause enormous recalls every time there is a positive test.

So, after all this, where do we come out?

1. Dr. Samadpour is obviously very intelligent and knowledgeable. We as an industry and any processor as a company would be foolish not to pay attention to what he says. He has obviously thought about food safety for a long time.
2. At the same time, most of his experience seems to be in meat, and produce is not meat. Tests that may not be “new” can be “new in produce,” which means we don’t have the data history to judge some of these things.
3. His involvement in so many aspects of the business inevitably raises questions, justified or not, about conflicts of interest. Many a produce executive wouldn’t hire a financial advisor who gets paid more by recommending certain funds as opposed to other ones. By the same standard, ideally, one would like a food safety consultant who doesn’t gain or lose by the decision to do testing as opposed to increase buffer zones.
4. The problem is not testing per se; it is the marketing of testing as part of a food safety program as opposed to quietly doing testing to verify everything else works.
5. Dr. Samadpour offers the industry important guidance in saying that testing is essential to verify we are investing food safety dollars in the right places.
6. He also correctly identifies customer-driven testing without a comprehensive program as likely to cause real problems.
7. A key piece of information is missing: How does the FDA feel about the use of testing to limit the scope of a recall?
8. The industry could benefit by additional data, and Dr. Samadpour should be encouraged to write and publish in a respected peer-reviewed journal an article on his test and methodology so that the science behind what he is doing is transparent and confirmed by independent scientists.

Basically, there are two ways that the industry can go: Fresh Express says it is so confident of its food safety systems that no such program is necessary. Companies can certainly go this route — and pay the consequences if they are incorrect about their own systems.

Alternatively, we can move to a “test and hold” system in which testing verifies, before the product leaves the plant, that the product is safe in a statistical sense. Once again, if the underlying food safety isn’t strong enough or the testing is not frequent enough or performed properly, companies have to be prepared to pay for that error.

As an industry, we will know soon enough which system works better.

What we have to stop is the “test and ship” programs that are basically guarantees of public recalls.

This is a tough and important industry issue, and we thank Dr. Samadpour for being willing to share his perspective with the trade. He deserves commendation for being willing to speak out on such a controversial issue.