Faulty Studies Mean Everything You Know About Nutrition Is Wrong

Pizza is healthy. At least that's what I was told in kindergarten. After all, it has all the major food groups, which (at the time) included a big ol' pyramid base of bread. I (and the other six-year-olds, I imagine) took that nutrition advice to heart well into adulthood. Being an adult means you can have pizza anytime you want.

Before you scoff at my poorly-informed nutritional choices, you should probably recognize that you have a fair amount of food misinformation swimming around your own head. Do you buy low-fat cheese and skim milk, count calories, go on juice cleanses — all because you think science says you should? 

Well, science has told you some lies over the years.

If the science surrounding nutrition and diet seems, well, confusing and awful, that’s because it usually is. Think about it. One day saturated fat is bad, then it's fine? Fat was once the devil, but now sugar is? Wine both causes and prevents cancer?

In recent weeks, the science supporting many of our ideas of what makes up a healthy diet came under fire and fell apart. Nutrition research is rife with weak methodology, human error, and the biases of the scientists behind them. For instance, a major study that set out to see if alcohol consumption came with health benefits was cancelled after accepting tens of millions of dollars from the alcohol industry and hinting at favorable results.

Yes, articles about each incremental study are part of the problem. But the problem goes much deeper — the conclusions of the studies themselves are contradictory.

The U.S. Food and Drug Administration's (FDA) most recent guidelines deem eggs too high in cholesterol and fat to be considered healthy, even though government agencies still list them as components of a healthy breakfast. There were a handful of studies intended to determine whether there was any distinction between “good” or “bad” Calories, but they became moot when the researchers failed to include a control group and allowed people who wanted certain results to weigh in, according to WIRED. Now we can’t even say for sure whether ten Calories of chocolate are inherently better or worse than ten Calories of broccoli.

And earlier this month, the cornerstone study that linked the Mediterranean diet to lower heart disease rates was retracted, according to The New York Times. Even though the study, titled “Primary Prevention of Cardiovascular Disease with a Mediterranean Diet” or PREDIMED, was amended with a correction after it was initially published, its authors argue that they can still make the same health claims that they did before. Some critics, however, remain unconvinced.

All this begs the question: why is it so hard to scientifically determine whether a certain diet or food is actually good for people? Are we doomed to endure the same, confusing mess of experiments that tell us that coffee is healthy one day but causes cancer the next?

If we want to do better, we need to figure out a better way to control all of the tiny little extraneous details that can throw off nutrition studies.

“Nutrition research is a difficult task given all the extraneous variables that need to be controlled, therefore it’s important to take into account or control for as many of these variables as possible, i.e. anything that affects our diet, sleep, living situation, time of the year, activity levels, medical history,” Kelly Pritchett, a nutrition and exercise scientist at Central Washington University and spokesperson for the Academy of Nutrition and Dietetics, told Futurism. “I see this as a ‘control the controllables' concept,” she added, arguing that researchers need to consider and remove variability from as many aspects of a study participant’s life as possible.

Taking better care of all these little unpredictable variables that can throw off a study's conclusion, called confounding factors in academic literature, may have saved the Mediterranean diet study. Ideally, study participants would have been randomly assigned a diet; instead, as The New York Times reported, ten percent of all the study’s participants were not. Instead, all the members of a family and sometimes even entire villages were given the same diet. These inconsistent practices were never reported in the final writeup of the study.

But, you know, randomization is hard. Can you really expect scientists to stick to true randomization, ensuring that the only differences in health outcomes among participants are truly due to whether or not someone adhered to the Mediterranean diet? What happens when neighbors complain that they got less appetizing food than other people in their village? Can you expect the researchers to tell parents that they need to cook separate meals for their children, just for the sake of better science?

Yes. Yes you can. Because the person in charge of that portion of the study decided to take the easy way and let some people (who volunteered to participate in an experiment centered around a controlled diet) have the same food, the entire project had to be overhauled. All of its findings, on which many people worldwide based their eating habits, had to be called into question, perhaps thrown out altogether.

Just because the researchers claim that the Mediterranean diet is still linked to improved heart health after throwing out the compromised data, their research isn't any less suspect, and it certainly doesn’t mean we can’t do better in the future.

“I think we do need to question the methodology and how the results are displayed, not just this article, but every article we read,” said Pritchett.

In an ideal world, diet studies would be conducted in a sterile lab, where everything from caloric intake to exercise and sleep of each participant is under the complete control of the scientists conducting the experiment. But, of course, this isn’t so simple — nutrition research like the Mediterranean diet study takes years to tease out any health benefits or risks of a certain food. And each time you impose a new control, you take away a little bit of real-world applicability.

After all, the people who might actually apply the conclusions of that research won’t live in a lab. They’ll be out in the world, trying to balance their diets as they work, study, exercise, sleep, and play.

Even providing food for participants instead of leaving them to plan out their meals could take away some of their stress, which has long been scientifically linked to a slower metabolism and increased fat retention.

In that sense, researchers will need to find ways to match their experiments to those lifestyles to find data that might be meaningful for peoples’ lives. But that doesn’t mean they can throw best scientific practices out the window, as some did while studying the Mediterranean diet.

Fortunately, there are new efforts to formalize how nutrition studies are done. The National Institutes of Health (NIH) recently funded research teams at Indiana and Tufts Universities to come up with clinical guidelines for randomized, controlled nutrition studies, Connie Weaver, a nutrition scientist at Purdue University who serves as director of the Women’s Global Health Institute, told Futurism.

“Currently, guidelines exist for drug and device trials but not for nutrition studies. Hopefully this effort will lead to guidelines and training that will improve scientific quality of diet studies,” Weaver added.

She also speculated that perhaps researchers could establish some guidelines to salvage some of the findings of tainted studies so that the next wave of scientists doesn’t need to start from zero.

It’s not yet clear what these guidelines will look like, but they will likely mirror the best practices of other scientific fields, emphasizing the randomization and blinds that the scientific community has come to expect and value — at least to whatever extent these techniques are possible out in the field.

But in the meantime, scientists may need to keep acting as public watchdogs, watching out for faulty science (and honest mistakes). Weaver told Futurism that she attended a recent talk about the PREDIMED study at which the speaker argued that public retractions and corrections will help keep scientists honest. As long as scientists are willing to point out mistakes in public, laypeople stand a chance of staying informed about what does and doesn’t count as a healthy meal.

Until these new guidelines go into effect, much of the onus for healthy eating may fall where it always has: on individuals and their choices.

“I do think that the general public tends to focus on the headlines or the abstract rather than reading the full article itself,” says Pritchett. “I would tell the general public the same as I tell my students: read the discussion first and work your way backwards to the introduction.”

Basically, make sure to look at the actual evidence before jumping on board with any glib claims. For instance, people scrolling through Facebook or Twitter right now would likely encounter headlines saying that pasta doesn’t cause weight gain. But what's not as easy to see? Articles that take the time to mention that the study was funded by pasta companies. Probably a good caveat to take into account before changing your dinner plans.

Alternatively, there’s the opposite approach. Since we no longer have science to say what’s healthy or unhealthy, I'll just be eating pizza for breakfast every day and filling my cereal bowl with bourbon.