New “Study” Recommends to Take Nutrition Studies with a Grain of Salt

Open any daily paper—or your Facebook feed—and you’re likely to come upon a health study dressed up like the news:

Coffee May Protect Liver From Booze, Study Finds
NBC News

This Kind of Fat Won’t Make You Gain Weight
Time Magazine

Or try this one on for size:

Lettuce Produces More Greenhouse Gas Emissions Than Bacon Does
Scientific American

photo by cookbookman17, via Flickr

photo by cookbookman17, via Flickr

What do all three headlines have in common? They boil down a scientific study or studies into click-bait. They take nuanced, lengthy, caveat-ridden academic work and squeeze it into a tweetable sexy swimsuit. And that means that, more often than not, the headlines we read about nutrition, food, health, and the environment over-simplify and even misrepresent the information.

So how can someone who cares—and really wants to drink both an extra cup of coffee and an extra beer while protecting their liver—find out what’s real from what’s superficial? Here are four steps to get you closer to the truth.

1. Go to the source.

Don’t take any headlines at face value. The sad truth is that in today’s media landscape, where many reporters have to churn out a lot of content, they lack the time to interrogate the studies themselves—or, often, to go much further than the press release.

Dan Nosowitz, writing for Modern Farmer, does a great job explaining why the lettuce-bacon polemic was such a pile of methane-rich dung. The reporting on the study, which gave bacon-lovers a moral upper-hand and the gift of rationalization, got great play in the media, but most headlines directly aped the press release and failed to account for the reality of what the study entailed. (Note to self: Believe nothing in the news that includes the word bacon. Bacon is the click-bait king of kings!)

If the topic at hand is something you earnestly care about, whether its chocolate or omega fatty acids or cancer treatment, put in the extra time and find out what study it stems from, what questions they asked, and how they performed their research. For a great article on the different ways nutritionists go about gathering data (it really is fascinating!) check out You Can’t Trust What You Read About Nutrition, by Christie Aschwanden at FiveThirtyEight.

2. Follow the money.

Studies do not all rise from a nameless, unbiased pot of gold. In fact, research is sorely underfunded in this country, and well-meaning researchers often have to turn to industry. It’s murky territory, as lauded nutritionist Marion Nestle points out, who, like a baseball statistician, has been keeping score of how many industry-funded studies favor the funder. (It’s the majority.) Full disclosure: Organic Valley actively invests in organic research. With the prevalence of chemical- and GMO-driven agriculture today, if the organic industry didn’t fund studies about organic food and farming, who would? And in a world where data trumps common sense, it’s important to be at the table.

You should still find out who funded the study at hand. That will put you in the position to decide how much merit to warrant the information. Funding bias is real, and while it doesn’t mean everything is without merit—there are responsible researchers and funders out there—it should factor into your opinion.

3. Find out whether the study aggregates many studies or stands alone.

The most reliable nutrition advice arises from what are called “meta-studies.” These big boys are created by researchers who look at a wide collection of studies that approach a topic from different angles and find the commonalities among them. It’s the equivalent of using photos from multiple angles to create a 3D image rather than making assumptions from a flat picture.

Find out if the research you’re reading stems from a meta-study or stands alone. If it does stand alone, and if you’re feeling ambitious, look for other studies that take a different perspective.

4. Never make drastic life changes based on a headline. Seriously.

Why would I ever do that? you’re thinking. But maybe you did jump on the agave train because of its low-glycemic index, only to discover that it may not be so good for you after all. (It is still sugar.) Real academic work around nutrition is sticky and takes time. It’s under-funded and has to muck around in the complex world of our individual guts, memories, and lives. (Do you remember what you ate three days ago? How big your portion of ribs was? Those kinds of squishy questions are sometimes the basis for nutrition research.)

. . .

Here is my nutrition advice—to be taken with a grain of salt:

Remember that your body is a unique ecosystem that gives you real-time feedback. Listen to it. Eat a lot of organic, seasonal vegetables—the kind you have to wash yourself. Listen to it some more. Feel good about eating a piece of chocolate because it is delicious and may have some benefits to boot. Don’t eat a whole chocolate bar because you think it will improve your memory. That’s hooey.