The New Clarion

The New Clarion header image 2

The Value of Null Findings

August 16th, 2009 by Mike N · 5 Comments · Uncategorized

Sandy Szwarc at JunkfoodScience has an in-depth look at a health science topic. Although it’s titled “The Myth About Unhealthy Belly Fat” the article’s theme is, ‘the importance of null findings’, and properly laments the fact that the media seldom reports them. This is very true and also very important. That’s because:

“Null findings enable true scientists to know they’re looking in the wrong direction and that it’s time to go back to the drawing board and develop a different hypothesis. They also enable us to stop needlessly worrying about something that doesn’t matter.”

Also, many if not most studies that purport to show a health problem actually turn out to be false and these revelations are often not published as well. Just as knowing that something is a threat to health and life is important, so is knowing when something is not a significant threat, especially when alleged professionals are telling us it is a threat. However, we also know that the media often prefers hype and sensationalism vs boring news about non-threats. So the media has a built in bias that is heavily weighted in favor of fear-hype while a study showing that last month’s health scare is actually false will tend not to get printed. This will mislead thousands if not millions of people. As the saying goes ‘you can’t unring an alarm bell.’

But this situation has other consequences. It creates a fearful citizenry eager to donate their money to NGOs (Non Governmental Organizations, mostly non-profit foundations) who promise to do research (more statistical studies) to find ways to protect them from all the alleged hazards of a modern, technological society.

It sometimes works like this: NGOs fund studies that try to determine what is bad for us. When a study is completed a press release is announced. The media picks up the release and reports the findings which are usually that X (say, fast foods) is bad for us. Politicians then stand on the floor of their chambers, news reports and studies in hand, announcing that congress must address this threat to public health by passing legislation more heavily regulating the use of X. They then stand before us proclaiming how good they are at protecting us from all the harmful Xs out there and therefore we should reelect them. That is what we see.

What we don’t readily see is that the funding NGO has been getting large donations from companies that produce products Y and Z (say, treadmills and diet books). Also mostly out of view is the donations by companies Y and Z to the politicians’ reelection campaigns. And even further out of sight is the fact that no one looked at the study, not the reporter, the editor nor the politician, to see if the study had any statistical significance or was a null finding being passed off as having significance. This last happens often.

We have to educate ourselves to what is valid science and what is pseudo science because the press isn’t doing it. Pseudo science isn’t just coming from NGOs either. Government agencies fund lots of it. I recommend reading the entire article and visiting JunkfoodScience often. Her blogroll is a good source for others who actually read these so called scientific studies and then examines them for our benefit.

5 Comments so far ↓

  • Myrhaf

    Yaron Brooks says a big problem is that children are no longer taught science in school. Instead they are fed environmentalist propaganda in science classes. They wouldn’t know good science from bad, but they know they’re supposed to agree with environmentalism.

  • Mike

    This is a bigger deal than I think your article is crediting it for being. I was educated in a fairly science-respecting fashion (private school, taught that consensus is meaningless in comparison to replicatable experiment, etc) and they barely touched upon the concept of null findings. Even at the university level, who will maintain grant funding by publishing a report saying that all such-and-such costly research show that nothing is happening?

    In the field of law, there is ample precedent and no raised eyebrows when a judge recuses from a case or a lawyer cites a conflict of interest and withdraws from representation. These are cases of professionals doing their job properly by NOT proceeding into developmental work on a case (or the evaluation of a case by trial). A scientist who did this might well find themselves at an empty spigot on payday and on the outside looking in with regard to scholarly publication.

    Excellent post. Thank you for raising this issue and the “refresher course” you coupled with it.

  • Mike N

    Myrhaf:
    Mr. Brook is right. Science is not taught in any kind of historical context so the kids don’t get to see how the scientists of the past arrived at their discoveries. The poor kids are never introduced to scientific methodology. Some teachers even believe that successful scientists just got lucky.

    Mike:
    “Even at the university level, who will maintain grant funding by publishing a report saying that all such-and-such costly research show that nothing is happening?”

    This is proof that the granting agencies whether universities, NGOs or government bodies like the NIH or EPA etc. are not interested in the truth but are looking for a problem of some kind that will allow them to justify their existence by claiming the need to solve it. It also demonstrates once again that: “Government encouragement does not order men to believe that the false is true, it merely makes them indifferent to the issue of truth and falsehood.” Ayn Rand in her essay Establishing of an Establishment.

    Thanks for your experienced comment, and I agree that this issue needs more attention. I plan to post on it again.

  • salika_l

    Yaron Brooks says a big problem is that children are no longer taught science in school. Instead they are fed environmentalist propaganda in science classes. They wouldn’t know good science from bad, but they know they’re supposed to agree with environmentalism.

  • Mike N

    Salika:
    Mr. Brook is right. The sad state of today’s schools is such that education per se, teaching children how to think, is not happening. Teachers who themselves don’t know how to think cannot pass along to students a knowledge they don’t possess. That is why sirs Brook and Peikoff have both said that the most critical need today is for a rational education.