2010 saw some interesting articles lately calling into question some of the most basic assumptions regarding the scientific method. In March there was an article by Tom Siegfried which argued that “the ‘scientific method’ of testing hypotheses by statistical analysis stands on a flimsy foundation.” Of course, the problem may not be so much with the method, but with the application. Siegfried’s point is that
Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous, and tests of medical dangers or treatments are often contradictory and confusing.
I’m no statistician, so I’ll let the more mathematically literate evaluate the claims in that article. I link to it because it resonates with what my former roommate (and frequent commentator on Savage Minds) once told me. He said that biological anthropologists frequently misunderstand the results of computer programs which produce genetic trees because they don’t properly grasp the underlying math. Some people argue that a similar problem nearly brought down the world economy.
Even when the science is done right, there are some serious problems that need to be addressed. When research isn’t published in fake peer review journals or ghost written by pharmaceutical companies, there are still inherent biases against publishing “negative results.” And even when everything is done right, strong empirical results are often impossible to replicate:
But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable.
But as the title should make clear, I link to these results not to debunk science but to praise it. Michael Bérubé has an intriguing review of the “science wars” in which he argues that both scientists and their critics have a shared interest in trying to move beyond the impasse of the nineties in order to face the twin threats of those who deny evolution (for religious reasons) and those who deny global warming (for economic reasons):
Fifteen years ago, it seemed to me that the Sokal Hoax was making that kind of deal impossible, deepening the “two cultures” divide and further estranging humanists from scientists. Now, I think it may have helped set the terms for an eventual rapprochement, leading both humanists and scientists to realize that the shared enemies of their enterprises are the religious fundamentalists who reject all knowledge that challenges their faith and the free-market fundamentalists whose policies will surely scorch the earth. On my side, perhaps humanists are beginning to realize that there is a project even more vital than that of the relentless critique of everything existing, a project to which they can contribute as much as any scientist–the project of making the world a more humane and livable place.
I sincerely hope that this is the case. What I like about science is that it is not afraid to ask tough questions. There is no reason to think that the scientific method can’t learn from all of the problems listed above and find ways to make scientific results even more robust than they were before. But I don’t think that science can do this on its own. These are also political problems, social problems, institutional problems, psychological problems, etc. and to find ways to make science better scientists will need to work together with anthropologists and others to find ways to overcome these problems. (See James Clifford’s talk on “The Greater Humanities.”) I like science because I think scientists understand this in the same way that the best economists understand that economics alone isn’t enough to solve economic problems.