An Urban's Rural View

"To err is human, to forgive divine," the poet said. He didn't mention "to correct errors," but doing that is almost as important as forgiving, and often easier to do.

A case in point is the correction of a coding error in an economic study that has captured attention of late. In 2010 two Harvard economists miscrunched the numbers in their analysis of how much governments can borrow without retarding economic growth. The miscalculation went unremarked until a few weeks ago, when three University of Massachusetts economists noticed the spreadsheet mistake and exposed it.

Because debt hawks had cited the study in support of their argument for balancing the federal budget, debt doves have had fun trumpeting its debunking. The lesson many commentators have taken away from the correction is that the government can safely borrow more than the Harvard economists thought.

The real lesson, I'd argue, is as applicable to the natural sciences as to the social, as relevant to studies of genetic engineering as to studies of gross domestic product. For agriculture and indeed most of us the incident is more important for what it says about scholarly blundering and how it does or doesn't get fixed.

What the incident teaches, say Betsey Stevenson and Justin Wolfers in an excellent column on Bloomberg (http://tiny.cc/…), is that scholars in every field must do a better job of checking each other's work. They need to try to replicate each other's results.

They don't do that as often as they should because it's tedious, unrewarding work. Every minute the scholar spends redoing experiments and rerunning numbers is a minute she could have spent on her own research. The problem is that if every scholar chooses the path of greater fun and glory and no one replicates others' work, errors go uncorrected, misleading science and society down dangerous paths.

In this case, the important thing is not so much that an error was made as that scholars did try to replicate the results. Thomas Herndon, Michael Ash and Robert Pollin reviewed the initial study and caught the coding error Carmen Reinhart and Kenneth Rogoff had made.

Hats off to the U-Mass three. We need more like them.

The blundering Harvard economists, Reinhart and Rogoff, actually deserve credit in the correction, too. They posted their data for all the world to see on the Internet and shared their spreadsheets with potential critics. As Stevenson and Wolfers point out, this is unusual.

"Even getting authors to share the data needed to replicate a study can be a challenge," they write. "In one study, a team of determined replicators tried to examine 54 articles published in a leading macroeconomics journal in the 1980s. Many authors never responded to repeated requests for programs and data. Others refused or sent raw and often unintelligible computer files. When replication was possible, it frequently uncovered errors. All told, the team was able to replicate the findings exactly in only two articles."

Economics isn't the only problem field. Psychology, Stevenson and Wolfers report, went through a wave of erroneous and in some case fraudulent studies. As a result, dozens of psychologists have signed on to something called the Reproducibility Project, which systematically works to replicate findings.

It seems safe to assume the natural sciences aren't exempt from undiscovered errors. And peer review, while important, isn't a solution. When scholars peer review a colleague's paper, they don't necessarily redo his experiments or rerun his numbers. It's entirely possible for two studies arriving at opposite conclusions to both be peer reviewed.

Everyone makes mistakes. But to win the trust of policy makers and the public, scholars must do a better job of correcting them.

Urban Lehner can be reached at urbanity@hotmail.com

(AG)

© Copyright 2013 DTN/The Progressive Farmer. All rights reserved.