How to stop Excel errors driving austerity economics

A global row has erupted over errors in research used to bolster the case for austerity. Open data might have spared us the pain, says Velichka Dimitrova of the Open Knowledge Foundation

Another economics storm has hit the headlines but this time it is not the grand global gambles of the banks that matter ? it is the minutiae of Excel spreadsheet entries. At the centre of it are two professors from prestigious Harvard University, Carmen Reinhart and Kenneth Rogoff.

In 2010 they wrote a paper, "Growth in a time of debt", which concluded that economic growth slows when the ratio of a nation's public debt to its gross domestic product (GDP) exceeds 90 per cent. It goes to the heart of the debate over austerity policies.

Reinhart and Rogoff's results were published in the Papers and Proceedings of one of the most prestigious economics journals ? The American Economic Review ? and had a powerful resonance in a period of serious economic and public policy turmoil. Some argue that the paper prompted governments around the world to slash spending in the hope of stimulating economic growth. For example, US congressman Paul Ryan and the vice president of the European Commission Olli Rehn both cited the 90 per cent threshold level as a benchmark in their policies.

This month Thomas Herndon, Michael Ash and Robert Pollin from the less prestigious University of Massachusetts Amherst tried to replicate the results of Reinhart and Rogoff but instead discovered errors that, when corrected, create a very different outcome.

Debt redeemed

Most widely reported among their three grounds of criticism is a single spreadsheet mistake that meant five countries were excluded from the sample used by Reinhart and Rogoff. The University of Massachusetts team say that, taken together, the problems they have identified in the original paper resulted in significant error in the conclusion that a large public debt cripples growth.

After correcting the mistakes, the researchers found that countries with a lot of public debt suffer only "modestly diminished" average GDP growth rates, rather than the stronger effect shown in the 2010 paper. Their analysis shows that when countries owe about 90 per cent of their GDP, they do indeed tend to have slower economic growth ? but the trend does not follow the data at all well. This means that if a country's public debt exceeds the threshold, its economic performance will not necessarily deteriorate. It may even get better.

The debate continues. Reinhart and Rogoff have admitted they made mistakes but say their core point remains valid. Herndon has responded that his results do not show that public-debt-to-GDP ratio above a certain threshold has a significant impact on growth.

So how did such errors go unnoticed for so long? The American Economic Review has an excellent policy on data availability, implemented in 2004 and a model for other economics journals . However, the Review does not apply it to Papers and Proceedings, an edition of the journal published each May to reflect the annual meetings of the American Economic Association and which is not subject to its usual rigorous peer review. As a result, it did not make the data and regression code of the Reinhart and Rogoff study available to other researchers.

Even if peer review did not apply, does this really exempt researchers publishing in the May issue from sharing their data and code? It is the policy of The American Economic Review "to publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication". Note that there is no differentiation between users of the data as peer-reviewers or as researchers who want to replicate the results: data and code should be available to any researcher.

Coding errors happen, but the greater problem was this failure to allow others to review and replicate the results by making the data openly available. If this had been done upon publication in 2010 ? or if the idea of open data was universally applied ? it might not have taken three years to prove these results wrong, and the odd case of Reinhart and Rogoff's slip of the keyboard wouldn't be making such huge waves around the world.

Profile

Velichka Dimitrova is project coordinator of Open Economics at the Open Knowledge Foundation, which advocates the release of datasets and code along with published academic articles. It defines open data as "a piece of data or content? free for anyone to use, reuse, and redistribute ? subject only, at most, to the requirement to attribute and/or share alike".

This article is based on a blog first posted on the London School of Economics website

If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.

Have your say

Only subscribers may leave comments on this article. Please log in.

Only personal subscribers may leave comments on this article

Subscribe now to comment.

All comments should respect the New Scientist House Rules. If you think a particular comment breaks these rules then please use the "Report" link in that comment to report it to us.

If you are having a technical problem posting a comment, please contact technical support.

Source: http://feeds.newscientist.com/c/749/f/10897/s/2b33f636/l/0L0Snewscientist0N0Carticle0Cdn234480Ehow0Eto0Estop0Eexcel0Eerrors0Edriving0Eausterity0Eeconomics0Bhtml0Dcmpid0FRSS0QNSNS0Q20A120EGLOBAL0Qonline0Enews/story01.htm

NIT Tournament clay matthews Ncaa Tournament 2013 2013 NCAA Bracket leprechaun ufc Chris Cline