replication studies in economics
TRANSCRIPT
IP Infringement at Universities:
Replication Studies in Economics
Altug YalcintasAnkara University
Economics as the queen of social sciences
“Economics is the queen of social sciences”*
* Paul Samuelson 1976 [1948]. Economics. NY: MacGraw-Hill: 6
Paul Samuelson (MIT)1970 “Nobel Prize” in Economics
(The first American to win the prize)
Kenneth Rogoff• Harvard University, Professor of economics• PhD (MIT, 1980), BA & MA (Yale, 1975)• Chief economist at IMF (2001-2003)• Elect member of National Academy of Sciences
• Macroeconomics, financial crises, independence of central banks
• Foundations of International Macroeconomics (1996, with Maurice Obstfeld)
• This Time is Different (2009, with Carmen Reinhart)• The Curse of Cash (2016)
Carmen M. Reinhart• Harvard University, Professor of economics• PhD (1988, Columbia), BA (1980, Florida International)• Senior economist at IMF (1988-1996)• Chief economist at Bear Stearns (1980s)
• Macroeconomics, international finance
• This Time is Different (2009, with Carmen Reinhart)• “The Twin Crisis” AER (1999, with G. L. Kamisnky)
Harvard University, Department of Economics• Robert Barro• Martin Feldstein• Stephan Marglin• Oliver Hart• Gregory Mankiw• Amartya Sen• …• In memoriam: John Kenneth Galbraith, Zvi Grilishes, David Landes• Home to the Quarterly Journal of Economics (est. 1886)
* First published as an NBER in 2013.
Carmen M. Reinhart and Kenneth Rogoff. 2010a. “Growth in a Time of Debt” AER: Papers and Proceedings 100 (Mayıs): 573-576
• 44 countries, 200 years of data• “Debt intolerance ceiling”: Public debts > 90% of GDP economic
growth diminishes by half• Result: public debt (health, education, and social security) should be
lowered• The significance of the article: the 2008 Financial Crisis and the
Austerity Measures (as well as the high scores of citations it has gotten)
Carmen M. Reinhart and Kenneth Rogoff. 2010a. “Growth in a Time of Debt” AER: Papers and Proceedings 100 (Mayıs): 573-576Cited in / by:
• Republican Parts in the US after the 2008 Financial Crisis “The Path to Prosperity” (“Paul Ryan Budget”) (2013)• Olli Rehn, EU Commissioner of Economic Affairs, ILO meeting (2013)• George Osborne (2010)
First published as a PERI Working Paper in April 2013.
Thomas Herndon, Michael Ash, Robert Pollin. 2013. “Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff” Cambridge J of Economics 38 (2): 257-279
• Fabrication and falsification of data: Errors in codes and unconventional statistical techniques• Exclusion of data from the following countries in the Excel Sheet (Post-
WWII): Australia, Belgium, Austria, Canada, New Zeeland• Miscalculations in the model
• Result: Public debt > 90% GDP economic growth: %2.2
The 2008 Financial Crisis and the Austerity Measures • Financial measures in various countries after the crisis• “Teğet geçen kriz” in Turkey• Arab Springs, the Gezi Protests, Crisis in Greece, Brexit, Trump etc.
• Two sources:(1) Inside Job. 2010, Yönetmen: Charles Ferguson(2) Barry Eichengreen. Aynalı Salon: Büyük Bunalım, Büyük Durgunluk ve Tarihin
Yanlış Kullanımı (2016, Çev.: A. C. Balcı, Efil Yay.)(3) Altug Yalcintas. 2016. Creativity and Humour in Occupy Movements:
Intellectual Disobedience and Beyond (London: Palgrave)
The 2008 Financial Crisis: The Crisis of the Global Economy and the Crisis in EconomicsRR accept their mistakes (NY Times, 17 April 2013)
However:• Data and codes are not submitted and disclosed• No process of peer review (and editorial review)• American Economic Review reject the article by Herndon et al.• AER does not retract the article
Replication StudiesTo test the validity of previously published article with the original (or updated) data and algorithm by “rerunning” the model
• The problem of disclosure: Making the data and the algorithm freely avialible to the reviewers, editors, and readers• The problem of irreplicability: Lost data, licenced data, real time data, and the
algorithm that are able to be rerun• The problem of originality: Absence of incentives and rewards for doing
research in replication• The problem of prestige: The difficult situation of the authors of replication
(Herndon presently at Loyola Marymount University)
Replication Studies in Economics• Replication Wiki (Jan Höffler):http://replication.uni-goettingen.de/wiki/index.php/Main_Page • Repliction Network (Maren Duvendack)https://replicationnetwork.com/ • Econ Journal Watch: Scholarly Comments on Academic Economicshttps://econjwatch.org/
The biggest replication project in economics Journal of Money, Credit, and Banking Project, 1982-1984Previously mentioned in Econometrica (1933), J of Political Economy (1975)
Replication Studies in EconomicsAmerican Economic Association, “Data Availability Project” (2010):https://www.aeaweb.org/journals/policies/data-availability-policy • Data of published article in AER, JEL, JEP at the website of Americal
Economic Association• Other journals with similar projects of data availability (selected list):Public Finance Review, Empirical Economics J of Human Resources, American J of Business, Econometrica, Economic Journal, R of Economics and Statistics and many other…
The CV of K. Rogoff, 25 October 2016: http://scholar.harvard.edu/files/rogoff/files/cv_rogoff_102516.pdf [Accessed March 2017]
Replication Studies in EconomicsHow common?
ASSA 2017 Meetings in Chicago:• Replication and Ethics in Economics: Thirty Years After Dewald, Thursby and
Anderson: Deirdre McCloskey, Maren Duvendack, Jan Höffler, Richard Anderson, Bruce McCullough vd.• Replication in Microeconomics: Daniel Hamermesh, Muriel Niederle vd.
Source: Yalcintas. 2016. Intellectual Path Dependence in Economics: Why Economists Do Not Reject Refuted Theories (Routledge): 57-70.
Replication Studies in EconomicsHow common?• 50 economics journals
since 1960• 188 articles of
replication
Source: Duvendack, Palmer-Jones, and Reed. 2017. “What is Meant by ‘Replication’ and Why Does It Encounter Resistance in Economics” Conference Paper Presented at the 2017 ASSA Meeting, Chicago.
Replication Studies in EconomicsHow common?Chang and Li. 2015. “Is Economics Research Replicable?” Conference Paper Presented at the 2017 ASSA Meeting, Chicago.
• 67 articles of macroeconomics in 13 prestigious journals, 2008-2013• 22 articles replicated without contacting with the authors• 29 articles replicated after contacting with the authors• 16 articles not replicable (“confidential data” in 6 articles)
Replication Studies in EconomicsHow common?Duvendack, Palmer-Jones, and Reed. 2015. “Replications in Economics: A Progress Report” Econ Journal Watch 12 (2): 164-186.
• 333 economics journals in the Web of Science, 162 replications, 1977-2014• 2 out of 3 cannot be verified• 80% of article with serious errors in models
So what?
• Replication crisis or replication failure• Intentional manipulations unintentional mistakes• The problem of irreversibility• The problems of the philosophy of science prior to the writings of
Hegel, Compte, and Popper• Research ethics in economics
So what?Some of the problems of research misconduct in economics:
• Replication failure• Conflicts of interest• Underrepresentation (of women, of unorthodox methodologies etc.)• Plagiarism• Processes of peer reviewing
Economics as the queen of social sciences
“Economics is the queen of social sciences”*
* Paul Samuelson 1976 [1948]. Economics. NY: MacGraw-Hill: 6
Paul Samuelson (MIT)1970 “Nobel Prize” in Economics
(The first American to win the prize)