Every Economist will sooner or later have stumbled upon journal rankings. Rankings inform decisions on what to read first and where to publish preferably. Rankings determine careers and individual honours. One good thing about them is their vast number so that everyone can pick the one with the most appropriate criteria.
It has been argued that determining impact based on journal reputation is preferable to a mere citation count: Citations do not allow us to determine whether research is novel. On the other hand, determining impact by how often someone makes it to top journals is no good alternative, too. It discourages innovative work because non-innovative work has it much easier to get published high.
Packalen and Bhattacharya (2017) have therefore recently developed a “Neophilia Ranking”. The ranking criteria is what fraction of articles mention relatively new ideas.
Here I explore the idea a ranking Journals by the long-term impact: What fraction of articles are still cited 20 years later?
To do so I look at a selection of major Economics Journals and compute the share of their 1996 articles that are cited at least once in 2016. I take all journals rated at least “A” by Combes and Linnemer (2010) in their General Economics and six field-specific rankings. This yields a total of 54 journals. I am looking at the share of articles published in 1996 and cited in 2016, according to Scopus (Elsevier’s Citation Database). Of those, 11 did not exist in 1996 (or Scopus did not index this year). For example, the Review of Economic Dynamics, ranked 35 in Combes and Linnemer (2010)’s main ranking, was founded in 1998.
The remaining 43 journals published 2375 articles in total in 2016, ranging from 170 (Canadian Journal of Economics) to 10 (Brookings Papers of Economic Activity). Of these, more than the half, namely 1230, were still cited in 2016. The ranking is led by Journal of Political Economy, with 91.11% of its 45 publications in 1996 cited 20 years later.
The following table ranks all these journals by share of 1996 publications that are cited at least once in 2016 according to Scopus, which is listed column “Share >0 citations in 2016”. The column before is the conditional citation count: Given that the article is cited in 2016, how often has it been cited on average?
|Journal||Publications in 1996||Cited in 2016||Average cond. citation count in 2016||Share >0 citations in 2016 (in%)||Share >10 citations in 2016 (in%)||Share >50 citations in 2016 (in%)|
|Journal of Political Economy||45||41||8.95||91.11||24.44||0.00|
|Quarterly Journal of Economics||41||37||10.32||90.24||34.15||0.00|
|Journal of Economic Literature||18||16||16.38||88.89||55.56||0.00|
|Journal of Financial Economics||47||41||15.34||87.23||36.17||2.13|
|Review of Financial Studies||37||32||6.38||86.49||16.22||2.70|
|Journal of Economic Growth||21||18||11.67||85.71||19.05||0.00|
|Journal of Finance||69||59||14.63||85.51||34.78||2.90|
|Brookings Papers on Economic Activity||10||8||6.00||80.00||20.00||0.00|
|Journal of Economic Perspectives||47||37||6.08||78.72||14.89||0.00|
|American Economic Review||149||115||8.46||77.18||16.11||2.01|
|Review of Economic Studies||28||20||6.40||71.43||10.71||0.00|
|Review of Economics and Statistics||60||42||6.40||70.00||11.67||1.67|
|Journal of Financial Intermediation||15||10||1.90||66.67||0.00||0.00|
|Journal of Econometrics||98||65||9.52||66.33||14.29||4.08|
|Journal of Labor Economics||28||17||3.53||60.71||0.00||0.00|
|Journal of Public Economics||92||55||4.16||59.78||1.09||1.09|
|Journal of Banking and Finance||86||50||3.60||58.14||5.81||0.00|
|Journal of Development Economics||63||36||6.11||57.14||11.11||0.00|
|Journal of Monetary Economics||57||32||6.00||56.14||7.02||0.00|
|The Journal of Economic History||33||18||2.06||54.55||0.00||0.00|
|Journal of Economic Surveys||11||6||3.67||54.55||0.00||0.00|
|European Economic Review||111||59||5.49||53.15||9.01||0.00|
|Journal of International Economics||41||19||5.05||46.34||7.32||0.00|
|Journal of Money, Credit and Banking||53||24||5.29||45.28||5.66||0.00|
|Journal of Economic Theory||104||42||5.19||40.38||2.88||0.96|
|Oxford Review of Economic Policy||30||12||4.33||40.00||3.33||0.00|
|European Journal of Political Economy||52||20||2.75||38.46||1.92||0.00|
|Journal of Economic Dynamics and Control||77||27||2.48||35.06||0.00||0.00|
|Journal of International Money and Finance||52||15||2.87||28.85||0.00||0.00|
|Scandinavian Journal of Economics||35||10||3.60||28.57||5.71||0.00|
|Canadian Journal of Economics||170||37||2.08||21.76||0.59||0.00|
|Journal of Macroeconomics||41||8||3.50||19.51||2.44||0.00|
|Journal of Policy Modeling||27||5||1.80||18.52||0.00||0.00|
|National Institute Economic Review||27||3||1.33||11.11||0.00||0.00|
|Open Economies Review||22||2||1.50||9.09||0.00||0.00|
|North American Journal of Economics and Finance||18||0||0||0.00||0.00||0.00|
Interestingly, the journals known as the “top 5” journals (The American Economic Review, Econometrica, the Journal of Political Economy, The Review of Economic Studies, and the Quarterly Journal of Economics) are not top 5 in this list. The Journal of Economic Literature, for example, is ranked third and has the highest conditional average citation count of about 16 citations. It also has the highest share of articles when we define long-standing articles as having more than 5 citations. This mirrors findings presented at this year’s ASSA conference’s Panel Discussion on Publishing and Promotion in Economics: The Curse of the Top Five. If define a long-lasting impact article by being cited more than 50 times in the 20th year, then Econometrica comes out first.
Interesting questions arise: Do these journals attract publications with a long impact? Do these articles still attract citations because they got published where they are? Or were the editors in 1996 simply good at picking the ‘right’ articles with presumably long-lasting impact?
Though the questions are not easy to answer, I would like to shed light on the differences among the journals by showing how the average long-lasting article (i.e. the one that is still cited 20 years later) performs over time. The following plot traces the average number of citations in a given year by journal to all long-lasting articles of 1996 (hover over the lines to see the journal names and zoom in by drawing a rectangle with your mouse – might not work on mobile versions).
As a further robustness check, here is above list with citations 10 years later for publications from 2006. That is, how many of a journal’s 2006 publications have been cited at least once in 2016? Again, the last column gives the average number of citations in the 10th year. The list is compiled from 2012 articles from 48 journals (only 6 did not exist in 2006 or were not indexed).
|Journal||Publications in 2006||Cited in 2016||Average cond. citation count in 2016||Share >0 citations in 2016 (in%)||Share >10 citations in 2016 (in%)||Share >50 citations in 2016 (in%)|
|Quarterly Journal of Economics||40||39||15.90||97.50||52.50||5.00|
|Journal of Political Economy||38||37||9.57||97.37||34.21||0.00|
|Journal of Finance||88||84||18.35||95.45||51.14||7.95|
|Journal of Financial Economics||88||82||16.09||93.18||36.36||6.82|
|Journal of Economic Perspectives||48||44||15.18||91.67||31.25||4.17|
|Journal of Economic Growth||12||11||6.09||91.67||16.67||0.00|
|Journal of Labor Economics||28||25||11.12||89.29||28.57||3.57|
|Review of Financial Studies||43||38||11.58||88.37||30.23||2.33|
|American Economic Review||191||165||9.34||86.39||22.51||1.57|
|Review of Economic Studies||44||38||7.92||86.36||22.73||0.00|
|Review of Economics and Statistics||58||50||11.96||86.21||25.86||1.72|
|Journal of the European Economic Association||61||47||4.30||77.05||4.92||0.00|
|Journal of Economic Literature||20||15||11.27||75.00||35.00||0.00|
|Journal of Development Economics||67||49||7.04||73.13||10.45||1.49|
|Journal of Econometrics||130||92||6.84||70.77||12.31||0.77|
|Journal of Public Economics||110||77||5.32||70.00||10.91||0.00|
|Journal of Economic Theory||76||53||3.58||69.74||1.32||0.00|
|Oxford Review of Economic Policy||33||23||5.83||69.70||12.12||0.00|
|Journal of International Money and Finance||66||46||4.85||69.70||6.06||0.00|
|Journal of International Economics||65||45||5.89||69.23||13.85||0.00|
|Journal of Monetary Economics||107||73||3.93||68.22||6.54||0.00|
|Journal of Banking and Finance||172||117||5.94||68.02||8.14||1.16|
|Journal of Money, Credit and Banking||99||66||4.55||66.67||6.06||0.00|
|Review of Economic Dynamics||33||22||2.91||66.67||3.03||0.00|
|Journal of Economic Surveys||29||19||4.63||65.52||6.90||0.00|
|European Economic Review||94||60||4.65||63.83||7.45||0.00|
|Journal of Financial Intermediation||24||15||4.33||62.50||4.17||0.00|
|Scandinavian Journal of Economics||37||23||3.09||62.16||2.70||0.00|
|The Journal of Economic History||36||22||2.00||61.11||0.00||0.00|
|Journal of Financial Stability||14||8||5.25||57.14||7.14||0.00|
|European Journal of Political Economy||53||30||4.53||56.60||7.55||0.00|
|Canadian Journal of Economics||60||30||2.00||50.00||1.67||0.00|
|Journal of Economic Dynamics and Control||112||55||2.75||49.11||0.89||0.00|
|Journal of Macroeconomics||50||22||1.77||44.00||0.00||0.00|
|Journal of Policy Modeling||77||33||2.06||42.86||1.30||0.00|
|Open Economies Review||28||11||2.36||39.29||0.00||0.00|
|Brookings Papers on Economic Activity||13||5||3.60||38.46||0.00||0.00|
|North American Journal of Economics and Finance||25||6||5.33||24.00||4.00||0.00|
|National Institute Economic Review||57||3||1.33||5.26||0.00||0.00|
Comparing both tables, we see some dynamics happening here, with the top 5 journals being closer to the top and closer together than in the ranking of 20 years-later citations. This also holds for some major field journals, like the Journal of Finance, Journal of Economic Growth or Journal of Labor Economics.
- Combes, P.-P. and L. Linnemer (2010)‘Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics‘, HAL Working Paper 520325.
- Packalen, M. and J. Bhattacharya (2017), ‘Neophilia ranking of scientific journals’, Scientometrics 110(1), 43–64.
- June 18th, 2017: Added shares for alternative definitions of long-lasting articles.