Need someone to criticise your paper? Look in your references!

Receiving comments from authors you cite is associated with more citations for your paper. It is highest when you cite them, you coauthor with them and you are in the same department, too.

There are many answers to the old but classic question about how many Economists are need to change a light bulb. It’s harder to answer the question how many Economists are needed to write a research paper.

One reason why it’s hard is the small number of authors on Economics papers. In our dataset covering Financial Economics, the average number of authors is 2.1. Physicists or life scientists, where author groups are so large that not all authors know each other, would laugh at this number.

The other reason why it’s hard is that authors alone do not write papers. In Economics, a lot of collaboration is informal, as Laband and Tollison (2000) show. This is collaboration without assignment of property rights, most commonly in the form of feedback to a manuscript. Those commenters, usually acknowledged in a separte section, constitute our novel database. The average paper acknowledges 8 commenters for informal intellectual input –
with considerable variation across journals.

Here I want to show that these commenters are not (only) being put there for strategic reason, in an effort for example to signal quality. And I show that the expected increase in citations associated with one additional commenter depends on how close you are with her.

I look at three roles acknowledged commenters might take on additionally: as coauthors, as colleagues, and as referenced authors.
These roles are sufficient (but not necessary) to indicate a meaningful interaction with the manuscript. Acknowledging referenced authors for example indicates information flow targeted at the manuscript, while commenting authors and colleagues indicate a repeated interaction.

I investigate 5,320 journal articles published in 6 major financial journals (JF, RFS, JFE, JFI, JMCB, JBF) between 1997 and 2011, which acknowledge at least one commenter by name for intellectual input. It is the the same dataset used in our various papers and my previous blogposts. I exclude known referees and the journal’s managing editors, but I include discussants and PhD advisors if stated explicitly. There are 11,802 distinct commenters (compared to 5,320 distinct authors), and for 9,031 of the commenter I inferre affiliation and coauthors from their publication history as indexed in Scopus.

As many commenters are acknowledged multiple times – some 66 times in 2009-2011 alone – the total number of given comments is 46,877. Of them, 12.764 (27.64%) are by (former) colleagues at at least one of the authors’ affiliation(s).1) 2,801 (5.98%) comments are by coauthors of at least one author. 9,677 (20.65%) comments are from authors of articles referenced in the paper they commented on.

Considering the overlap, a total of 16,063 comments are by coauthors and/or colleagues and/or referenced authors. That’s 34.26% of all the comments that one can deem not predominantly strategic since a deeper relationship with these commenters is going on.2) And it’s sensible to continue with discussants for example.3)

Looking at the article level, 20% of the articles have at least a share of 50% of explained commenters, as seen from the first five bars in below figure (reversed x axis):

Are comments from different commenter roles equal in their impact on citation count?

To answer this question I proceed in two steps. First, I identify the most important roles in a simple OLS regression. Then, I estimate a Negative Binomial regression to quantify the effect of each role.

Here is the OLS regression result with the the number of commenters according to different roles:

Number of acknowledged commenters that are (1) (2) (3)
pure colleague 4.726*** 3.461** 0.565
(1.358) (1.345) (1.341)
pure referenced 7.402*** 4.712*** 2.447
(1.629) (1.642) (1.626)
pure coauthors -0.185 -1.581 -1.606
(2.775) (2.744) (2.697)
ref’d colleague 28.111*** 23.936*** 22.204***
(3.338) (3.318) (3.265)
coauthor-colleague 4.404 3.558 1.878
(2.873) (2.843) (2.797)
ref’d coauthor 28.773*** 26.258*** 25.186***
(4.301) (4.252) (4.181)
ref’d coauthor-colleague 29.332*** 23.861*** 23.925***
(4.650) (4.613) (4.533)
unexplained -0.286 -0.362 -0.537
(0.608) (0.603) (0.595)
# of authors 15.888*** 12.264***
(2.675) (2.649)
# of pages 2.124*** 1.479***
(0.238) (0.276)
Constant 23.160*** -55.115*** 33.790**
(8.095) (10.881) (13.332)
Year-FE Yes Yes Yes
Journal-FE No No Yes
N 5,320 5,310 5,310
Adjusted R2 0.105 0.125 0.156

Two most interesting findings:

  1. The number of comments received from pure coauthors is statistically insignificant
  2. The number of comments received from referenced colleagues, referenced coauthors, and referenced coauthor-colleagues is economically and stistically significant – even after controlling for basic paper characteristics and journal.

Because citations are skewed, non-negative count data, I marginal effect size of comments by commenters with different additional roles in 7 Negative Binomial regressions. Ordinary Least Squares coefficients would be distorted with this distribution. Marginal effects answer the following question: If you hold all other variables constant at their mean and increase the independent variable by 1 unit, what is the percentage increase in the dependent variable from its the mean? Because the variables are estimated at the mean of the other variables, I test one of the role variables at a time, controlling for publication year, journal, number of authors and page count. The average paper in the sample has 2.1 authors, consists of 26.4 pages and has been cited 87.56 times so far.

The following coefficient plot visualizes effect sizes and 95% error bars (all coefficients are highly statistically significant in their individual regressions):

Coefficient plot highlighting marginal effects

Acknowledging one more colleague, that you do not cite and who is not your coauthor, is associated with an increase of 3.8% more citations at the mean, given that you already acknowledge one colleague (that is the mean). Acknowledging one more author that you also cite, but who is not your coauthor and not your colleague, is associated with an increase of 5.1% more citations, given that you already acknowledge one referenced author. The respective number for a pure coauthoring commenter is 3.8% with a mean of 0.5.

It becomes very interesting when you look at interactions of referenced authors and other groups. The coefficient can be as high as 26.4%, namely for colleagues, that are both coauthors and that you cite. Part of the high coefficient however is the low sample mean of 0.02; only 802 of the 4,508 articles acknowledge commenters that fall in this category.

The tendency is clear: Receiving comments from referenced authors is associated with higher citation counts than comments from coauthors, and this effect grows when the referenced authors are also colleagues and/or coauthors. Results hold also when I regress on the intensive margin (i.e. whether someone acknowledges at least one coauthor, etc.).

Here I can only speculate about the reasons.

A compelling candidate is firstly the existence of social ties, and secondly the strength thereof.

From sociological literature we know that tacit knowledge requires a social tie to diffuse (Hansen, 1999). Simply reading a paper might not be enough to fully grasp its scope. There might be private information available only when you talk to its authors, e.g. what did not work and what did not make it into the publication. And when the paper you’re citing is pivotal to your work, it’s no wonder talking to its authors can help you.

But also the frequency of the interaction matters, not only its existence. When knowledge is very complex other things are necessary too, such as trust or hands-on learning (Hansen, 1999). This is the kind of interaction you tend to have with colleagues and coauthors. This argument however cannot explain why the association of comments from comments from coauthors with citation count is statistically insignificant. Thoughts?

  1. I define a commenter to be a colleauge if in years t, t+1, t-1 and t-2 a commenter lists the same affiliation on at least one published article as at least one of the authors of the paper under consideration, which is published in t.
  2. The share of explained commenters is likely to be higher. Scopus was not always able to extract the affiliation correctly nor could I always obtain a paper’s list of references.
  3. For example, there are are 1037 mentionings of presentations at conferences with discussants (meetings of AEA, AFA, NFA and EFA, as well as NBER Summer Insitutes)in acknowledgements. If every paper acknowledges all its discussants, that’s already 2% of all commenting relationships.

Women are less prominent in the social network of financial economics because of men

Despite networking more in order to improve a paper, women are less central in the social network of financial economics. Is it because of men?

Spurred by the observation that female scientists are on average less successful than their male counterparts, there is increasing interest in understanding this “productivity puzzle”. Women publish less and less prestigous. And this really matters: Women are less often tenured and receive less prizes.

They are also less central in the social network of informal collaboration, as we show in our various research. Of the 100 most betweeneess central (a measure of importance in information flow) or the 100 most eigenvector central (a measure of influence) researchers, not even 15% are female. (Head over to our rankings to see them online.) In an unpublished manuscript, Ductor, Goyal and Prummer (2017) for example show that female Economists have less co-authors on average, but tend to collaborate more often with them.

This suggest that female researchers participate less in the global flow of knowledge, which seems to be so crucial for research. Let’s see why women are less central in the social network of financial economics.

Our dataset contains a total of 14529 researchers that somehow appeared in published research articles in major finance journals (JF, RFS, JFE, JFI, JMCB, JBF) between 1997 and 2011, either as author or as acknowledged commenter, or both. Based on the researchers firstnames I estimated their gender with Varying from year to year, 15% to 20% are female, which in itself is a very low number (there are some undetermined first names which I report as unknowns). In increased over time, which is probably a good sign. 18% of the authors are female, comparing to 11% of all acknowledged commenters (including authors when they have been acknowedledged).

The first observation is that women are acknowledged less often. Over all the years, each male is on average acknowledged 5.5 times for helpful comments. His female counterpart is acknowledged less than 4 times. There is however some improvement over the years, as seen from below graphic:

Why is this? A long standing hypothesis is that women network differently (read: less effective). There is anecdotal evidence that women shy away from networking.

We can check this too, because our dataset does not only contain information on who acknowledged whom, but also on who presented where on seminars and conferences. I classify articles according to their gender composition as (all-)male, mixed, and (all-)female, with the few unknowns assuming to be male. 4147 articles are male, 298 are female, and 1314 articles are mixed.

Here a very interesting picture emerges: Articles written by women report more informal collaboration per person than articles written by males only, not less. Mixed author groups interestingly report the lowest intensive margin of all the three groups. (The extensive margin, i.e. the propensitiy to acknowledge, is very stable over all the groups with close to 85% of all articles).

As the following graphs shows, female authors speak to 50% more commenters per author during the writing, present at 30% more conferences or seminars. Though the standard deviations are higher for articles written by females only, the differences between the groups are higly statistically significant, as pairwise independent t-tests reveal.

Why is it then that women are acknowledged less often? It’s not that women do not go out and make use of their network – it’s the men who do not talk to women: The average share of acknowledged female commenters is about 9% for papers written by males only, about 11% for mixed-author papers, and 12% for articles written by females only. The groups are pairwise highly statistically significant except for mixed vs. females.

Pooling articles written by at least one women already makes this very obvious: The t-test statistic by comparing articles written by males only with articles written by at least one women is -5.17 with p=0.0 – the mere fact that a women writes an article completely changes the gender composition of acknowledged commenters significantly.

This echoes a recent nature blog post: “Women aren’t failing at science, science is failing at women”. In our case it is “male scientists are failing at women”.

  • Ductor, Lorenzo, Sanjeev Goyal and Anja Prummer (2017) ‘Gender and Social Networks‘, unpublished manuscript.
  • October 25th, 2017: Added numbers for articles by gender-composition.

Ranking journals by the share of long-standing articles

Assessing the long-term impact of research: What fraction of articles are still cited 20 years later?

Every Economist will sooner or later have stumbled upon journal rankings. Rankings inform decisions on what to read first and where to publish preferably. Rankings determine careers and individual honours. One good thing about them is their vast number  so that everyone can pick the one with the most appropriate criteria.

It has been argued that determining impact based on journal reputation is preferable to a mere citation count: Citations do not allow us to determine whether research is novel. On the other hand, determining impact by how often someone makes it to top journals is no good alternative, too. It discourages innovative work because non-innovative work has it much easier to get published high.

Packalen and Bhattacharya (2017) have therefore recently developed a “Neophilia Ranking”. The ranking criteria is what fraction of articles mention relatively new ideas.

Here I explore the idea a ranking Journals by the long-term impact: What fraction of articles are still cited 20 years later?

To do so I look at a selection of major Economics Journals and compute the share of their 1996 articles that are cited at least once in 2016. I take all journals rated at least “A” by Combes and Linnemer (2010) in their General Economics and six field-specific rankings. This yields a total of 54 journals. I am looking at the share of articles published in 1996 and cited in 2016, according to Scopus (Elsevier’s Citation Database). Of those, 11 did not exist in 1996 (or Scopus did not index this year). For example, the Review of Economic Dynamics, ranked 35 in Combes and Linnemer (2010)’s main ranking, was founded in 1998.

The remaining 43 journals published 2375 articles in total in 2016, ranging from 170 (Canadian Journal of Economics) to 10 (Brookings Papers of Economic Activity). Of these, more than the half, namely 1230, were still cited in 2016. The ranking is led by Journal of Political Economy, with 91.11% of its 45 publications in 1996 cited 20 years later.

The following table ranks all these journals by share of 1996 publications that are cited at least once in 2016 according to Scopus, which is listed column “Share >0 citations in 2016”. The column before is the conditional citation count: Given that the article is cited in 2016, how often has it been cited on average?

Journal Publications in 1996 Cited in 2016 Avg. cond. citation count in 2016 Share >0 citations in 2016 (in%) Share >10 citations in 2016 (in%) Share >50 citations in 2016 (in%)
Journal of Political Economy 45 41 8.95 91.11 24.44 0.00
Quarterly Journal of Economics 41 37 10.32 90.24 34.15 0.00
Journal of Economic Literature 18 16 16.38 88.89 55.56 0.00
Journal of Financial Economics 47 41 15.34 87.23 36.17 2.13
Review of Financial Studies 37 32 6.38 86.49 16.22 2.70
Journal of Economic Growth 21 18 11.67 85.71 19.05 0.00
Journal of Finance 69 59 14.63 85.51 34.78 2.90
Econometrica 56 47 15.89 83.93 16.07 5.36
Brookings Papers on Economic Activity 10 8 6.00 80.00 20.00 0.00
Journal of Economic Perspectives 47 37 6.08 78.72 14.89 0.00
American Economic Review 149 115 8.46 77.18 16.11 2.01
Review of Economic Studies 28 20 6.40 71.43 10.71 0.00
Review of Economics and Statistics 60 42 6.40 70.00 11.67 1.67
Journal of Financial Intermediation 15 10 1.90 66.67 0.00 0.00
Journal of Econometrics 98 65 9.52 66.33 14.29 4.08
Journal of Labor Economics 28 17 3.53 60.71 0.00 0.00
Journal of Public Economics 92 55 4.16 59.78 1.09 1.09
Journal of Banking and Finance 86 50 3.60 58.14 5.81 0.00
Journal of Development Economics 63 36 6.11 57.14 11.11 0.00
Journal of Monetary Economics 57 32 6.00 56.14 7.02 0.00
Economic Policy 11 6 2.50 54.55 0.00 0.00
The Journal of Economic History 33 18 2.06 54.55 0.00 0.00
Journal of Economic Surveys 11 6 3.67 54.55 0.00 0.00
European Economic Review 111 59 5.49 53.15 9.01 0.00
Economic Journal 91 48 7.10 52.75 8.79 0.00
Journal of International Economics 41 19 5.05 46.34 7.32 0.00
Journal of Money, Credit and Banking 53 24 5.29 45.28 5.66 0.00
Journal of Economic Theory 104 42 5.19 40.38 2.88 0.96
Oxford Review of Economic Policy 30 12 4.33 40.00 3.33 0.00
European Journal of Political Economy 52 20 2.75 38.46 1.92 0.00
Economic Inquiry 48 18 3.67 37.50 2.08 0.00
Journal of Economic Dynamics and Control 77 27 2.48 35.06 0.00 0.00
Labour Economics 20 7 1.86 35.00 0.00 0.00
Economics Letters 212 63 2.44 29.72 0.47 0.00
Journal of International Money and Finance 52 15 2.87 28.85 0.00 0.00
Scandinavian Journal of Economics 35 10 3.60 28.57 5.71 0.00
Canadian Journal of Economics 170 37 2.08 21.76 0.59 0.00
Journal of Macroeconomics 41 8 3.50 19.51 2.44 0.00
Journal of Policy Modeling 27 5 1.80 18.52 0.00 0.00
Manchester School 22 3 1.33 13.64 0.00 0.00
National Institute Economic Review 27 3 1.33 11.11 0.00 0.00
Open Economies Review 22 2 1.50 9.09 0.00 0.00
North American Journal of Economics and Finance 18 0 0 0.00 0.00 0.00

Interestingly, the journals known as the “top 5” journals (The American Economic Review, Econometrica, the Journal of Political Economy, The Review of Economic Studies, and the Quarterly Journal of Economics) are not top 5 in this list. The Journal of Economic Literature, for example, is ranked third and has the highest conditional average citation count of about 16 citations. It also has the highest share of articles when we define long-standing articles as having more than 5 citations. This mirrors findings presented at this year’s ASSA conference’s Panel Discussion on Publishing and Promotion in Economics: The Curse of the Top Five. If define a long-lasting impact article by being cited more than 50 times in the 20th year, then Econometrica comes out first.

Interesting questions arise: Do these journals attract publications with a long impact? Do these articles still attract citations because they got published where they are? Or were the editors in 1996 simply good at picking the ‘right’ articles with presumably long-lasting impact?

Though the questions are not easy to answer, I would like to shed light on the differences among the journals by showing how the average long-lasting article (i.e. the one that is still cited 20 years later) performs over time. The following plot traces the average number of citations in a given year by journal to all long-lasting articles of 1996 (hover over the lines to see the journal names and zoom in by drawing a rectangle with your mouse – might not work on mobile versions).

As a further robustness check, here is above list with citations 10 years later for publications from 2006. That is, how many of a journal’s 2006 publications have been cited at least once in 2016? Again, the last column gives the average number of citations in the 10th year. The list is compiled from 2012 articles from 48 journals (only 6 did not exist in 2006 or were not indexed).

Journal Publications in 2006 Cited in 2016 Avg. cond. citation count in 2016 Share >0 citations in 2016 (in%) Share >10 citations in 2016 (in%) Share >50 citations in 2016 (in%)
Quarterly Journal of Economics 40 39 15.90 97.50 52.50 5.00
Journal of Political Economy 38 37 9.57 97.37 34.21 0.00
Journal of Finance 88 84 18.35 95.45 51.14 7.95
Journal of Financial Economics 88 82 16.09 93.18 36.36 6.82
Journal of Economic Perspectives 48 44 15.18 91.67 31.25 4.17
Journal of Economic Growth 12 11 6.09 91.67 16.67 0.00
Journal of Labor Economics 28 25 11.12 89.29 28.57 3.57
Review of Financial Studies 43 38 11.58 88.37 30.23 2.33
American Economic Review 191 165 9.34 86.39 22.51 1.57
Review of Economic Studies 44 38 7.92 86.36 22.73 0.00
Review of Economics and Statistics 58 50 11.96 86.21 25.86 1.72
Econometrica 60 51 12.39 85.00 20.00 6.67
Journal of the European Economic Association 61 47 4.30 77.05 4.92 0.00
Economic Journal 83 63 5.87 75.90 9.64 1.20
Journal of Economic Literature 20 15 11.27 75.00 35.00 0.00
Journal of Development Economics 67 49 7.04 73.13 10.45 1.49
Journal of Econometrics 130 92 6.84 70.77 12.31 0.77
Economic Policy 20 14 4.14 70.00 0.00 0.00
Journal of Public Economics 110 77 5.32 70.00 10.91 0.00
Journal of Economic Theory 76 53 3.58 69.74 1.32 0.00
Oxford Review of Economic Policy 33 23 5.83 69.70 12.12 0.00
Journal of International Money and Finance 66 46 4.85 69.70 6.06 0.00
Journal of International Economics 65 45 5.89 69.23 13.85 0.00
Journal of Monetary Economics 107 73 3.93 68.22 6.54 0.00
Journal of Banking and Finance 172 117 5.94 68.02 8.14 1.16
Journal of Money, Credit and Banking 99 66 4.55 66.67 6.06 0.00
Review of Economic Dynamics 33 22 2.91 66.67 3.03 0.00
Journal of Economic Surveys 29 19 4.63 65.52 6.90 0.00
European Economic Review 94 60 4.65 63.83 7.45 0.00
Journal of Financial Intermediation 24 15 4.33 62.50 4.17 0.00
Scandinavian Journal of Economics 37 23 3.09 62.16 2.70 0.00
The Journal of Economic History 36 22 2.00 61.11 0.00 0.00
International Finance 17 10 2.20 58.82 0.00 0.00
Journal of Financial Stability 14 8 5.25 57.14 7.14 0.00
Labour Economics 37 21 2.71 56.76 0.00 0.00
European Journal of Political Economy 53 30 4.53 56.60 7.55 0.00
Canadian Journal of Economics 60 30 2.00 50.00 1.67 0.00
Journal of Economic Dynamics and Control 112 55 2.75 49.11 0.89 0.00
Economic Inquiry 54 26 2.88 48.15 1.85 0.00
Journal of Macroeconomics 50 22 1.77 44.00 0.00 0.00
Journal of Policy Modeling 77 33 2.06 42.86 1.30 0.00
Economics Letters 291 122 2.61 41.92 0.69 0.00
Open Economies Review 28 11 2.36 39.29 0.00 0.00
Brookings Papers on Economic Activity 13 5 3.60 38.46 0.00 0.00
Manchester School 49 18 2.28 36.73 2.04 0.00
North American Journal of Economics and Finance 25 6 5.33 24.00 4.00 0.00
Macroeconomic Dynamics 30 7 4.57 23.33 3.33 0.00
National Institute Economic Review 57 3 1.33 5.26 0.00 0.00

Comparing both tables, we see some dynamics happening here, with the top 5 journals being closer to the top and closer together than in the ranking of 20 years-later citations. This also holds for some major field journals, like the Journal of Finance, Journal of Economic Growth or Journal of Labor Economics.

  • June 18th, 2017: Added shares for alternative definitions of long-lasting articles.

Economics: Always a Small World?

Has our profession been a Small World since the 2000s or earlier? An assesment on the network of formal and informal collaboration.

In a popular study of co-author networks in Economics, Goyal, van der Leij and Moraga-González (2006) show that our profession is a Small World only since the 1990s. I repeat their analysis on our network of intellectual collaboration.

A Small World Network – introduced by Watts and Strogatz (1998) – is a very handy representation of the world: Many short distance links, few long distance links. Think of many groups well connected intra-group but sparsely connected inter-group. While most people do not know each other, they are connected indirectly via hubs. Whether a network displays small-world properties or not is relevant for reaching consensus among agents, maintaining power grids and fighting diseases (this is the motivating example in Watts and Strogatz (1998)).

A social network is said to have small-world properties, if the average shortest distance between all nodes does not grow too fast (at most Ο(log N)) as the number of nodes N increases. For their study, Goyal, van der Leij and Moraga-González (2006) reformulate this mathematical requirement into four necessary and sufficient conditions:

  1. number of nodes is large as compared to the average number of neighbors
  2. There exists a giant component (in which a large share of nodes is somehow connected via possibly intermediate steps)
  3. Average distance between nodes is small
  4. Clustering – share of one’s neighbors that are also neighbors themselves – is high

Goyal, van der Leij and Moraga-González (2006) argue that the world of Economics is becoming smaller, precisley by examining small world properties. The networks they study are three co-author networks, one for each decade 1970-1979, 1980-1989, and 1990-1999. Over time, they show that our profession has become more connected.

The question is whether co-author networks are the right network representing our profession.

Laband and Tollison (2000) as well as we in our papers have shown that a large extend of intellectual collaboration is of informal nature. That is, researchers other than authors contribute to the production of science. For research articles, these informal collaborators are acknowledged typically on the frontpage of an article.

I examine networks, each rebuild from full research articles published between 1997 and 2011 in six major finance journals (The Journal of Finance, The Review of Financial Studies, the Journal of Financial Economis, the Journal of Financial Intermediation, the Journal of Money, Credit & Banking, and the Journal of Banking and Finance). Each network is five years long, so that we have three networks: 1997-2001, 2002-2006, and 2007-2011. In this network two academics are connected when either they have jointly published an article, or one acknowledges the other, or both. For comparison I build a classical co-author network. The following tables present the relevant small-world figures:

Table 1: Statistics of the co-author network
1997-2001 2002-2006 2007-2011
Total nodes 1,825 2,683 4,225
  Average 1.78 1.93 2.31
  Standard deviation 1.40 1.56 1.86
Giant component:
  Size 161 672 1,560
  Percentage 9% 25% 37%
Second-largest component 58 23 23
Isolated nodes:
  Number 220 257 267
  Percentage 12% 10% 6%
Clustering coefficient 0.37 0.39 0.48
Distance in giant component:
  Average 10.13 12.88 10.74
  Standard deviation 6.44 5.05 3.65

The figures are quite in line with Table 1 in Goyal, van der Leij and Moraga-González (2006), except for the existence of a giant component, which only emerged in the 2000s – and even there it is debateable whether a share of less than 40% constitutes a giant component.

Let’s add acknowledgements:

Table 2: Statistics of the network of intellectual collaboration
1997-2001 2002-2006 2007-2011
Total nodes 4,754 6,532 9,624
  Average 7.80 8.68 10.73
  Standard deviation 12.53 13.93 18.54
Giant component:
  Size 4,507 6,234 9,156
  Percentage 95% 95% 95%
Second-largest component 11 11 12
Isolated nodes:
  Number 21 18 25
  Percentage <1% <1% <1%
Clustering coefficient 0.48 0.49 0.54
Distance in giant component:
  Average 2.35 2.22 1.92
  Standard deviation 0.99 0.90 0.79

We can immediately see that accounting for author-commenter links increases almost all numbers (except those for distance, which decrease): The the networks’ connectivity improves dramatically. Most notably the giant component, which connects virtually all nodes in the network. This implies information can reach every node in the network, which it could not in the co-author network. It also flows much faster, as the lower distances show.

One caveat is in order though, and that is the inflation of links between authors and commenters. We generally do not know which author spoke to which commenter on a multi-authored article. Therefore we assume a link between every author and every commenter. This inflates degree strongly distance somewhat, so these figures should be digested with caution. And while it has an unknown effect onclustering, too many edges do not affect the size and share of the giant component.

We also look at a much smaller network of only 6 journals and three years, while Goyal, van der Leij and Moraga-González (2006) study a network of an unreported number of journals (they use the EconList database) and 10 years. We believe however that our claim remains valid: The inclusion of informal links between economists is the right way to go because it accounts common information flows.


Who do we acknowledge, and why?

Being acknowledged on a research paper tells something about you and about the author: The author perceives you as important.

In the last post I reported evidence that many more researchers contribute to a research paper than just authors. The most visible difference between authors and commenters — and our only way to judge on the effort — is that authors receive all the credit while commenters are “just” acknowledged.

Working with others is that common, that it naturally rises the question: Where do we meet other researchers to get their opinion? Most likely, when it’s not faculty colleagues and friends, it is conference guests or seminar visitors to your faculty.

It’s an open question. The fact however that an author  – call him A – receives advice from another researcher – call him C – reveals information about both: For one, it means that A perceives C as important in the very field, at least important enough to get insights. It may also be because A want to signal quality by showing that he knows C. Vice versa, the fact that C gives a comment for which he is acknowledged shows that he is obviously willing to spend time on someone else’s research.

Oettel (2012) has termed this behaviour as “helpful”, saying that a helpful colleague (one that is often acknowledged) matters more to you than a productive colleague (one that publishes many papers). In a similar fashion, we created a ranking of how often financial economists have been acknowledged in six financial economics journals. Our dataset consists of more than 5,600 full research articles.

The distribution of how often someone is acknowledged is very skewed. The vast majority of commenters is being acknowledged twice or less, with very few individuals being acknowledged very often (zoom in the upper tail by dragging a rectangle):

However, we can uncover more. When A and C agree to exchange ideas, we can think think of their relationship as a tie in a network. In network lingo, a tie is being formed when both nodes (here A and C) mutually. Now take many more researchers and eventually more form a tie with C. This likely puts C in a very central position, which both displays and transfers influence. For this reason we computed the rank according to eigenvector centrality.

The idea behind this measure is: When you know important people, you are probably important as well. Importance here is measured iteratively as the degree to which you know important people. Google uses the same idea for its PageRank.

Interestingly, being acknowledged often does not automatically put researchers in central position. In our paper we explore this relationship and find that Spearman correlations for different periods never exceed 0.65.

In a related blogpost on VOX from last year we propose this ranking as a new measure of influence. This makes sense because under the assumption of mutually beneficial tie formation the market automatically puts the “right” people in the center – the network as the aggregated market wisdom on which researchers matter more for financial economics.

Please refer to our website for an ranking for the period 2009-2011 and others.


Informal Collaboration increases over time – But more so for top journals

Scientists do not work alone. Colleagues help them by providing feedback and commentary. Upon publication, helpful input is being acknowledged, which constitutes our novel database.

The notion of the sole scientist working endlessly in an ivory tower until a scientific contribution emerged in the form of a long book is a romantic one, but it is wrong. Science is social endeavor, and collegiate help comes in may forms. Reading and commenting on a colleague’s draft a very common one, and presenting in seminars, workshops and conferences may be even more common. In fact, collaboration is so common that tips and advices for academic presentations are everywhere.

Financial Economics is no exception to this, as we show in our papers. Financial economists commonly acknowledge helpful input by their colleagues: Whether they have commented on a draft, discussed at a conference or gave feedback after listening to an idea. Additionally, authors list conferences and universities where they presented their work so as to acknowledge the audience.

We have used 6407 full research articles from six major financial economics journals published in the 1997-2011 period. These journals are The Journal of Finance (JF; 1153articles), The Review of Financial Studies (RFS; 891), the Journal of Financial Economics (JFE; 1157), the Journal of Financial Intermediation (JFI; 278), the Journal of Money, Credit & Banking (JMCB; 816) and the Journal of Banking and Finance (JBF; 2110). The journals JF, RFS and JFE are commonly referred to as top finance journals (see for example the JF’s annual reports, which uses RFS and JFE as benchmarks).

There are three forms of informal collaboration present in Economics and its subfields: direct feedback by individuals, seminar presentations, and conference presentations. For our analysis workshops count as conference.

In 2011 the vast majority of articles report at least one form of informal collaboration all articles report any of the three forms of informal collaboration, but the share differs from journal to journal:

While virtually all articles published in the three top journals (JF, RFS, JFE) report at least one form of informal collaboration, it is usually 80% for JBF articles.

Just as the extensive margin the intensive margin increased over time as well. Of all 6407 articles in our dataset, a total of 5633 articles acknowledge at least one commenter or one seminar presentation or one conference presentation. We counted them and calculated the average per year and journal:

There are two remarkable trends: First, informal collaboration increases over time. In all journals, the average article acknowledges more commenters, more seminars and more conferences. Laband and Tollison (2002) have reported a similar increase in the number of commenters for three top Economics journal publications.

But the amount of informal collaboration in top journals (JF, RFS, JFE) has always been higher than in the other three journals (JFI, JMCB, JBF). Only the JFI tends to be become a top journal, at least in terms of informal collaboration.

While the correlations are striking, the causal effects are hard to identify. It might be quality-improvement, but it might also be a selection bias on the side of the author – i.e. only presenting the better papers -, or selection bias by the commenters, seminar venues and conferences – i.e. only accepting the better articles (Brown, 2005).

But even when there are other effects at work too: The fact that editors such as the editors of the the top finance journal editors Green, O’Hara and Schwert (2002) (among other editors) advise authors “to circulate their papers and give seminars to colleagues to receive constructive criticism before submitting to a journal” is evidence that informal collaboration has its merits.