Geology & Geophysics AGU News

Diversifying the Reviewer Pool

After a successful trial to actively encourage authors to recommend diverse lists of reviewers for their papers, a reminder statement about diversity will now be rolled out across AGU journals.

By and

Since the publication this past January of a study showing underrepresentaton of women as reviewers of papers submitted to journals of the American Geophysical Union (AGU), the organization has taken multiple steps to address issues of diversity and bias in peer review. AGU has been striving to deepen its understanding of disparities in its review process related to such factors as gender, age, and geographical location and to explore ways to counteract those biases.

The conclusions of the study, conducted by the authors of this article and published in Nature, were that women were being used less as reviewers in nearly all age groups compared to expectations based on their higher success rates as authors and their distribution with age in the AGU membership. This bias initiated from author suggestions and persisted through invitations from editors. It was exacerbated by a higher decline rate among women in the younger age cohorts.

Along with those findings of gender bias in peer review, overall data clearly show that AGU needs to engage more younger scientists in reviewing and also more scientists from Asia and Africa. AGU recruits a large proportion of its reviewers from the United States, Canada, and Europe, whereas accepted authors reside roughly equally in North America, Europe, and Asia.

Action and Results

Since we began to analyze the data for the Nature study, we have been discussing these issues with AGU journal editors and their editorial boards. In turn, the society’s editors in chief have worked to expand diversity on their teams (both editors and associate editors) and to discuss the larger bias among peer reviewers.

This past spring, we also conducted an experiment in explicitly encouraging authors to recommend diverse reviewers. On 1 April 2017, with the support of the editors of our largest journal, Geophysical Research Letters (GRL), we added a note to the GRL submission form that asked authors to help the journal improve the diversity of its reviewer pool by including women, young scientists, and members of other underrepresented groups in their lists of suggested reviewers.

We have now analyzed the results from that experiment and found that with regard to gender diversity, this simple step had a salutary effect.

Fig. 1. Change in gender distribution of reviewers suggested by corresponding authors (male and female) who submitted papers to Geophysical Research Letters (GRL) before and after 1 April 2017. On that date, GRL introduced its reminder statement regarding diversity. The before data are from 2,893 matched reviewers; the after data are from 2,732 matched reviewers.

Figure 1, which shows a comparison of the percentage of female reviewers suggested by authors in the 3 months before and after this note was included, reveals that gender diversity increased significantly. A comparison with data across all other AGU journals rules out other factors in producing this change, such as awareness of our study published in January 2017.

Given the success of the recent test, we will now be including the statement, which is quoted in its entirety below, across all our journals, not only in the instructions for authors but also as a reminder to the editors when they select reviewers.

The statement will read as follows (as it did on the GRL site):

Evaluation of our journals’ peer review practices suggests that women were less likely than men to be asked to review. Please help us improve the diversity of our reviewer pool by including women, young scientists, and members of other underrepresented groups in your suggested reviewers (e.g., age, ethnic, and international diversity).

Fig. 2. Percentage of researchers by gender declining an invitation to review a paper in an AGU journal during the past 6 years.

Other data show a marked increase in the acceptance of the invitation to review, especially by women, in 2017 to date, as shown in Figure 2. This increase likely reflects a broader awareness of this issue prompted by our Nature paper, as well as our conference presentations about that study and other coverage of its findings. In that study, we merged information on age and gender from AGU membership data with editorial data on authors and reviewers. This provided a large sample of activities and allowed accounting for age in exploring gender bias.

More Work to Do

Fig. 3: Percentage of female reviewers suggested by four different author or editor cohorts for all papers submitted to all AGU journals from January 2012 through June 2017.

As shown in Figure 3, AGU journals continue overall to expand the gender diversity of our reviewer pool.

Although the gender diversity of reviewers has improved, the data for 2017 indicate that the GRL reminder statement has not so far led to a significant change in recruiting reviewers for that journal who are younger or from countries outside North America and western Europe. Thus, there is more work to do.

To help, AGU editors have been conducting workshops on reviewing internationally, including recently at the International Geological Congress meeting in Cape Town, South Africa, and the Japan Geoscience Union–AGU meeting in Chiba, outside of Tokyo, and in visits to universities and research institutions in China. AGU Publications has also conducted several webinars on reviewing, including the most recent on “How to Write Effective Reviews (and Improve Your Own Manuscript),” and will expand these efforts further.

AGU will continue to collect and report data on reviewer diversity in its scientific publishing and provide regular updates. We hope that these results encourage other journals and publishers to take more direct efforts to expand and diversify the reviewer pool.

—Brooks Hanson (email: [email protected]), Senior Vice President, Publications, AGU; and Jory Lerback, Data Analyst, Publications, AGU; now at Department of Geology and Geophysics, University of Utah, Salt Lake City

Citation: Hanson, B., and J. Lerback (2017), Diversifying the reviewer pool, Eos, 98, Published on 04 October 2017.
© 2017. The authors. CC BY-NC-ND 3.0
  • Tomdance

    Regarding “young scientists” perhaps you mean “early career”? Many of my students are veterans or otherwise older students; many of these are first-generation students as well. By conflating young with early career, one can make some incorrect assumptions that become particularly off-track when one considers institutions of higher-ed that cater to students lower on the socioeconomic scale.

  • Rina Schumer

    Maybe this is a coincidence, but in the last 6 months, the number of review requests I get has gone through the roof. Whereas I used to measure in # of requests per month, I have now changed to units of requests per week. I hope this is a sign that the initiative is working… and I hope that the pool of women reviewers grows as well.