Editor’s Note: This post coincides with Peer Review Week (September 19-26), a global event that celebrates the indispensable role peer review plays in maintaining scientific quality. The theme for this year’s event is Recognition for Review.
I recently received a particularly critical set of reviews. The manuscript was rejected, and the editor’s decision definitively closed the door to resubmission.
All rejections hurt, and there is always at least a split-second—and sometimes much longer—in which I consider quitting science altogether. But this rejection was particularly deflating, both because I had invested two years in the science, and because two of the three reviewers dismissed the very conception of the analysis. Indeed, one reviewer essentially said that the most convincing aspect was to clarify that what we had attempted should never be attempted again.
In response to my persistent moping, one of my graduate students sent me a link to the Stuff My Reviewers Say Tumblr. (The actual address doesn’t use the word “stuff.”) The mission statement reads, “Collecting the finest real specimens of reviewer comments since 1456.”Authors post reviewer comments to the site, and both the quotations and the hashtags are priceless. After an hour of reading through the archives, I was laughing so hard that I cried—the tears were, as my teenage children would say, literally rolling down my cheeks and off my chin.
I decided to re-read the reviews that I have received as first author, starting with my first submission back in the fall of 2001. Looking back at 15 years of reviews was both sobering and instructive. For one thing, it reminded me that my most recent negative reviews were not that unusual, just the most recent. But it also reminded me how fundamentally my science has benefited from the generous contributions of so many volunteer reviewers, both in the case of each and every paper, and over the longer term development of my research. Taken together, the community of almost entirely anonymous reviewers has acted as an indispensable and almost completely uncredited co-author and mentor.
This has also been my overwhelming observation during my time as editor of Geophysical Research Letters (GRL). My conservative estimate is that I have read ~4,000 reviews in the past seven years. Ultimately, all reviews provide a final evaluation. But reading these thousands of reviews has convinced me that in the vast majority of cases, the primary function of peer review is to provide honest and genuine guidance, insight and advice about the science that has been submitted. It is truly heartening to see just how constructive reviews tend to be, and how much effort reviewers invest in helping the authors. Indeed, even when a reviewer recommends rejection, in most cases they go on to provide a host of detailed, constructive suggestions for how to improve both the science and the presentation.
Although AGU and other organizations are trying to find ways to better acknowledge the hard work of reviewers, the reality is that whether the paper is accepted or rejected, most of the reviewer effort is never known to anyone other than the authors and editor. And in most cases the authors don’t know who so graciously gave them such valuable and generous feedback. And yet, the voluntary generosity of peer reviewers is an indispensable pillar of science, both institutionally and in the case of essentially every published paper.
Although AGU and other organizations are trying to find ways to better acknowledge the hard work of reviewers, the reality is that whether the paper is accepted or rejected, most of the reviewer effort is never known to anyone other than the authors and editor.
I note that GRL does occasionally receive reviews that contain careless—or in some cases downright nasty—comments. Although I am glad to report that these are the rare exception, personal attacks and mean language are unprofessional and inappropriate, and don’t have a place in scientific discourse. On behalf of the GRL Editorial Board, I would like to remind the community to be careful about the language that is used, and to refrain from demeaning critiques and personal attacks. And, on the flip side, I would like to remind authors that reviews (and editor decisions) are offered in good faith within the confines of the peer-review process—it is a violation of that good faith (and copyright) for authors to post the content of reviews publicly without permission. But it is OK to post small snippets, as on the site above.
Going through the reviews I have received, I did find some gems, which I have assembled into a personal Top 10 List. These are all from manuscripts on which I was the first author. And in all cases, they made the paper much better. So thank you, anonymous reviewers—whoever you are!
The Top 10 “Stuff My Reviewers Say” (my personal highlights, 2002–2016)
10. “Two reviewers pointed out that the main conclusion of the paper is tautological or meaningless.”
9. “I am personally a big fan of the single complex figure that tells the whole story, but this figure is beyond me.”
8. “Fig. 1 is pretty terrible… At a minimum, spend some time to make a better-looking pie chart.”
7. “The paper appears at present to be somewhat overcomplicated given it is using some basic maths to make its points.”
6. “One general challenge is that the abstract and conclusions don’t really say anything new”
5. “The use of ‘unprecedented’ is inappropriate or even nonsensical when referring to a subset of the period of record. For example, my birthday was unprecedented in the past 11 months.”
4. “This section only demonstrates that the crank can be turned, not that the results have any credibility.”
3. “This paper is a source of serious confusion for me… I am not even able to judge the scientific merits of this paper as it stands, though the parts I understand appear to be questionable.”
2. “The approach that is used can be compared to a tapas or dim sum meal; lots of little morsels, but not much depth in any instance.”
1. “As written, the paper reads like a first draft in which you unloaded your stream of consciousness on the computer.”
—Noah Diffenbaugh, Editor-in-Chief, Geophysical Research Letters; email: diffenbaugh@stanford.edu
Citation:
Diffenbaugh, N. (2016), Stuff my reviewers say, Eos, 97, https://doi.org/10.1029/2018EO059191. Published on 19 September 2016.
Text © 2016. The authors. CC BY-NC-ND 3.0
Except where otherwise noted, images are subject to copyright. Any reuse without express permission from the copyright owner is prohibited.