Monthly Archives: April 2014

“Reviewer Declines Assignment”…

It’s a routine, albeit vaguely frustrating, part of being an editor: You get a new submission, you look it over to get a sense of the content, you search for referees, you send out requests, and then wait for the declines to start trickling in. It’s worse than sending out wedding invitations.

But twice in the last two days I’ve had decline responses which have been a positive delight to receive: The referee suggested alternative people I could contact! Sometimes they’re ones I’ve already contacted, but sometimes it’s a new person I hadn’t considered.

Referees, consider doing the same, if you can. Extra bonus points if you can provide an email or webpage for the people you suggest.

Why Do Referee Reports Take So Long?

Every person who has submitted a paper to a journal has at some point or another asked themselves this question. With so much riding on publication, the wait for referee reports and an editor’s decision can seem interminable. Six months, nine months, a year later…”Why do referee reports take so long? Why must I sit in agony for so long?”

I’ve been there; and now, as a relatively new member of the ranks of journal editorial board members, I’m gaining experience on the other side of the equation when, as an editor, I’m awaiting referee reports from my referees with as bated breath as I do as an author. And in that capacity, I’ve started to ask this question more seriously: Why is it that referee reports take so long?

Reflecting on my own practice, the actual time spent on a referee report generally breaks down into this:

  • My first read through a paper is at a pretty superficial level, focusing on proofreading. Where are the spelling and grammar errors, where are the repetitive parts, where are the things that make me go “hunh?” and require more thought. I do this pass first because typos and grammatical errors drive me nuts, and if I don’t have them marked out, I cannot set them aside and focus on content.
  • My second read through is then focusing on the content: Are the arguments cogent, is there material that isn’t addressed that should be, do the things which made go “hunh” on the first read still do so, or do they make more sense now that I’m paying more attention to the content?
  • This second read-through often prompts a literature search where I go to either double-check my memory of something, or double-check what the authors say about a particular fact, or to compile a list of references that would be beneficial to the author to consult.
  • If the paper is technical, then it is after the second read-through that I sit down and work through any of the proofs that were not completely transparent.
  • Then, I sit down with my annotated copy of the paper and go through the laborious process of typing up my notes and turning them into a useful report (see What Makes a Good Referee Report).

Each of these stages individually takes, on average, less than an afternoon. Thus, there is in principle no reason why I shouldn’t be able to turn around referee reports in a week.

But of course, in principle, that doesn’t happen. There are often things higher in the priority queue which cause the tedious task of writing referee reports to get pushed down, so I don’t get around to printing off the paper right away, or maybe a week elapses between each of the steps. Of course, some delays are legitimate: If a referee request arrives while I’m traveling, and I don’t have access to a printer, there is an enforced delay before I can start. Or perhaps I’m on vacation, or I’m home with a sick child. But even with such delays, the only real excuse I can give for not getting the majority of referee reports turned in within a month is procrastination.

The more that I think about this, the more I think that this aspect of our publishing practices needs to change. Many editorial boards (my own included) have a policy of 60- or 90-day initial deadlines for reports, and the motivation for this policy is that it gives the referees time to do their jobs thoroughly and without being rushed, so that the end result is of higher quality. But given my own experience as a report writer, I have to question whether longer deadlines actually do result in better reports, rather than just later ones. Working under the assumption that a referee will take roughly the same amount of time to write his report whether he is given 28 days, 60 days, or 90 days, what are the potential drawbacks of having shorter referee periods? The most obvious is that more people will turn down the request, due to legitimate reasons for not having the time in the up-coming four week (say) block, due to conferences, holidays, other non-standard obligations, etc. The next worry is that the reports received will be of a lower quality, or shorter, or not as detailed. These are legitimate worries (and there may be more: Please share yours in the comments!) but I think both can be addressed.

In the first case, we have a referee who would otherwise have accepted to review a paper but declines because the referee period is too short. This can be accommodated by the introduction (or higher uptake) of another practice: Asking the referee herself to set a reasonable deadline (which will still likely be less than 60 or 90 days, even if more than 4 weeks) for the report. The few times that I’ve been approached by a journal which defaults to a 4 week, or otherwise similarly short period, and have been uncertain if I could accommodate that deadline, I’ve replied suggesting an alternative, usually only 1-2 weeks later, and these suggestions have been met with pleasure. I’ve also found, personally, that when I set my own deadline, I’m much more likely to meet it (or complete it early) than if I’m working under a deadline set by someone else — even if the latter is further in the future.

Regarding the second, I’ve already discussed above how in my own practice, having more time to write the referee report doesn’t necessarily result in me actually spending more time to write it. I expect this is similar for many others. In the case where this actually is a problem, where, e.g., two reports are received within the four week period and they are not sufficiently detailed or useful enough for the editor to make his decision, then one can simply either ask a third referee or to ask one or both of the referees who’ve already responded to expand on unclear parts in their report. Either of these options is likely to still result in a faster turn around time, from receipt of the submission to the decision by the editor.

In principle, the length of time given for referee reports seems to me to be one of philosophical culture and practice, rather than of genuine need. In other fields, short turn around time for referee reports is the norm, and the practice works well because it is a part of the academic culture. In computer science, most conferences require full papers at the time of submission, and these are rigorously refereed sometimes with extremely short turn around times (my husband has been on the programme committees of conferences where he’s given 2-3 papers to report on in a 7-10 day period). Because these papers are technical in nature, and thus take more time to do a thorough reading of the proofs and definitions, the amount of work that goes in to producing a useful report is, in many cases, significantly more than goes into refereeing a philosophy paper. Of course, one difference between this situation and that of ordinary journal submissions is that programme committee members know roughly when they will be receiving their submissions and have to write the reports on them, and so can block out the relevant time on their calendar at the time they agree to be on the committee, whereas referees for journals are generally contacted out of the blue at any given time. Nevertheless, given a deadline of 4 weeks instead of 10 days deals with much of the possibility of receiving a request when one is simply unable to deal with it in the required time, and the possibility of suggesting an alternative deadline means that such cases can be effectively and easily dealt with.

At this point, I have myself pretty well convinced that there is not much reason to have extended deadlines for referee reports (and intend to take this issue up with my editor-in-chief). I would love to hear arguments in favor of the status quo in the comments!
https://fromtheeditors.wordpress.com/2014/04/
fromtheeditors.wordpress.com
crawl=CC-MAIN-2017-26
81890
application/http; msgtype=response
sha1:43RAPPK62XBJD2EUDX4ILPEFGQE2VS24
<urn:uuid:3ab95c9f-3777-49f2-8dca-b5094a4f656d>
2017-06-29T07:10:50Z
192.0.78.13
text/html
sha1:XBLLU3OGCVDUGZQQ4USGAVLCN2W47HN2
<urn:uuid:6b2a9fc5-7a19-49f9-9b5a-8c0b07cd2001>
https://fromtheeditors.wordpress.com/2014/04/
length
response
<urn:uuid:f1fc3a44-1a4d-43ab-8f96-4cbcf6a4f966>
robots: classic
hostname: ip-10-111-190-150.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2017-26
operator: Common Crawl Admin
description: Wide crawl of the web for June 2017
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf