Monday, December 9, 2013

Your Paper Is Really Boring


I am late with my review of a paper for the Reputable Society Journal. I am the tie-breaker referee and have been trying to do my duty diligently, really go through the paper with a fine-tooth comb, analyze the response to each of the other two referees, and carefully weigh the pros and cons for the publication of this paper.

The problem is that the paper is really, really boring and I am not convinced that it should be published at all. I have picked it up and put it down several times. I don't think there is anything wrong with the technical part of the work, I believe that it's been competently done and that it is correct (within the constraints of the techniques they use, of course). But the overarching question is: why, oh why did they do this work? And, more importantly, why oh why did they write this paper? I mean, the work is topical, it explores a sort-of hot area, but the authors ask questions to which we actually already know the answers, and write the paper in a way that doesn't really reveal anything, new or otherwise.

I have read the paper several times, and I still have no idea what the conclusion is, what one is supposed to take away from this paper. I know as much about this topic after having read this paper as before having read it, I am just older and more weary. The paper does represent a huge amount of work, but other researchers would have probably done something more appealing with the data than the authors.

There should be a reason to write a paper, any paper. You might identify an existing open problem and try to solve it; you might identify a flaw in how things were previously done and do it better or do it differently so that new features are revealed; you might identify an unexplained phenomenon, and offer a hypothesis explaining it with as good of a proof as you can; you might attempt to prove or disprove an existing hypothesis of others... Whatever you do, your paper should tell us something about the world that we didn't know before. It's your duty, as the author, to frame this story for the reader. We need to know what you did, why you did it, how you did it, and what the point of all that is, i.e. what we are supposed to remember your paper for.

In this paper, I know what they did  and how, but I don't know why and I don't know what I am supposed to remember, as whatever resembles a conclusion is pretty trivial. Sometimes, not coming across as a jerk referee is a tall order.

10 comments:

Anonymous said...

In the immortal words of Lee Neuwirth, mathematician and former editor of Mathematical Reviews:

"this paper fills a much needed gap in the literature".

Anonymous said...

Full disclosure: I am a grad student and have only done a handful of reviews. But I'd like to know: suppose you have an idea for an interesting thing they could do with their data. Could you not reject the paper -- citing as your reasons that the motivation is not clear, and a compelling case as to why this is an important contribution to the field has not been articulated -- and then suggest that they resubmit and consider examining the question you propose with their data? Is it unprofessional to make suggestions like these to authors?

GMP said...

Anon at 3:39, this is a good question. There are a lot of people who do work that I wouldn't because I don't find it particularly interesting. However, as long as they are pushing the state of the art, revealing something new about the studied systems, do it rigorously and write it up convincingly, I am on board. Maybe not for the hottest of the journals, but certainly for solid journals.

I have in the past tried to suggest how the authors' data can be presented in a more compelling fashion or the message polished and made more clear. See here.

In the case of this particular paper, it has already been reviewed and revised in response to the comments of two other referees, but it's not looking good. It's too late for a massive overhaul, which is what they really need, because the journal will not consider it further if it's still unacceptable after the second review (perhaps only minor revs are OK).

Phindustry said...

Aren't those that benefit from the stewardship of public funds responsible for properly using the money and disseminating the information in a way to explain how the public funds have been spent?

People shouldn't be rewarded with publications (especially to Reputable Society Journals) for improperly spending funds by performing a bad, unnecessary study that is poorly disseminated to the community. You're not coming off as a jerk-you're coming off as responsible.

Mike Taylor said...

Anonymous, PLEASE don't do this. There is literally nothing I hate receiving from reviewers more than "You should have written this OTHER paper instead".

Review what's in front of you.

Anonymous said...

Delete paragraph 1 and there's your report right there.

pyrope said...

Second on Mike Taylor's comment. I remember that I used to make suggestions about alternative projects in reviews as a grad student and postdoc. Don't do this. Your job as a reviewer is to assess the quality of the submitted work, not design a new research project.

Iain Steele said...

An important part of the scientific method that many people forget about is repeatability. So while repeating something already done/known may not be the most exciting thing ever, it can still have value and deserves to be reported in my view.

DRo said...

I disagree with Mike Taylor. I have gotten reviews back that did not reject the paper, but had great suggestions on how to reframe it and make it more interesting to a wider audience. I appreciated it and incorporated their suggestions. Perhaps the key difference is that they did not force me to make the change, by rejecting the paper.

stevenb said...

I like yoru point about the author being able to state WHY the paper is worth reading and how it will make a difference for the reader. When I review manuscripts, if I don't see this addressed somewhere in the introduction I'm much more likely to reject or request more significant revisions.

I'm amazed by the number of papers that simply tell the reader that the paper is about topic X - and fail to reflect on what is new or different about that topic - and why the reader should care. Eventually I may come across that point - perhaps buried somewhere in the paper - and wonder how the author missed emphasizing it right at the start.

And, yes, sometimes a manuscript is just downright boring - and all you can do is to make suggestions for how to create more reader enthusiasm - often starting with a more interesting title.
http://academic-jungle.blogspot.com/2013/12/your-paper-is-really-boring.html
academic-jungle.blogspot.com
crawl=CC-MAIN-2015-06
112021
application/http; msgtype=response
sha1:QGY4O3YAKPZQFFLDYTBB5XCBKGURGOUX
<urn:uuid:f1070fae-4628-44d6-812a-1ef46324c308>
2015-01-28T16:11:15Z
74.125.228.12
sha1:4TFJLPZGDAL5JMZHV6HFWQKW5VL22GIC
<urn:uuid:2c33c118-3ab7-403f-8760-35e6dd693e7b>
http://academic-jungle.blogspot.com/2013/12/your-paper-is-really-boring.html
length
response
<urn:uuid:fc2f903b-b484-4974-b40a-b5ebd93cab51>
robots: classic
hostname: ip-10-180-212-252.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2015-06
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for January 2015
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf