ArsTechnica

Scientific Method / Science & Exploration

DOIs and their discontents

Why Ars has standardized providing reference information using DOIs in its …

Those of you who are regular readers of Ars' science content are probably aware of our use of Digital Object Identifiers, or DOIs, which act as online reference information, taking readers directly to the papers being discussed. Readers almost never comment about that feature, except when it fails, in which case we invariably hear about it—and it fails at least once a month. We've tried explaining both our reasons for using them and the reasons they break in the forums, and have recently linked to Ed Yong's excellent discussion of the system and its problems. Within a week, we were dealing with complaints due to a broken DOI. So, this is an attempt to provide a comprehensive description of the DOI system, why we use it, and why it doesn't always work smoothly.

Referencing, effort, and reward

For most of our readership, reading an Ars science article is the beginning and end of their exposure to a topic. But we also have a notable population of scientists who read, and they may find themselves interested in reading the academic paper that led to our coverage. There are any number of good reasons for doing that: the paper may be relevant to their work, they may want details we did not provide in our coverage, they suspect we might have gotten something wrong and want to correct us, etc. As a result, some form of reference to the paper is a definite good—it's a benefit for some of our readership, and may help correct errors that are read by the rest of our audience.

Traditionally, academic references have been handled with text that identifies (at a minimum) the journal, authors, and time of publication. There are several problems with using this. For starters (as anyone who can remember the pre-PubMed days knows), it's error prone. Since it's a lot of work to get right, it would add significantly to the workload of our authors, who already go well beyond the call of duty when it comes to effort. It also adds to the effort involved in navigating to the appropriate issue of the journal and finding the paper, so it doesn't serve our readership.

Finally, this approach is simply becoming obsolete. Many online-only journals, including a number from the Public Library of Science, have given up on the idea of publishing in volumes and issues—when a paper is ready, they simply publish it. It's also future-proof, as DOI's should still function if scientific publishing switches to video or some format you can just download directly to your brain.

The DOI system as a solution

For precisely this reason, PLoS handles publishing documents using DOIs. Each manuscript, prior to publication, is given a string that acts as a unique identifier and helps you locate the document. In short, it is a reference, updated for the digital age. If you have a DOI, you can visit dx.doi.org, plug it in, and you will be taken to the paper. This will happen even if the journal that originally published it has changed names, changed owners, or moved its servers to a different country. Better yet, you can simply append the DOI to dx.doi.org, and you'll be redirected to the document.

This works because the first portion of a DOI identifies the organization that owns the document, and what the relevant URL is. So, if a publisher decides to move or rename their server, they can simply update the DOI record for that organization, pointing it to the new location. On the receiving end, the second part of the DOI record comes in, as the server must implement a document resolver that takes the unique identifier and directs a browser to the appropriate document.

If everything's working properly, the DOI system should be the perfect balance of convenience (it's relatively easy to create a DOI link for an article) and utility (the link is more convenient than a traditional reference, and more likely to persist than a standard URL to a publisher's website).

Why DOIs fail

If it's the perfect system, why the regular complaints that a DOI isn't working? To begin with, a DOI doesn't eliminate the possibility of error, either on our end or at the journal's. We simply copy and paste the DOI from the original document, however, so the errors on our end have been rare.

Instead, problems typically arise because, as press, we're given full access to both papers and their DOIs well before they appear online. Most journals provide this access under an embargo: we agree not to release our articles until the time that the journals specify. Unfortunately, even for the most fast-moving journals, that time is typically several hours before the actual articles appear on the journal's website. So, anyone who's reading quickly is likely to find that the DOI fails.

But that rule only applies to the fast-moving journals, like Nature and Science. Many other journals can take a few days between when they allow journalists to write about a paper and when it becomes available to the scientific community—PNAS, which is a major source of material for us, falls in that category. Ed Yong noted one case where it took months.

Ethically, it's debatable whether it makes sense to give journalists access to papers ahead of the community that those papers are intended to inform. But it's definitely not appropriate to allow extensive public commentary on a research publication without allowing any working scientists the opportunity to vet the accuracy of that commentary. When the embargo lifts, the paper should be available.

If it's not, your best course of action isn't a complaint to our writers or in our forums. It's to contact the journal involved, and let them know that their system is creating problems. Unless they hear from the community that they are ostensibly serving (scientists), they'll have little impetus to fix the system.

Expand full story

31 Reader Comments

  1. I don't think these complaints about DOI are really about the DOI system itself. The problems are with the individual publishers that make DOI documents available. There's no way to have full compliance on that. We don't criticize the DNS system because some websites haven't been set up yet.
    105 posts | registered
  2. One of the reasons I come to Ars is for the intelligently written and thoughtful articles. Whether that article comes out the same day the journal article was released or 3 weeks later means little... as long as the high standards Ars is famous for are upheld.

    Why not just wait until the original journal article is available to the public?
    685 posts | registered
  3. Thanks for the great article about ad-bloc... wait I mean DOI's. I was actually not really aware of how the system worked until now.
    830 posts | registered
  4. syncline wrote:
    Why not just wait until the original journal article is available to the public?


    Because its in the media at the time Ars publishes usually (if its big enough to garner a 30 second sound bite in the national news). I'd rather read the Ars article than the fluff piece - its far more likely to be an accurate picture.
    4812 posts | registered
  5. Thanks for the nice write-up Dr. Jay. As someone else pointed out, the problem really isn't with DOI, but with the publishers. That said, your brief couple of sentences on how DOI is arranged got me curious about the specifics (e.g. who controls the index of publisher's DOI resolving servers). Lo and behold, DOI is backed by some serious technical chops, an IETF spec or 3, and the cooperation of a number of different standards organizations. I don't know how interesting it would be to others, but I'd certainly enjoy an expanded piece on the DOI system specifically.
    1349 posts | registered
  6. Incentive, not impetus?
    149 posts | registered
  7. Doesn't the DOI system create an epic single point of failure? From what I can see, it's a non-open database provided by one single organization. If it dies, we have a catastrophic breakdown of reference, as every text that references documents using a DOI now needs to be fixed. I see no information on their website as to how the complete database can be obtained.
    10 posts | registered
  8. I really love your science reporting, but I found this article completely unconvincing as a justification for providing *only* DOIs with your articles. The entirety of the argument is that using an author's name and the journal's name is too difficult. This is just left to be taken as fact, but it's kind of hard to accept since you bother to write 500-800 word summaries of articles, and copy and paste the DOI. Compare: Young et al, Nature. vs. doi:10.239487.snth-thoevwd. We already know the date since your article goes up the moment the embargo is lifted.

    Is there another reason? Do you think that naming a scientist in the press has the potential to distort the science? Are author names censored in the manuscripts that you're provided? Do you think that something that looks like a real citation will scare off commenters because they'll feel like they need to read the article before commenting? Does your content management system die whenever a word contains an umlaut or an accent? I'd really believe almost anything, but I can't believe your stated justification.

    In any case, keep up the good work that you do with the reporting. I'll keep complaining when you post a broken link and refuse to provide easily available information that would rectify the situation. (Seriously, you expect us to complain to a journal when you don't even give us the journal name?)
    139 posts | registered
  9. llama-lime wrote:
    In any case, keep up the good work that you do with the reporting. I'll keep complaining when you post a broken link and refuse to provide easily available information that would rectify the situation. (Seriously, you expect us to complain to a journal when you don't even give us the journal name?)

    Um, we do give you the journal name. Hadn't you noticed?

    In any case, the issue that I was trying to point out is this: if the DOI's not working, chances are really good that the paper isn't online at all. So, providing any more information doesn't really get you anything, because it's not like you're going to be able to track it down until the journal posts it. Go right ahead and continue to complain to us, but we're not able to do anything about this.

    Anyway, we don't object to naming the authors in the text, but most papers these days have multiple authors, and no clear indication of what specific author did which experiment. Since we actually describe experiments in our coverage, it's difficult to impossible to actually work the names in.
    6233 posts | registered
  10. Two suggestions:

    One to put a "What is this?" Link to a version of this article when ever a DOI link is provided.

    Slightly more work would be a status next to the DOI that the paper is pending / paper available. I'm not sure if that it could be automated or would have to be a manual task.
    27 posts | registered
  11. kettch wrote:
    Two suggestions:

    One to put a "What is this?" Link to a version of this article when ever a DOI link is provided.

    Slightly more work would be a status next to the DOI that the paper is pending / paper available. I'm not sure if that it could be automated or would have to be a manual task.


    That sure sounds like the best way to avoid more DOI complaint comments me, plus the articles about the DOI in response to the DOI complaint comments, plus even more complaints in the articles about... never mind...
    81 posts | registered
  12. note: I have never complained about this personally :-)
    81 posts | registered
  13. "Slightly more work would be a status next to the DOI that the paper is pending / paper available. I'm not sure if that it could be automated or would have to be a manual task. "

    This sounds like a great idea to me... but would take a lot of work... you would have to do something like have the DOI give you the papers status when you hover over it, defaulting to something like a "paper pending" status... and then setting up some sort of bot that runs periodically to
    1) check for a new DOI
    2) adds that DOI to database of DOIs to check the link
    3) check links of all DOIs in DOI-check database
    4) If link works, change the hover status to "paper available" and drop the DOI from the database.

    rinse and repeat
    69 posts | registered
  14. This strikes me as another excuse for the growing number of journalistic errors casting doubt on the quality of the article.
    16 posts | registered
  15. kettch wrote:
    Two suggestions:

    One to put a "What is this?" Link to a version of this article when ever a DOI link is provided.

    Slightly more work would be a status next to the DOI that the paper is pending / paper available. I'm not sure if that it could be automated or would have to be a manual task.

    The first part is definitely in the works, and why we decided to write this up. As for the availability check, it's way too much to keep track of manually, but it might be possible to do in an automated fashion. Unfortunately, the guys who would code this for us have to prioritize features that benefit the site as a whole, so the DOI thing falls into the "well, if i have some spare time..." category.

    gthurman wrote:
    This strikes me as another excuse for the growing number of journalistic errors casting doubt on the quality of the article.

    I'm not sure what you mean there - could you clarify this?
    6233 posts | registered
  16. I don't find this justification very compelling. It's a bad system with (as said above) a single point of failure.

    Further, what happens to this information when the grant money or whatnot runs out? Then what?

    To provide the DOI without the standard citation information is purely bad form and tantamount to not caring whether or not a claim can be verified.

    I can assure you that in law we wouldn't let this garbage fly one foot.
    113 posts | registered
  17. DOI *IS* the most canonical citation information. The future is now!

    I personally don't understand why the scientific community has rallied around DOI when it doesn't seem to do more than the CNRI Handle system DOI is built on, and you pay per handle with DOI. To me, that's like a CMS or Web server that makes you pay per published page or a DNS that makes you pay per entry. Maybe for some small institutions, it's worth not having to pay $100/yr to have a handle prefix, but in my institution, we hand out persistent identifiers like candy.
    9261 posts | registered
  18. Geofferic wrote:
    I don't find this justification very compelling. It's a bad system with (as said above) a single point of failure.

    Is there something bad about it aside from the potential for the single point of failure? Because i just did an experiment using Google. I picked two random articles, one from Science, one from Current Biology - put the DOIs into Google, and it found both of them. In addition, as noted, every publisher's site has to run a DOI resolver for this to work. Once you identify the URL for that resolver (and it's not difficult to do), you can skip relying on dx.doi.org to take you to the article.

    So it's not really a single point of failure, as far as i can tell.

    Quote:
    To provide the DOI without the standard citation information is purely bad form and tantamount to not caring whether or not a claim can be verified.

    What can i tell you? I disagree. Even if it weren't much more convenient, i think it's a better system, and it's one that most of the major journals have standardized on. If we didn't care about enabling others to verify our claims, we could easily skip all referencing entirely.
    6233 posts | registered
  19. Dr. Jay wrote:
    kettch wrote:
    Two suggestions:

    One to put a "What is this?" Link to a version of this article when ever a DOI link is provided.

    Slightly more work would be a status next to the DOI that the paper is pending / paper available. I'm not sure if that it could be automated or would have to be a manual task.


    The first part is definitely in the works, and why we decided to write this up. As for the availability check, it's way too much to keep track of manually, but it might be possible to do in an automated fashion. Unfortunately, the guys who would code this for us have to prioritize features that benefit the site as a whole, so the DOI thing falls into the "well, if i have some spare time..." category.


    And how much time/effort/money does it cost to create an article like this meant to help deal with the problem plus all those response comments you feel compelled to leave on the science articles when someone complains about it as opposed to having a web dev write that code? Not suggesting it's worth it, I think the link to this article next to the DOI would be enough. Just making a point.
    81 posts | registered
  20. The whole DOI system is redundant; people should just use URIs. If the publishers could just figure out how to use URIs properly, we wouldn't have to deal with this whole separate DOI thing. Some people complain that URLs change, but the existence of the DOI system is proof that people are willing to commit to stable identifiers. So just use those stable identifiers in your URIs--problem solved. And if the publisher goes kaput, then the DOI doesn't really help you since it still requires that the publisher run a resolver.

    The DOI system was designed by people who didn't really get how the web works, so they thought they could do better. But the whole world runs on URIs now; it's time to let go of DOIs.
    24 posts | registered
  21. How about putting a full citation below the DOI? There are definitely times that the DOI resolution isn't working but the journal is providing early access to everyone, especially for newsworthy articles.
    21 posts | registered
  22. Dr. Jay,

    thanks for the update. wondered about that sometimes. :)
    1300 posts | registered
  23. lid wrote:
    How about putting a full citation below the DOI? There are definitely times that the DOI resolution isn't working but the journal is providing early access to everyone, especially for newsworthy articles.

    Could you give me some indication of how often, and for how long this is the case? As I tried to point out, putting in a full citation is seriously labor intensive and easy to screw up. I'd want to know that there's a serious, persistent benefit to doing so.
    6233 posts | registered
  24. Dr. Jay,

    Thanks for such a commonsensical and accurate explanation.

    Regarding the comment asserting that the DOI system represents a single point of failure, actually it's quite the opposite. The underlying system that resolves and routes DOIs is the Handle System created by Dr. Robert Kahn, co-inventor of TCP/IP and recipient of numerous awards recognizing him as the primary architect of the Internet. Just like TCP/IP, which has no "single point of failure" in that data packets will travel over any path(s) available, the Handle System does not rely on one single resolver node but instead represents a distributed network of servers that in principle could be infinite in number. You can demonstrate this today by resolving any Handle not only via the link http://dx.doi.org/doi_number, but also through any other Handle proxy server (e.g. my company Linkstorm's proxy at http://dx.linkstorm.net/doi_number).

    Getting back to Dr. Jay's points, it's true the system ultimately relies on the content owner providing timely and thorough maintenance of these pointers, just as DNS can't be blamed if someone hasn't yet launched their website (as someone pointed out). It was precisely because of these practical, real-world considerations that I started my company Linkstorm (formerly Content Directions) as the first commercial registrar for the Handle System, offering value-added tools that address such issues. Although Handle System adoption doesn't yet seem to have spread very far beyong the STM Journal industry, and so as a small startup we've been forced to "pay the rent" by focusing instead on the Online Advertising industry, we fervently hope that someday we will see widespread adoption of this powerful linking system, which has many advantages over today's URLs. Permanent/persistent linking is only the first of these advantages.
    1 post | registered
  25. Dr. Jay wrote:
    lid wrote:
    How about putting a full citation below the DOI? There are definitely times that the DOI resolution isn't working but the journal is providing early access to everyone, especially for newsworthy articles.

    Could you give me some indication of how often, and for how long this is the case? As I tried to point out, putting in a full citation is seriously labor intensive and easy to screw up. I'd want to know that there's a serious, persistent benefit to doing so.

    http://arstechnica.com/science/news/200 ... ng-new.ars

    I'm not sure how often it's happened... I recall there being at least 2 other instances in which I found the paper while the DOI wasn't working (and I don't read that many science articles here). If you can't give us the entire citation, at least the title and first author would still be very helpful.
    21 posts | registered
  26. lid wrote:
    I'm not sure how often it's happened... I recall there being at least 2 other instances in which I found the paper while the DOI wasn't working (and I don't read that many science articles here). If you can't give us the entire citation, at least the title and first author would still be very helpful.

    PNAS has all sorts of problems with the way they handle embargoed material, so i wouldn't use them as an example of anything except what not to do. In any case, as i tried to say, if a publisher has agreed to use the DOI system, and they're not updating it properly, the best solution is to tell them it's a problem. They're the only ones who can control that.
    6233 posts | registered
  27. Dr. Jay wrote:
    As I tried to point out, putting in a full citation is seriously labor intensive and easy to screw up. I'd want to know that there's a serious, persistent benefit to doing so.


    I can understand the potential for error, but labor intensive? To create an APA citation for a single article? I mean... that's what? 90 second? Authorlast, F. (year). Article name. Journal name, ##.#, pp. Regarding the argument for experiments and listing each author - that makes no sense. List authors as you would for any citation. As above, or with a second or third author, or "et al." for multiples. Even if you are describing experiments performed by a few, you are still linking to (citing) the whole article, so the citation would be the same.
    211 posts | registered
  28. Why cares about the Journals? As an scientist myself I think it is much more important that the reader can see the actual paper. Including the DOI should indeed be done. But why not with publishing until the research is actually available online (or link to a preprint). Blaming the Journal doesn't help the researchers and is a cop out.
    13 posts | registered
  29. Simbah wrote:
    Why cares about the Journals? As an scientist myself I think it is much more important that the reader can see the actual paper. Including the DOI should indeed be done. But why not with publishing until the research is actually available online (or link to a preprint). Blaming the Journal doesn't help the researchers and is a cop out.

    Because we have no idea when the journals will make it live. Do you honestly want us to sit there hitting reload on the journal's home page, hoping that this time the article will actually appear? Believe it or not, we would rather spend that time researching and writing.
    6233 posts | registered
  30. Having the full title of the paper in the article itself would be helpful, both as a hedge against broken DOIs and for its own sake. The journal websites are often paywalled, but I've very often been successful in finding copies on websites of the authors' institutions, simply by doing a quoted search of the full title of the paper in Google.
    5 posts | registered
  31. surrealize wrote:
    The DOI system was designed by people who didn't really get how the web works, so they thought they could do better. But the whole world runs on URIs now; it's time to let go of DOIs.


    DOIs are URIs and should be represented as URLs: https://khellekson.wordpress.com/2011/0 ... -and-urls/

    Does that help?
    24 posts | registered
You must login or create an account to comment.

You May Also Like

    Need to register for a new account?

    If you don't have an account yet it's free and easy.

    Register
http://arstechnica.com/science/2010/03/dois-and-their-discontents-1/?comments=1&post=6888
arstechnica.com
crawl=CC-MAIN-2013-20
108512
application/http; msgtype=response
sha1:HMOEZAX35U4PUMDAYX3ZAC7OYMABK2AW
<urn:uuid:ead68bfa-68d6-4ae1-b9ae-0f723b7c3111>
2013-05-25T12:07:22Z
50.31.151.33
sha1:3VQOUV7JXO25PSH2DEU3W77TBEB7NKOE
<urn:uuid:6e4d260d-7df5-49a6-ba15-a9fac4a466ea>
http://arstechnica.com/science/2010/03/dois-and-their-discontents-1/?comments=1&post=6888
response
<urn:uuid:ae63cadb-5136-4a83-a856-bf3239ceee13>
robots: classic
hostname: ip-10-60-113-184.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2013-20
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for Spring 2013
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf