SHARE

Monday, April 26, 2010

Link redirection and ID stability

We have recently updated BHL to allow us to better handle situations where titles or items are taken offline. The key problem that occurs when things are taken offline is that the links to those items are now broken. Anyone who has linked to those titles/items or uses our data exports and/or APIs to access them is affected.

In some cases, titles/items are taken offline because of serious quality issues with the scans or because of copyright concerns. In those cases, there is little we can do other than "break" the links to those items.

In other cases, however, items may be rescanned and replaced by a new item. Or, two libraries might contribute the same title to BHL, and we at BHL combined those two titles into one. These are the cases for which we can preserve the links.

The change we have made is to allow our librarians to designate a "replaced by" identifier when titles/items are taken offline. So, if item 1000 is rescanned as item 2000, we can now take item 1000 offline, but redirect incoming requests for that item to item 2000. The same is now possible when two (or more) titles are merged into one... requests for the titles that are taken offline as a part of the merging process can now be redirected to the title that remains online.

Following is a more detailed explanation of how the various parts of BHL are affected.

biodiversitylibrary.org

If a title/item has been set to redirect to another title/item, then when you attempt to access the original page you will be automatically redirected to its replacement.

For example, if you navigate to http://www.biodiversitylibrary.org/bibliography/3935 you will be redirected to http://www.biodiversitylibrary.org/bibliography/7347 . Likewise, navigating to http://www.biodiversitylibrary.org/item/23127 will result in a redirect to http://www.biodiversitylibrary.org/item/28474 .

Data Exports

The bulk data exports that are generated on a monthly basis contain only active items and published titles. Because of this, redirects are not indicated in the data exports (since redirects apply to item that are no longer active and/or published).

Keep in mind, though, that if a third party has a copy of an older data export which contains titles/items that have subsequently been replaced, then when that third party follows the links contained in the exports they will be automatically be redirected to the correct title/item.

API

The following API methods honor the redirects:

GetItemByIdentifier
GetItemMetadata
GetItemPages
GetTitleBibTeX
GetTitleByIdentifier
GetTitleEndNote
GetTitleItems
GetTitleMetadata

Using the earlier examples, a call to GetTitlemMetadata with a parameter of 3935 will return the metadata for title 7347. Likewise, a call to GetItemMetadata with a parameter of 23127 will return the metadata for item 28474.

Detailed information our APIs can be found here: http://www.biodiversitylibrary.org/api2/docs/docs.html

Final Thoughts

It's possible that you may find some titles/items that have a replacement item but have not been properly redirected. If that is the case, kindly let us know. Also, keep in mind that some titles/items may be taken offline and there may not be a valid replacement available. Please accept our apologies if you are affected by this situation.

No comments:

Post a Comment
http://blog.biodiversitylibrary.org/2010/04/link-redirection-and-id-stability.html
blog.biodiversitylibrary.org
crawl=CC-MAIN-2017-26
134112
application/http; msgtype=response
sha1:IN4HEA5NVPY7SKMA5TXZQZFYNPDYN66J
<urn:uuid:fd47c313-87fa-44f1-9de8-c23ed370c86a>
2017-06-23T05:04:58Z
172.217.7.211
application/xhtml+xml
sha1:V67OHGYH7MH2SOVOF5YXHJ3Z2FLY44JZ
<urn:uuid:8db2c00e-874a-45f7-a67b-e9bc89bbbe8f>
http://blog.biodiversitylibrary.org/2010/04/link-redirection-and-id-stability.html
response
<urn:uuid:a59fa5f1-7d82-445a-8f0f-8d585e6bdeba>
robots: classic
hostname: ip-10-185-224-210.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2017-26
operator: Common Crawl Admin
description: Wide crawl of the web for June 2017
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf