[OAI-implementers] Moving records in and out of sets

Tim Brody tdb01r@ecs.soton.ac.uk
Thu, 23 Oct 2003 12:17:29 +0100


Greg Lindahl wrote:

>On Tue, Oct 21, 2003 at 09:33:59AM -0400, Caroline Arms wrote:
>  
>
>>I'd like to concur with Thom that deletion/creation with a new ID would
>>"be a cure worse than the problem it is solving."  Records for OAI are not
>>usually managed independently.  The record IDs may play a role in managing
>>the content or be generated outside the OAI repository.  
>>    
>>
>
>Allow me to second that, but from another side. One way that I'd like
>to use OAI is to take records originated at other sites, and add
>metadatda. I'd rather have records disappear and reappear silently
>than to change IDs, because a changed ID means that my added metadata
>is lost. But an ideal solution would have permanent IDs, and correctly
>include information about disappearing and appeard records in
>incremental updates.
>
This discussion is related to a previous one on this list:
http://www.openarchives.org/pipermail/oai-implementers/2003-April/000829.html

Quoting Andy: "The item identifier is not the same as the resource 
identifier - because the item is not the same as the resource."

Therefore an OAI harvester should not rely upon the OAI identifier being 
in any way persistent for the resource. In a distributed system - 
especially with author-contributed resources - it is likely that there 
will be dupes, revisions, parts under multiple OAI identifiers and 
multiple repositories using the same OAI identifiers for different 
resources.

It is the *metadata* that describes the resource, not the OAI header, 
and it is the metadata that should contain a persistent, globally unique 
identifier for that resource (e.g. a DOI in DC.identifier).

I can't think of another solution to this set maintainence bug than 
"changing id" (flagging the old OAI item deleted, creating a new OAI 
item). Creating new sets  would seem to create more problems than it 
would solve (which is moving the goal-posts on harvesters).

All the best,
Tim.
http://www.openarchives.org/pipermail/oai-implementers/2003-October/001078.html
www.openarchives.org
crawl=CC-MAIN-2019-04
4469
application/http; msgtype=response
sha1:QQAU4XHGQDLUW3D7TDXDQFAVN5A74SSA
<urn:uuid:b13bf2af-7fb3-4265-badf-9115ad892429>
2019-01-23T01:40:02Z
132.236.27.218
text/html
sha1:OFRIZTX6EUKUSNEZDIWJGCCLSKIPNCXR
<urn:uuid:ea44893a-40a7-4489-b12b-3e29cd9dba8a>
http://www.openarchives.org/pipermail/oai-implementers/2003-October/001078.html
response
<urn:uuid:7acfc06a-36cf-425b-805d-bbfd14bfbc05>
isPartOf: CC-MAIN-2019-04
publisher: Common Crawl
description: Wide crawl of the web for January 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-182-67-179.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/