6.1.3 FEDERATED ARCHIVES

From wiki.dpconline.org
Revision as of 14:50, 5 October 2015 by Hlhours (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
OAIS Community Logo small.png

Community Forum | OAIS Community | OAIS Structure | OAIS Blog Posts | Active Topics and News

Federated Archives are conceptually Consumer-oriented. In addition to the Local Community (i.e., the original Designated Community served by the Archive), there exists a Global Community (i.e., an extended Designated Community) which has interests in the holdings of several OAIS Archives and has influenced those Archives to provide access to their holdings via one or more common finding aids. However, the Local Consumers are likely to have access priority over the global Consumers.

At the federated level of association, external elements can be introduced to improve inter- operability. For example, figure 6-3 illustrates a functional architecture to solve the Access problem described in 6.1.2, using an entity external to the Federated OAISes. Here, two OAIS Archives that have similar Designated Communities have decided to federate to allow Consumers to locate Information Packages of interest from either OAIS with a single Search Session. The Common Catalog & Manager is the external (global) binding element that serves as a common access point for the information in both Archives. DIPs containing the finding aids from each OAIS are ingested into the Common Catalog as is shown by the dotted lines in figure 6-3. The Common Catalog may limit its activity to being a finding aid or it may include common dissemination of products from either or both OAISes as shown by the dashed lines in the figure.

NOTE – Extra access ports are added to the diagram to illustrate the potentially differing views of each consumer community.

Figure 6-3 An OAIS Federation Employing a Common Catalog 650x0m2.jpg

Figure 6-3: An OAIS Federation Employing a Common Catalog

Federated Archives may be further classified into three levels of functionality.

– Central Site: Global access is accomplished by the export of a standard-format Associated Description to a global site. The global site independently manages a set of descriptors from many Archives and has finding aids to locate which Archive owns a collection of interest. The Consumer is given a combined view of the holdings of multiple sites, which is maintained centrally. To view details of the documents, the user must access the site that contains the actual document. This is made easier when sites and clients support a standard set of protocols.

– Distributed Finding Aid: Global access is accomplished by having a global node that can distribute a query to multiple local Archives. This means that the local Data Management entity must store an additional Associated Description in the global format or have a translator from the global queries to local queries. An option in this case is to establish a common DIP format to ease the load on Consumers who may be ordering products from many Archives.

– Distributed Access Aid: Global access is accomplished through addition of a standard ordering and dissemination mechanism, available through the global nodes to the functionality of the Distributed Finding Aids discussed above. This is a fully functional, federated system. Here, the global system may influence the Associated Description schema designs in each local Archive; it would be optimal to build new local Archives based on the global schemas and finding aids to ensure high degrees of inter-operability.

There are several major policy/technology issues that must be addressed when an OAIS joins a Federation or several independent OAISes decide to create a Federation. These issues include:

Unique AIP Names for each AIP in the Federation. An OAIS has the responsibility to provide each of its AIPs with a unique identity. When an OAIS joins a Federation, there is no assurance that some of its current OAIS AIP identifiers are not already used by other members of the Federation. An example of a general solution to this problem is to form the AIP identifiers in the Federation by assigning a Unique ID for each OAIS in the Federation and concatenating it to each AIP preserved by that OAIS. This OAIS name could be formatted according to a standard that gives the Customer or other Federation members the information needed to establish a connection to the OAIS that contains the AIP Interest. An example of a standard that accomplishes this is the ISO X.500 Directory Services Naming.

Duplicate AIPs in several different OAISes with different AIP names. This problem is caused by the fact that prior to Federation, some OAISes will have duplicated Content Information from AIPs in other OAISes to enable local user access. In this case a Global Consumer will see all the copies as separate, uniquely identified AIPs. Detailed examination of the PDI associated with the Content Information should allow the Consumer to locate the original, authoritative copy, but the search process could be very frustrating to the user. A practical way to handle this is to have a field in the Associated Description of all AIPs that identifies whether they are the original or a copy. This technique is not effective if, prior to federation, two or more Archives received the content information from the Producer to Archive. In this case the federated Archive would view these duplicate AIPs as unique, original AIPs.

The Preservation of Federation Access to AIPs when an OAIS terminates operations. Unfortunately, many Archives will close while their holdings still are of value to the Federation community. The federation should have an agreement for each member OAIS, which states which OAIS has the responsibility to take over the preservation of a closed OAIS’s holdings.

User Authentication and Access Management for global users. If an OAIS has a policy that restricts the access to some of its AIPs or charges for the dissemination of some Information Packages, there is a problem of how to identify and authenticate the user who is making requests through the central node. Each OAIS will have implemented an Authentication and Access Management system for its Local Users and the infrastructure for this function will have to be extended to include Global Users. Some examples of techniques used for this in current systems are:

• Default priorities where all members from the Global Node share a common set of access constraints and the Global Node handles all the authentication to verify the Consumer as a legitimate user of the Global node. The authentication at member OAIS is done assuming that all requests from the global node are from a single user.

• User Credential passing where the specific remote user can be authenticated by any of the Federation OAISes and the global node simply acts as an intermediary to carry the authentication dialog.

There are many factors influencing the decision of which of these techniques should be used by a specific Federation. The major criterion is the granularity of the Access Constraints in the Federation. If there is little private data and no charge for data dissemination, a policy that determines a user’s access constraints by the source he uses to discover and order AIPs is very reasonable. This involves little modification to the OAIS Authentication system, simply adding the Global Node as a Consumer. The Global Node will have to include mechanisms to identify Global Users to each of the Federated OAIS Authentication mechanisms.

If there are charges for disseminating archived information or significant private data that needs individual user authentication, the proxy techniques will not be sufficient, and User Credential passing techniques such as passwords and Certificates must be applied. The technologies to enable these supporting mechanisms are still evolving.


--Please retain original text above for reference. Propose amendments or additions below this line or respond using the Discussion tab above--


OAIS Community Logo small.png

Community Forum | OAIS Community | OAIS Structure | OAIS Blog Posts | Active Topics and News

These wiki pages are licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License. Attribute as "Community forum for digital preservation and curation standards http://wiki.dpconline.org/". The content on this wiki represents the opinions of the author and not the Digital Preservation Coalition. This wiki is not associated with ISO, the OAIS Standard or the CCSDS.
https://wiki.dpconline.org/index.php?title=6.1.3_FEDERATED_ARCHIVES&oldid=2840
wiki.dpconline.org
crawl=CC-MAIN-2021-49
26871
application/http; msgtype=response
sha1:UV55VQQMLIOMVCNIV64JXDXZYKN7ALUP
<urn:uuid:9f86c565-d2d5-40d2-9d41-0f8083f980b3>
2021-12-01T18:30:54Z
139.162.245.179
text/html
sha1:6BFUADYF6VCP77HZPYAZFU2SR3CYHERG
<urn:uuid:7e7f0043-90c8-4d98-9319-606c156ab4b0>
https://wiki.dpconline.org/index.php?title=6.1.3_FEDERATED_ARCHIVES&oldid=2840
response
<urn:uuid:fcd4de2d-ebbf-47dd-ad91-3824adf98149>
isPartOf: CC-MAIN-2021-49
publisher: Common Crawl
description: Wide crawl of the web for November/December 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-171
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.3-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/