A schema-based peer-to-peer infrastructure for digital library networks [Elektronische Ressource] / von Wolf Siberski

-

English
120 pages
Obtenez un accès à la bibliothèque pour le consulter en ligne
En savoir plus

Description

A Schema-based Peer-to-Peer Infrastructurefor Digital Library NetworksDer Fakultat¨ fur¨ Elektrotechnik und Informatik¨der Gottfried Wilhelm Leibniz Universitat Hannoverzur Erlangung des GradesDoktor der NaturwissenschaftenDr. rer. nat.genehmigte Dissertation vonDipl.-Inform. Wolf Siberskigeboren am 10. Februar 1966 in Gottingen¨2006Referent: Prof. Dr. Wolfgang NejdlKo-Referenten: Prof. Dr. Karl AbererProf. Dr. Udo LipeckTag der Promotion: 15. Dezember 2006Ben Zoma said: Who is wise? He who learns from every man, as it is said:“From all my teachers have I gained understanding”Pirkei Avot 4,1iACKNOWLEDGEMENTSFirst and foremost, I would like to thank my advisor Prof. Dr. Wolgang Nejdl. He introducedme to methodical research, always had time for scientiﬁc discussions, gave me the freedom topursue my research goals, and provided an excellent research environment, to mention just afew points. In short, this thesis would not have been possible without his ample support andguidance.I also would like to thank my other referees Prof. Dr. Karl Aberer and Prof. Dr. Uwe Lipeckfor their very helpful comments and suggestions.I’m grateful to Prof. Dr. Heinz Zulligho¨ ven and Prof. Dr. Christiane Floyd, who have shapedmy understanding not only of software, but also of computer science in general.

Sujets

Informations

Publié par
Publié le 01 janvier 2006
Nombre de lectures 6
Langue English
Signaler un problème

A Schema-based Peer-to-Peer Infrastructure
for Digital Library Networks
Der Fakultat¨ fur¨ Elektrotechnik und Informatik
¨der Gottfried Wilhelm Leibniz Universitat Hannover
zur Erlangung des Grades
Doktor der Naturwissenschaften
Dr. rer. nat.
genehmigte Dissertation von
Dipl.-Inform. Wolf Siberski
geboren am 10. Februar 1966 in Gottingen¨
2006Referent: Prof. Dr. Wolfgang Nejdl
Ko-Referenten: Prof. Dr. Karl Aberer
Prof. Dr. Udo Lipeck
Tag der Promotion: 15. Dezember 2006Ben Zoma said: Who is wise? He who learns from every man, as it is said:
“From all my teachers have I gained understanding”
Pirkei Avot 4,1
iACKNOWLEDGEMENTS
First and foremost, I would like to thank my advisor Prof. Dr. Wolgang Nejdl. He introduced
me to methodical research, always had time for scientiﬁc discussions, gave me the freedom to
pursue my research goals, and provided an excellent research environment, to mention just a
few points. In short, this thesis would not have been possible without his ample support and
guidance.
I also would like to thank my other referees Prof. Dr. Karl Aberer and Prof. Dr. Uwe Lipeck
for their very helpful comments and suggestions.
I’m grateful to Prof. Dr. Heinz Zulligho¨ ven and Prof. Dr. Christiane Floyd, who have shaped
my understanding not only of software, but also of computer science in general.
The collaboration and discussion with my colleagues at L3S Research Center, University of
Hannover, and elsewhere was an indispensable source of information and has spawned a lot of
insights for this thesis. But what is even more important, our joint work was always a pleasure,
and we had a lot of fun together. I would like to thank all of my colleagues for their cooperation
and openness, especially Dr. Uwe Thaden, Dr. Wolf-Tilo Balke, and Dr. Peter Dolog.
It is tremendously helpful to work in a smooth administrative and technical environment. Katia
Capelli, Thomas Losch,¨ Dr. Christoph Strutz, Iris Zieseniß, Claudia Saalbach, and Marko
Brosowski provide such an environment for L3S, and were always very supportive when I
came to them with my minor or major requests.
During the creation of a thesis, it is probably inevitable to face some stumbling blocks. The
guide of Dr. Alexandra Fischer-Flebbe helped me in overcoming mine.
I will always be grateful for the love and care of my parents. They gave me self-conﬁdence
and intellectual curiosity, the basis for all my work.
Finally, my wife Susanne and my children Dana and Jona bore it with exceeding patience that
I couldn’t spend enough time with them, and sustained me every day with their their love and
affection.
iiABSTRACT
A Schema-based Peer-to-Peer Infrastructure for Digital Libraries
in
English
In today’s connected world, users are not content with searching only one local library or
archive, but want and need to take a substantial number of collections into account when
looking for relevant information. Currently, most digital libraries and catalog systems only
support local search, and only few facilities offer federated search over several libraries. One
reason is that central federation instances cause signiﬁcant infrastructure costs, and there are
only limited incentives for libraries to offer such services. An appealing solution is to avoid
a central federation instance and use a completely distributed infrastructure instead, thus also
distributing the infrastructure efforts. In this thesis, we will present such an infrastructure
which combines peer-to-peer, distributed database and Semantic Web technology to provide
seamless search in an open network of digital libraries.
The proposed solution is based on a super-peer topology, where the most powerful nodes
form a network backbone and take over mediator-like responsibilities to distribute queries and
merge results. The network content is modeled as a database fragmented over all nodes. Our
basic algorithm, SPQR (super-peer-based query routing), allows processing of queries accord-
ing to the classic relational algebra, and is shown to always produce the correct result set with
respect to this fragmented database. We present an implementation of our approach which en-
ables the interconnection of library systems conforming to established Open Archive Initiative
standards. An extension of SPQR for preference-based queries allows users to retrieve ’best
matches’ for their queries instead of only exact matches. Extensive evaluations based on a
peer-to-peer simulation framework show the algorithm’s performance and scalability.
Keywords: peer-to-peer networks, distributed databases, digital libraries
iiiABSTRACT
A Schema-based Peer-to-Peer Infrastructure for Digital Libraries
in
Deutsch
Die heutige Vernetzung bringt es mit sich, dass Nutzer von Bibliotheken und Archiven sich
nicht mehr mit einer einzigen Informationsquelle begnugen,¨ wenn sie nach relevanter Informa-
tion suche, sondern eine mehr oder weniger große Anzahl von Informationsanbietern konsul-
tieren wollen und mussen.¨ Momentan unterstutzen¨ die meisten Katalogsysteme und digitalen
Bibliotheken nur lokale Suche, und es gibt nur eine geringe Anzahl von Serviceangeboten fur¨
¨ ¨ ¨foderierte Suche uber viele Bibliotheken hinweg. Ein Grund dafur ist, dass solche Services
merkliche Infrastrukturkosten mit sich bringen, und es fur¨ jede einzelne Bibliothek wenig An-
reize gibt, diese Kosten zu tragen. Eine attraktive Losung¨ fur¨ diese Problematik ist, zentrale
Services ganz zu vermeiden, und stattdessen eine vollstandig¨ verteilte Infrastruktur zu verwen-
den; auf diese Weise werden auch die Aufwendungen fur¨ die Infrastruktur uber¨ alle beteiligten
Bibliotheken verteilt. In dieser Arbeit stellen wir eine solche vor, die Ansatze¨
aus Peer-to-Peer-Netzwerken, verteilten Datenbanken und dem Semantic Web kombiniert, um
transparente Suche in einem offenen Netzwerk digitaler Bibliotheken zu ermoglichen.¨
Die vorgeschlagene Losung¨ basiert auf einer Super-Peer-Topologie, in der die leistungsfahig-¨
sten Knoten ein Netzwerk-Backbone formen und Mediator-Aufgaben der Verteilung von An-
fragen und Zusammenfuhrung¨ der Ergebnisse ubernehmen.¨ Die im Netzwerk angebotenen
Informationen werden als uber¨ alle Knoten fragmentierte Datenbank modelliert. Zur Ver-
arbeitung relationaler Anfragen in dieser verteilten Datenbank dient der Algorithmus SPQR
(Super-peer-based Query Routing), dessen Korrektheit gezeigt wird. Weiterhin wird die Im-
plementierung eines auf SPQR basierenden Netzwerks beschrieben, mit dem Bibliothekssys-
teme vernetzt werden konnen,¨ die konform zu etablierten Standards der Open Archive Initia-
tive sind. Aufbauend auf SPQR stellen wir einen Algorithmus fur¨ die Verarbeitung praferenz-¨
basierter Anfragen vor, der es erlaubt, ’beste Treffer’ fur¨ Benutzeranfragen zu identiﬁzieren.
Umfangreiche Evaluierungen mit Hilfe eines Simulationsframeworks fur¨ Peer-to-Peer-
Netzwerke zeigen die Efﬁzienz und Skalierbarkeit der prasentierten¨ Algorithmen.
Stichworte:Peer-to-Peer-Netzwerke, Verteilte Datenbanken, digitale Bibliotheken
ivContents
1 Introduction................................................................. 1
1.1 A Short History of Library Catalogs..................... 1
1.2 Digital Libraries ............................... 5
1.3 Problem Statement and Outline ....................... 8
2 Foundations.................................................................. 11
2.1 Relational Databases............................. 1
2.2 Distributed ............................ 16
2.3 Semantic Web ................................ 20
2.4 Peer-to-Peer Networks 27
3 Design Dimensions of Schema-Based Peer-to-Peer Networks.................... 32
3.1 Network Properties 3
3.2 Data Storage and Access........................... 36
3.3 Data Integration ............................... 37
3.4 Overview of Schema-Based P2P Algorithms and Systems ......... 38
3.5 Summary................................... 41
4 Super-Peer-Based Query Routing............................................. 42
4.1 Assumptions ................................. 43
4.2 The HyperCuP Super-Peer Topology .................... 44
4.3 Model .................................... 47
4.4 Index Structures 48
4.5 Query Routing ................................ 50
4.6 Index Updates 52
4.7 A Simulation Framework for Schema-based Peer-to-Peer Networks .... 54
4.8 Evaluation .................................. 57
v5 A Digital Library Network Prototype for Open Archives....................... 62
5.1 The Open Archives Initiative Protocol for Metadata Harvesting ...... 62
5.2 Edutella Architecture and Implementation ................. 64
5.3 A Query Exchange Language ........................ 6
5.4 OAI-P2P Architecture and 69
5.5 Experiences ................................. 71
6 Preference-based Query Evaluation for Super-Peer Networks.................. 72
6.1 Preference-based Querying for Relational Databases ............ 73
6.2 Basic Scoring Functions for Document Search ............... 74
6.3 Progressive, Preference-based SPQR .................... 7
6.4 Evaluation .................................. 82
7 Summary and Future Work................................................... 85
7.1 Summary................................... 85
7.2 Future Work ................................. 86
Appendices...................................................................... 89
B Publications 90
List of Figures................................................................... 94
List of Tables.................................................................... 95
Bibliography 96
viChapter 1
Introduction
Most experts agree that the future of libraries lies in entering the digital information realm.
By combining their experience in selection and provisioning of high-quality with
the dissemination opportunities offered by new technology, libraries will be able to play an
important role in the information age. Some even go so far to state that “digital libraries
can become the universal knowledge repositories and communication conduits of the future,
a common vehicle by which everyone will access, discuss, evaluate, and enhance information
of all forms” [76]. While this may be a bit exaggerating, we can safely assume that it will at
least become partly true. A major requirement for this vision is that digital libraries become
open not only from a social, but also from a technical point of view [174]. As long as users
searching for information have to consult each library separately, it too tedious for them to ﬁnd
relevant documents, and they will favor the classic Web as information source instead. On the
other hand, if users could search transparently on a network of interconnected library sites as
easy as they can now on the Web, they presumably would often prefer the controlled and thus
more reliable content offered by libraries and other managed digital archives.
In this chapter, we identify characteristics of libraries, especially their catalogs, from library
history and derive requirements for the envisioned search capabilities. Based on this require-
ments analysis, we conclude with the problem statement for our work, and an overview of the
remaining chapters.
1.1 A Short History of Library Catalogs
Since over 2000 years libraries and archives are formed by their main purpose: preserving
documents for later use, and thus also preserving the knowledge embedded in them. For this
aim, they need ﬁrst to ensure safe storage of these documents, and second to provide means
11.1 A Short History of Library Catalogs 2
for retrieving documents when requested. We only consider the second point in this thesis.
Already the ﬁrst famous library, the Alexandrina, in existence about from 300 BC to 400 AD,
1exhibited a structure which is still (of course now in a very developed form) prevalent :
• A unique identiﬁcation for each document was established. At the Alexandrina, the
2author’s name and a kind of title were used to identify a work .
• A catalog was maintained to decouple search for documents from the physical document
storage. With catalogs, people can look for relevant items ﬁrst, and fetch them later (or
let librarians fetch them) using their identiﬁcation. At the Alexandrina, documents were
classiﬁed into subjects, such as Drama, Laws, Philosophy, History, Medicine, Mathe-
matics, etc. Probably subcategories were also in use, but for them no ﬁrm historical
evidence exists. The catalog consisted of a document list for each category, containing
the author/semi-title pair used as identiﬁcation.
Since then, both identiﬁcation and classiﬁcation have evolved signiﬁcantly. Regarding iden-
tiﬁcation, with the advent of printed books, the now common bibliographic metadata prop-
erties publisher and publication date complemented the already established author and title
attributes. Over time, more properties were added to this list, but the mentioned core properties
are still prevailing.
Until the 19th century, each library had its own rule set for identiﬁcation. Typical examples
where rules widely varied were the issues of books with anonymous author or books published
by an institution, but without explicit author or editor. One of the driving forces behind stan-
dardization of these rules was the aim to union complete library catalogs. In the 1850s, a team
of librarians under Charles Cofﬁn Jewett started to compile a union catalog of US libraries, and
found many difﬁculties in identifying duplicates, due to differing identiﬁcation policies. Con-
sequently, he formulated a new set of uniﬁed rules which formed the starting point for further
initiatives reconciling cataloging rules. This was (and is) a very tedious and long-lasting effort.
For example, the institution-as-author issue mentioned above was resolved in the US only in
1967 with AACR (Anglo-American Cataloging Rules, [8]). While some minor differences
between identiﬁcation approaches on international level still exist, nowadays an agreement on
all substantial issues has been achieved.
The main reason these standardization efforts are so difﬁcult to perform is that no hierarchical
organization is imposed on libraries. Each library is funded locally, and between most libraries
no strong organizational ties exist. Of course, umbrella organization have been founded, but
still libraries have a tendency to cultivate their independence and autonomy.
1The description of catalog history presented here mainly follows [185].
2At that time, titles were not yet common. Therefore, typically the ﬁrst few words of the document served as
title supplement.
https://www.youscribe.com/catalogue/documents/savoirs/a-schema-based-peer-to-peer-infrastructure-for-digital-library-1429613
www.youscribe.com
crawl=CC-MAIN-2020-40
125014
application/http; msgtype=response
sha1:T6RR7BGQPY4SQG7P2FGMZDUGKIXIZYKB
<urn:uuid:0faaaf9d-e1ad-4bd1-9d67-a9f60e0dd6d6>
2020-09-21T10:46:05Z
35.186.237.217
application/xhtml+xml
sha1:IFP5SFPF4E327UUKRGBAVHCQSZRPPLZD
<urn:uuid:341393db-d159-4634-b97c-2908e1701c8b>
https://www.youscribe.com/catalogue/documents/savoirs/a-schema-based-peer-to-peer-infrastructure-for-digital-library-1429613
response
<urn:uuid:b3fb2890-0fb4-489c-8260-0f6bc31b8b00>
isPartOf: CC-MAIN-2020-40
publisher: Common Crawl
description: Wide crawl of the web for September 2020
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-205.ec2.internal
software: Apache Nutch 1.17 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/