LIWA – Living Web Archives

The PoWR project identified a number of technical challenges which made certain types of content – particularly that with a Web 2.0 flavour – particularly difficult to manage and preserve in an effective way. My attention has recently been drawn to an EU-funded project which hopes to overcome a number of these technical problems, as well as others that are applicable to large-scale archiving such as the problem of spam content.

LIWA – Living Web Archives – began in early 2008, but as with many EU projects, its startup phase involved a lot of internal activity without much of a public face. As a result we didn’t pick up on its work in the JISC-PoWR handbook, but I’m sure we’ll rectify this omission in any future revisions.

To pick one example of LIWA’s areas of interest, it intends to develop tools which make it easier to take a temporal view of web archives and to maintain temporal consistency. Temporal consistency – or rather its absence – will be familiar to anyone who has spent time exploring sites in the Internet Archive, where different pages, or even portions of the same page (such as images) will have been archived on different days. This can lead to occasional surprises when navigating through archived content, with links taking one to pages that don’t have the expected content.

LIWA’s partner’s include Hanzo, a UK-based web archive services company that we covered briefly in the handbook; I hope we can explore their potential value to UK HE in the future.

1 thought on “LIWA – Living Web Archives

  1. Mark Middleton

    Thanks for highlighting our work in LiWA.

    Your readers may be interested to know that Hanzo is also working on a couple of open source web archiving projects.

    WARC Tools are a collection of command line and web tools for the creation and manipulation of ISO 28500 web archive (WARC) files, funded by IIPC.

    Search Tools adds full-text and metadata search capabilities to WARC Tools, funded by JISC. The HE angle on this is that Search Tools is a deliverable of the World Wide Web of Humanities project, a collaboration with Oxford Internet Institute in UK and Internet Archive in US.

    More details and source code for these projects can be found here:
    * WARC Tools: http://code.google.com/p/warc-tools/
    * Search Tools: http://code.google.com/p/search-tools/

Comments are closed.
https://jiscpowr.jiscinvolve.org/wp/2009/03/06/liwa-living-web-archives/
jiscpowr.jiscinvolve.org
crawl=CC-MAIN-2022-49
53708
application/http; msgtype=response
sha1:IUCLYOGDI7A36C6LLAA6KQP2IXPPBP2R
<urn:uuid:5eebf6a2-2aa4-4b51-a9db-7fa86f1b7751>
2022-11-27T03:02:18Z
34.253.169.74
text/html
sha1:VVPUTD7R3N3ZMEOHGLLP7RE34UIKHDDM
<urn:uuid:50bc3144-6753-43e8-bf2d-c29366920d07>
https://jiscpowr.jiscinvolve.org/wp/2009/03/06/liwa-living-web-archives/
response
<urn:uuid:03f51dd4-1e21-441b-8dca-d2c2c01061cc>
isPartOf: CC-MAIN-2022-49
publisher: Common Crawl
description: Wide crawl of the web for November/December 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-77
software: Apache Nutch 1.19 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.4-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/