Diving into Digital Ephemera: Identifying Defunct URLs in the Web Archives
Olivia Meehan, who worked on the web archiving team at the US Library of Congress, evaluates how well online archives of the Papal Transition 2005 Collection from 2005 have survived: Based on the results I have so far and conversations I’ve had with other web archivists, the lifecycle of websites is unpredictable to the extent that accurately tracking the status of a site inherently requires nuance, time, and attention — which is difficult to maintain at scale. This data is valuable, however, and is worth pursuing when possibleÂ. Using a sample selection of URLs from larger collections could make this more manageable than comprehensive reviews.

Of the content originally captured in the Papal Transition 2005 Collection, 41% is now offline. Without the archived pages, the information, perspectives, and experiences expressed on those websites would potentially be lost forever. They include blogs, personal websites, individually-maintained web portals, and annotated bibliographies. They frequently represent small voices and unique perspectives that may be overlooked or under-represented by large online publications with the resources to maintain legacy pages and articles.

The internet is impermanent in a way that is difficult to quantify. The constant creation of new information obscures what is routinely deleted, overwritten, and lost. While the scope of this project is small within the context of the wider internet, and even within the context of the Library’s Web Archive collections as a whole, I hope that it effectively demonstrates the value of web archives in preserving snapshots of the online world as it moves and changes at a record pace.

Read more of this story at Slashdot.

By admin
https://phishing.ws/index.php/2022/08/06/diving-into-digital-ephemera-identifying-defunct-urls-in-the-web-archives/security-world-news/admin/?doing_wp_cron=1660022355.2245950698852539062500
phishing.ws
crawl=CC-MAIN-2022-33
70644
application/http; msgtype=response
sha1:JRJHLG52XCPP3FZB2SXUWQFBAZ4T7UNB
<urn:uuid:87c7a2b9-0be6-4615-8271-97b042027812>
2022-08-09T05:19:41Z
148.72.152.192
text/html
sha1:4NPF7KXCWLKGDKZU252KMDWUVYYOCMYK
<urn:uuid:ed696075-e77d-4043-9a64-fccf31f6fc2f>
https://phishing.ws/index.php/2022/08/06/diving-into-digital-ephemera-identifying-defunct-urls-in-the-web-archives/security-world-news/admin/?doing_wp_cron=1660022355.2245950698852539062500
response
<urn:uuid:d02810df-ae41-4a29-9390-7914db2a4d19>
isPartOf: CC-MAIN-2022-33
publisher: Common Crawl
description: Wide crawl of the web for August 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-115
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.4-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/