After the Harvest: Preservation, Access, and Researcher Services for the 2016 End of Term Archive

This presentation discusses the End of Term Archive and methods for identifying and selecting in-scope content (including using registries, indices, and crowdsourcing URL nominations ["seeds"] through a web application called the URL Nomination Tool), new strategies for capturing web content (including crawling, browser rendering, and social media tools), access models including both an online portal as well as research datasets for use in computational analysis, and preservation data replication between partners using new export APIs and experimental tools developed as part of the IMLS-funded WASAPI project.
Date: December 13, 2016
Creator: Bailey, Jefferson; Grotke, Abigail & Phillips, Mark Edward
System: The UNT Digital Library
https://digital2.library.unt.edu/search/?t=fulltext&fq=untl_collection%3AUNTSW&fq=untl_institution%3AUNTDP&fq=str_year%3A2016&fq=str_day%3A13&display=list&sort=creator
digital2.library.unt.edu
CC-MAIN-2024-10
106221
application/http; msgtype=response
sha1:72J5VNVIJSPLWK4VS5I2NZOXBX73I7HZ
<urn:uuid:68dc67ca-581f-48ec-bed3-9dbafb1dcecf>
2024-02-21T00:39:32Z
129.120.90.158
text/html
sha1:7OLUKMW33LJOKYAPRNP2U54RL534WP3I
<urn:uuid:2207c3ef-7b41-4cb3-90b8-d9a0f36b63e3>
https://digital2.library.unt.edu/search/?t=fulltext&fq=untl_collection%3AUNTSW&fq=untl_institution%3AUNTDP&fq=str_year%3A2016&fq=str_day%3A13&display=list&sort=creator
response
<urn:uuid:8ec3ee16-b3c1-4bf2-9879-c3194b14a2f2>
isPartOf: CC-MAIN-2024-10
publisher: Common Crawl
description: Wide crawl of the web for February/March 2024
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-169
software: Apache Nutch 1.19 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.5-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/