Loading…
FORCE2019 has ended
Back To Schedule
Wednesday, October 16 • 10:30am - 11:00am
Perpetual access machines: archiving web-published scholarship at scale

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
In 2018, the Internet Archive undertook a large-scale project to build as complete a collection as possible of scholarly outputs published on the web, as well as to improve the discoverability and accessibility of scholarly works archived as part of these global web harvests. This project involved a number of areas of work: targeted archiving of known OA publications (especially at-risk “long tail” publications); extraction and augmentation of bibliographic metadata and full text; integration and preservation of related identifier, registry, and aggregation services and datastores; partnerships with affiliated initiatives and joint service developments; and creation of new tools and machine learning approaches for identifying archived scholarly work in existing born-digital and web collections. The project also identified and archived associated research outputs such as blogs, datasets, code repositories and other secondary research objects. The beta API and public interface - code-named "fatcat" - can be found at https://fatcat.wiki/.

Project leads will talk about the project’s current status and upcoming work, focusing on content acquisition, indexing, discoverability, the role of machine learning, service provisioning, and their collaborative work with libraries, publishers, and non-profits. Conceptually, the project demonstrates that the scalability and technologies of "archiving the web" can facilitate automated ingest, enrichment, and dissemination strategies for a variety of web-published primary and secondary scholarly record types that have traditionally been collected via more custom and manual workflows. The project strategic goal is to provide open infrastructure for the perpetual discoverability of and access to archived scholarship.

Speakers
avatar for Jefferson Bailey

Jefferson Bailey

Director, Web Archiving & Data Services, Internet Archive
Jefferson Bailey is Director of Web Archiving & Data Services at Internet Archive. Jefferson joined Internet Archive in Summer 2014 and manages Internet Archive’s web archiving services including Archive-It, used by over 600 institutions to preserve the web. He also oversees web... Read More →


Wednesday October 16, 2019 10:30am - 11:00am BST
Plenary Room
https://force2019.sched.com/event/U2mp/perpetual-access-machines-archiving-web-published-scholarship-at-scale?linkback=grid
force2019.sched.com
crawl=CC-MAIN-2021-04
35162
application/http; msgtype=response
sha1:KCXZWQC3HIFG4T3VVGMBF5SRIFGLOODE
<urn:uuid:af902456-0c62-48e6-b84a-42fb241ffc24>
2021-01-17T10:32:08Z
108.62.103.29
application/xhtml+xml
sha1:2X36AQK5FMCBL3LSRLXVSWYAK5WPJKDD
<urn:uuid:bd3a7d1f-6257-48f7-b33b-8a983eb365ab>
https://force2019.sched.com/event/U2mp/perpetual-access-machines-archiving-web-published-scholarship-at-scale?linkback=grid
response
<urn:uuid:de242ff7-e1b6-44d8-8029-d4f14f4a4e1a>
isPartOf: CC-MAIN-2021-04
publisher: Common Crawl
description: Wide crawl of the web for January 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-99.ec2.internal
software: Apache Nutch 1.17 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/