Suggestions

:bulb:

Full-page archiving

Store a copy of the saved page in Larder so it is permanently accessible. This becomes expensive pretty quickly, so I’m not keen on it, but I’m not averse to adding it if there’s an overwhelming call for it.

8 votes

Tagged as New feature

Created 03 May 2016 by Josh Sharp

Moved into Suggestions 13 January 2017
https://changemap.co/hellocode/larder/task/1717-full-page-archiving/
changemap.co
crawl=CC-MAIN-2018-05
11835
application/http; msgtype=response
sha1:E5C26NNBPEII7WJBL6YIK3NFDZVDX5CM
<urn:uuid:5b698d2e-7fb3-4feb-9777-481eab2f00c9>
2018-01-22T20:12:03Z
139.162.43.194
text/html
sha1:7N5YB5CNRQDITEXKL2BFVQTDV66QOUPL
<urn:uuid:2afcb5e3-73a3-424c-803e-f11463016dc8>
https://changemap.co/hellocode/larder/task/1717-full-page-archiving/
length
response
<urn:uuid:beccf61c-80ac-4c3d-8487-409df46c7f20>
robots: classic
hostname: ip-10-179-56-52.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2018-05
operator: Common Crawl Admin
description: Wide crawl of the web for January 2018
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf