Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | Go Back
View Post [edit]

Poster: pegs223 Date: Oct 1, 2008 10:04pm
Forum: general Subject: How is website decided

How is it decided which websites to archive? Is some of it just automatic or is permission asked/granted? Can you contact a Webmaster?

Reply to this post
Reply [edit]

Poster: kustota Date: Oct 2, 2008 1:23am
Forum: general Subject: Re: How is website decided

http://www.archive.org/about/faqs.php#The_Wayback_Machine
http://archive.org/post/211987/how-is-website-decided
archive.org
crawl=CC-MAIN-2014-41
16506
application/http; msgtype=response
sha1:VPRIFNEYJ4VFYT3HLWMBME4LHRICYJAG
<urn:uuid:9ad071c1-de65-4942-a73c-f5f06ea6a7db>
2014-09-20T22:38:14Z
207.241.224.2
sha1:RGVDZSIMU63DKG2EIV3WOTRWX7PBOGJF
<urn:uuid:c866a3b2-9a8c-4f53-8458-f795f743f233>
http://archive.org/post/211987/how-is-website-decided
length
response
<urn:uuid:59f6b080-b57b-403c-bb51-0ba8d44ae0d0>
robots: classic
hostname: ip-10-196-40-205.us-west-1.compute.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2014-41
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for September 2014
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf