View Single Post
Old December 29, 2012, 09:11 PM   #5
James K
Staff
 
Join Date: March 17, 1999
Posts: 19,682
I have never done it, and don't plan to, but it is nice to know it can be done and the post was interesting. I have a feeling it is not quite that simple, though.

Jim
__________________
Jim K
James K is offline  
 
Page generated in 0.04190 seconds with 7 queries
http://thefiringline.com/forums/showpost.php?p=5343198&postcount=5
thefiringline.com
crawl=CC-MAIN-2014-42
13717
application/http; msgtype=response
sha1:I4XC7GO67EVPWULNUNQMOXVA3YCU6XB5
<urn:uuid:c262eb2f-ad21-4662-a588-7b8042b828b4>
2014-10-31T06:28:39Z
162.243.102.188
sha1:5EMW7UKJRSMFACC24GJ4PJKPJA3SMCGN
<urn:uuid:191731dd-4cde-4a3f-a28d-47122bcda438>
http://thefiringline.com/forums/showpost.php?p=5343198&postcount=5
response
<urn:uuid:d151a578-1ab3-4736-8227-045e618b16d4>
robots: classic
hostname: ip-10-16-133-185.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2014-42
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for October 2014
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf