View Single Post
  #25  
Old 4th November 2011, 10:07
skoena skoena is offline
Senior Member
 
Join Date: Sep 2009
Location: Emmen, Netherlands
Posts: 421
Thanks: 77
Thanked 14 Times in 12 Posts
Send a message via MSN to skoena Send a message via Skype™ to skoena
Default

Ok added an announcement to the feeed.
Reply With Quote
 
http://www.howtoforge.com/forums/showpost.php?p=266652&postcount=25
www.howtoforge.com
crawl=CC-MAIN-2014-42
16884
application/http; msgtype=response
sha1:7RPRKS3RXGFQQP35QX3UUX5R7K4BVX2I
<urn:uuid:ea3e85bd-ace9-496f-93e1-4f9fa5ba7f81>
2014-10-24T09:39:04Z
188.40.16.205
sha1:6OJ4SI4SM2QOIWJUSSAHEKO44U74PUHW
<urn:uuid:951cc589-24d7-491b-a8b1-9f1ef6bfb9c5>
http://www.howtoforge.com/forums/showpost.php?p=266652&postcount=25
response
<urn:uuid:433072ff-202f-4b14-badf-b7da71c38e94>
robots: classic
hostname: ip-10-16-133-185.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2014-42
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for October 2014
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf