
Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.



BDFL - Memuneh
52102 posts

Uber Geek
+1 received by user: 6003

Administrator
Trusted
Geekzone
Subscriber

Topic # 112592 13-Dec-2012 14:05 Send private message

Just received:


This email is to inform you of the NLNZ intent to undertake its 3rd Whole of Domain Web Harvest.

The planned dates for this exercise are the 4th to the 22nd of February 2013.

The harvest will be undertaken by the Internet Archive on our behalf. More details can be found on the web harvest website:  http://natlib.govt.nz/publishers-and-authors/web-harvesting/2013-nz-web-harvest 


Note that the robot "NLNZ_IAHarvester2013" will obey standard robots.txt instructions. Need more information? Hit the link in the quote.





Create new topic
659 posts

Ultimate Geek
+1 received by user: 76

Subscriber

  Reply # 732065 13-Dec-2012 14:36 Send private message

I find it interesting that the harvest is being done by Archive.org.
I am guessing that the wayback machine will have an entry for every .nz site for Feb next year then!




Create new topic




Twitter »
Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.
http://www.geekzone.co.nz/forums.asp?ForumId=86&TopicId=112592
www.geekzone.co.nz
crawl=CC-MAIN-2015-22
30801
application/http; msgtype=response
sha1:E2NYCD2ZA5EKXVVUU4FNZ4RFJ6ZZQ5RS
<urn:uuid:b8a4bdf2-e42c-41e0-8d9e-3b0cd149655e>
2015-05-23T05:49:04Z
104.20.6.176
sha1:PDU4ZZ5ULFTA2S5W6QTY2O3WRL6VPOHX
<urn:uuid:9b78db10-b8d0-4bc4-ac91-2675a9ae9c9f>
http://www.geekzone.co.nz/forums.asp?ForumId=86&TopicId=112592
length
response
<urn:uuid:37ddd48e-e075-4183-b6a4-d77629df407b>
robots: classic
hostname: ip-10-180-206-219.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2015-22
operator: CommonCrawl Admin
description: Wide crawl of the web for May 2015
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf