Daily Archives: April 22, 2016


webternity
Ortelio is working with the University of Passau (Digital Libraries and Web Information Systems lab) to add new semantic features in the Webternity crawler. We expect to release the new version of the crawler by the end of August 2016. The Webternity crawler is based on a state of the […]

Webternity
http://ortelio.co.uk/2016/04/22/
ortelio.co.uk
crawl=CC-MAIN-2017-26
41169
application/http; msgtype=response
sha1:WGEMIJME7VKIKOWW25A7R2WKHMGB3QGM
<urn:uuid:ea657c79-0c6a-490c-b320-8338ab8691e3>
2017-06-24T17:13:36Z
207.38.94.46
text/html
sha1:4ABQ4ERZONUUIXBQPNLHENBQGKRHAAE3
<urn:uuid:9b35ef17-5017-4635-a842-95bfb0f88ab0>
http://ortelio.co.uk/2016/04/22/
length
response
<urn:uuid:28546169-4894-4a41-b9ce-3dc30a073c62>
robots: classic
hostname: ip-10-166-47-158.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2017-26
operator: Common Crawl Admin
description: Wide crawl of the web for June 2017
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf