Wednesday, May 12, 2010

Site Surfing #2: Exploring the Deep Web



Ask anyone what to do when searching for a term, pictures, news, word definition, map and guides, etc., and expect the answer is to Google it. The popular search engine is closely synonymous to the World Wide Web. However, it's stored information is nowhere what the Web really has to offer.

Ever heard of the Deep Web? It is part of the online world where no search engines has entered. In other words standard search engines such as Google and Yahoo! do not have indexes of these possible info-rich pages.

Naturally one would ask how to dig this mine of information. I discovered by chance this list of search engines that specializes in crawling the Deep Web. What you may initially think as unavailable in the web might just be lurking around in the corner.

Some of these search engines are:


Databases of academic journals, electronic books, bulletin boards, mailing lists, online library card catalogs, articles, and many other resources await where not even Google Scholar haven't found.

Be aware that finding that oh-so-precious knowledge is just as hard as looking for a needle in a haystack; you need to know the proper search terms to get results you want. 
The web around Wikipedia. Click to enlarge.
http://www.conanhughes.com/2010/05/site-surfing-2-exploring-deep-web.html
www.conanhughes.com
crawl=CC-MAIN-2017-13
142711
application/http; msgtype=response
sha1:J2QOAEGQLYVW5KDJPTNVKFUO7XTP6ONN
<urn:uuid:de485f7a-9245-43a0-b923-ca0fc52fc904>
2017-03-24T15:58:40Z
172.217.3.51
sha1:M5HBXUWBYGSSL2SUYRAOU5E7HX3ASI44
<urn:uuid:0d4a014d-b3ae-409e-a7ce-1758456e79db>
http://www.conanhughes.com/2010/05/site-surfing-2-exploring-deep-web.html
response
<urn:uuid:e8b90391-d92e-480a-b9f2-b678d6cf0073>
robots: classic
hostname: ip-10-233-31-227.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2017-13
operator: CommonCrawl Admin
description: Wide crawl of the web for March 2017
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf