About | Blog | Contact | Subscribe

Grabbing the long tail of search engines

A revamped search engine from Heriot-Watt University in the UK makes the most of the long tail allowing searchers to drill way down into the most obscure, but nevertheless useful, pages across dozens of technology databases and sites.

“It’s a prerequisite for any successful search service in technology
subjects to have a ‘Long Tail’ (or large inventory), Heriot-Watt’s Roddy MacLeod explains, “This is because the majority of search queries made by technologists, or by others seeking information in technology-related subjects, tend to be very specific. It’s in the nature of the subjects, and the real information retrieval needs of those involved in these subjects, for granularity to be important.”

MacLeod heads up the team behind TechXtra http://www.techxtra.ac.uk/. This search engine aggregates content from a vast array of databases with technology-related content, and provides meta results that are essentially invisible to general search engines. “A search of TechXtra will search across more than 4 million records of various kinds – articles, technical reports, digital theses and dissertations, books, eprints, news items, job announcements, video, learning & teaching resources, key websites, and more – most of which relate to technology subjects,” MacLeod adds.

You can restrict searches to a particular format (technical reports, or
articles, or books, and so on), or select only specific databases among those listed with the ubiquitous Advanced Search option.

Among the most interesting of the databases for the Youtube generation is perhaps the Open Video Project, which is a growing repository of digitized video. And serious tech research can provide the full text of thousands of theses, eprints from arXiv, earthquake engineering
technical reports from Caltech, and almost half a million articles in computer and information science from CiteSeer. In addition many articles in Digital Open Access Journals (DOAJ), are available.

I’ve added a TechXtra link to the Sciencebase science search toolbox and will upgrade that to a fully-fledged search link in the next few days.

One thing which is a fairly recent development is the number of freely available full text digitised theses that TechXtra can now access. “These theses are the results of considerable research,” MacLeod told me, “they can be excellent resources on specialised topics.

HW University also produces the superb Internet Resources Newsletter.

Photos | Music | Book
http://www.sciencebase.com/science-blog/grabbing-the-long-tail-of-search-engines.html
www.sciencebase.com
crawl=CC-MAIN-2014-23
54756
application/http; msgtype=response
sha1:NR4J3LFQ5LEY3DN46UPA7QCSQAPWOSMQ
<urn:uuid:d4dfe11b-6d64-483e-a3d3-cc7a52762916>
2016-02-11T00:30:34Z
192.185.16.138
sha1:RHUVJJVJFE3GDT2FCOVWQNJHBHOZ3G6D
<urn:uuid:7e594cc2-4e3e-4e82-9514-64d55398f07a>
http://www.sciencebase.com/science-blog/grabbing-the-long-tail-of-search-engines.html
response
<urn:uuid:7465041e-f83d-4e41-bc75-ddaba3dd87c1>
robots: classic
hostname: ip-10-236-182-209.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2016-07
operator: CommonCrawl Admin
description: Wide crawl of the web for February 2016
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf