Re: Challenging assertions

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Thu, 13 Mar 2008 14:07:50 +0000

On Thu, 13 Mar 2008, Peter Crowther wrote:

> > From: Stevan Harnad
> >
> > (3) Do you really believe that in the online age, any prefabricated
> > classification system could ever beat boolean full-text search?
>
> Depends. How much automation am I allowed to have in my prefabrication,
> how often can I update it, and at *exactly* which task(s) am I supposed
> to beat Boolean full-text search? I can conceive of tasks at which
> dynamically-generated classification systems can beat text search, but
> these systems certainly aren't of the coarse grain of Dewey or LCC and
> I wouldn't expect humans to enter or update the terms.

I think Peter's point is relevant and correct:

What I meant by "prefabricated classification" was something like
Dewey hand-tagging of articles at source (the IR), a-priori.

Of course boolean search alone is beaten by boolean search PLUS
dynamically-generated supplements such as Google Page Rank, download
stats, user tags or full-text-based classification algorithms.

But not by Dewey-tagging at source; and I doubt that even boolean search
PLUS Dewey pre-tagging produces incremental gains worth the pre-tagging
effort.

(I also bet you algorithms applied to the full-text database a-posteriori,
as a whole, could generate a pseudo-Dewey classification scheme
by computation alone that was at least as good as individual Dewey
hand-tagging a-priori.)

Stevan Harnad
Received on Thu Mar 13 2008 - 14:08:45 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:49:16 GMT
https://www.southampton.ac.uk/~harnad/Hypermail/Amsci/7021.html
www.southampton.ac.uk
crawl=CC-MAIN-2019-26
7500
application/http; msgtype=response
sha1:KP6EDYQALCW4FPLQ3G3KCBKKZJEMSYTN
<urn:uuid:a2e01edc-7c10-482c-8c88-639bbf4772e1>
2019-06-20T15:53:21Z
152.78.118.52
application/xhtml+xml
sha1:AMDFFAZMWNIHIJYIS24J6RBRIVPRGW6S
<urn:uuid:1ea8744f-652e-4541-a202-d3411e593720>
https://www.southampton.ac.uk/~harnad/Hypermail/Amsci/7021.html
response
<urn:uuid:887b389b-bc73-4c74-aec3-4fd67f3a75c6>
isPartOf: CC-MAIN-2019-26
publisher: Common Crawl
description: Wide crawl of the web for June 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-187-26-184.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/