Log in
Free trial

Twelve Point Plan for Quantitative and Qualitative Evaluation of Databases

By Miller, Ron | Computers in Libraries, July-August 2008 | Go to article overview
Save to active project

Twelve Point Plan for Quantitative and Qualitative Evaluation of Databases


Miller, Ron, Computers in Libraries


The assessment of a database's value to your library is not a simple matter. A thorough evaluation must account for many variables in order to arrive at a true measure of database value. Many vendors promote a simplified quantitative approach: calculate the cost per full text title and find out who gives you more full text titles for your money. Simple enough, but a bit too simple. There is much more to factor into the equation. We wouldn't choose our print reference purchases on a "cost-per-page" basis, so why should the database purchase be reduced to such a simplistic calculation? The quality of the library patron's user experience and the number of truly relevant results retrieved must weigh significantly in any evaluation of database value to a library.

Twelve points to consider:

Quality of Content

Quality of content must enter the calculation of database value.

Ask how are journals selected for inclusion in the database. Are journals selected by recommendation from a panel of subject experts and experienced librarians, or is there an "include everything but the kitchen sink" approach to create inflated full text numbers?

Peer-review numbers can be deceiving. Look for a database vendor that sets an unimpeachable standard when deciding which publications qualify to be designated as peer-reviewed. The database publisher should guarantee that the peer-review label means literally that an independent scholar has reviewed and recommended the article for publication. Peer-review must not be applied simply to any journal that is not a popular or trade title.

Titles Appropriate to Database Subject

Are the titles selected for inclusion in the database really relevant to the subject area? The presence of extra titles in a full text list does not provide a better user experience, rather it will tend to provide the highly unsatisfactory user experience of searching for journal articles and receiving a results set filled with inappropriate articles or even government produced pamphlets masquerading as full-text journals. Ask yourself if your library patron will be satisfied when a search of what is advertised as a complete education resource brings back articles from the likes of National Tax Journal, or Omega: Journal of Death and Dying.

Controlled Vocabulary

Is there a controlled vocabulary based on the literary warrant of the actual collection of items in the database, or has a database aggregator simply adapted a "one-size-fits-all" vocabulary specific to nothing and of little true value in bringing users all of the available items on a topic?

Relevance Ranking That Works

Is there a relevancy ranking option available, and does the relevance-ranking algorithm understand that words found in fields like subject and abstract are more indicative of the "aboutness" of an article than random occurrences elsewhere in the article?

Search and System Efficiency

Does the database require executing a second search in order to sort the articles by relevance? This provides impressive usage statistics that make the database appear to be your most popular database, but does your patron really want to execute two searches when one could bring the best results? Ask yourself if those impressive statistics are truly a measure of database value, or more likely an indication of database inefficiency.

Ease of Access to Best Results

Does the database give library patrons quick and easy access to the information that they seek whether they are novice searchers or experienced professionals? Does the results set come back sorted only by date of publication in reverse chronological order, or are they sorted with the goal of giving users the best articles available, automatically placed at the top of the results set in one search?

The rest of this article is only available to active members of Questia

Sign up now for a free, 1-day trial and receive full access to:

  • Questia's entire collection
  • Automatic bibliography creation
  • More helpful research tools like notes, citations, and highlights
  • Ad-free environment

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
Loading One moment ...
Project items
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited article

Twelve Point Plan for Quantitative and Qualitative Evaluation of Databases
Settings

Settings

Typeface
Text size Smaller Larger
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

While we understand printed pages are helpful to our users, this limitation is necessary to help protect our publishers' copyrighted material and prevent its unlawful distribution. We are sorry for any inconvenience.
Full screen

matching results for page

Cited passage

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

Cited passage

Welcome to the new Questia Reader

The Questia Reader has been updated to provide you with an even better online reading experience.  It is now 100% Responsive, which means you can read our books and articles on any sized device you wish.  All of your favorite tools like notes, highlights, and citations are still here, but the way you select text has been updated to be easier to use, especially on touchscreen devices.  Here's how:

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select.

OK, got it!

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.

Are you sure you want to delete this highlight?
http://www.questia.com/read/1G1-181155924/twelve-point-plan-for-quantitative-and-qualitative
www.questia.com
crawl=CC-MAIN-2014-10
124830
application/http; msgtype=response
sha1:VCGD4JNVVZUYXVBIZBNNTIA6QTTJTPQR
<urn:uuid:8717177a-54ca-4b8b-b1c3-d1b80103716e>
2014-03-14T00:12:41Z
69.32.208.13
sha1:SAN3XJUSMHRECCM5IQXAGZAFDKL3O2IJ
<urn:uuid:58afc424-43fc-40da-b724-276fe2697334>
http://www.questia.com/read/1G1-181155924/twelve-point-plan-for-quantitative-and-qualitative
response
<urn:uuid:8f124c58-dcf5-4c9f-a1b7-106bb97b421b>
robots: classic
hostname: ip-10-183-142-35.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2014-10
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for March 2014
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf