Computing and Information Services CIS Home Page Brown Home Page STG Projects section
<faculty grants>     <projects>     <about>     <staff>     <publications>    <search>

Typographic Regularization in the WWP Textbase 


STG lead(s): Jacque Russom

In materials printed before 1630, the letters v, u, j, and i did not have the values that they have today. The word that we spell ivory was written iuory, the word that we spell jury was written iury, etc. This difference in typographical conventions makes early texts more difficult to read and compromises the matching of forms in information retrieval tasks.

The Women Writers Project uses SGML tagging to encode a regularized spelling for such typographical variants, thereby allowing the option to display and search on either the original form or the regularized form. Encoding this information by hand was time-consuming and inefficient, since a great many high-frequency words require such tagging (e.g, haue, loue, iudge).

STG undertook to develop a system to automatically tag words subject to this typographic convention with the regularized form. To a great extent, pattern matching rules based on the linguistic principles for English consonants and vowels can be invoked to identify and tag words appropriately. These rules are supplemented by programs that recognize SGML markup indicating such things as word division across a line break, errors or abbreviations within a word, or structural elements to be excluded from regularization.  

The Brown University Women Writers Project's main undertaking is an SGML-encoded full-text database of pre-Victorian women's writing in English. This collection currently includes nearly 200 texts representing a broad cross-section of the literate culture of pre-Victorian Britain. The WWP supports teaching and research in a wide variety of disciplines such as English, history, women's studies, comparative literature, and religious studies. 

Principal Investigator(s) or Parent Project Lead(s):
Allen Renear
Scholarly Technology Group, Brown University 

Research domains: humanities computing 
Type: report 
Components and related resources:
Brown University Women Writers Project Home Page  http://www.wwp.brown.edu 
Links:
Typographic Regularization in the WWP Textbase  http://www.nyu.edu/its/humanities/ach_allc2001/papers/russom/index.html 

STG involvement initiated: September 1998   Status:maintenance 
  Funding support: Computing and Information Services
Contact:STG_info@brown.edu 

  Record last modified: 13-Sep-2005 


Home | Information Technology

Home

Quick Links

Announcements

Nov. 27 - 5:30 pm

If you are having trouble connecting your device (computer, smartphone, etc.) to Brown-Secure wireless or would like to report a coverage issue, please stop by one of our upcoming clinics:

  • Thursday 11/...
Nov. 20 - 4:26 pm


On Thursday, November 21, Brown's Shibboleth login screens are getting a new look. Though you may not know it by name, you have probably used Shibboleth to log in to many access-restricted Brown websites and...

Nov. 4 - 2:29 pm

Google updated its look recently, changing the top menu bar. We've gotten a lot of questions about how to find some apps and features. Here's where to look:

Finding Apps
Previously, you...
http://www.stg.brown.edu/projects/indexcard/displaycard.php3?card=19
www.stg.brown.edu
crawl=CC-MAIN-2013-48
37905
application/http; msgtype=response
sha1:6CYQT6ZH3R2FDEGVL3ANA2SRV7IPYVMO
<urn:uuid:08326232-1913-4bb0-88a5-11ed2a4790a2>
2013-12-06T16:51:40Z
128.148.252.67
sha1:5EHT7XFGDOQNV6UCQ6LUEPXBU26PSCHQ
<urn:uuid:f1dd954b-923a-4fec-8bfd-d581a608ea91>
http://www.stg.brown.edu/projects/indexcard/displaycard.php3?card=19
length
response
<urn:uuid:18c7a1bc-bc3f-4f40-8dbe-2544f185fd50>
robots: classic
hostname: ip-10-33-133-15.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2013-48
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for Winter 2013
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf