Applying a Hybrid Query Translation Method to Japanese/English Cross-Language Patent Retrieval
\Large Masatoshi Fukui$^{\dagger}$~~Shigeto Higuchi$^{\dagger}$~~Youichi Nakatani$^{\dagger}$~~\medskip
\Large Masao Tanaka$^{\dagger}$~~Atsushi Fujii$^{\ddagger}$~~Tetsuya Ishikawa$^{\ddagger}$
This paper applies an existing query translation method to cross-language patent retrieval. In our method, multiple dictionaries are used to derive all possible translations for an input query, and collocational statistics are used to resolve translation ambiguity. We used Japanese/English parallel patent abstracts to perform comparative experiments, where our method outperformed a simple dictionary-based query translation method, and achieved 76\% of monolingual retrieval in terms of average precision.
a4params.tex
\textwidth=16cm%15cm
\textheight=23cm%22cm
\topmargin=-.3in%0cm
%\headsep=1cm
%\headheight=1cm
%\footheight=1cm
%\footskip=1cm
\oddsidemargin=0cm%1cm
\evensidemargin=0cm%1cm
%\marginparwidth=2.5pt
%\columnsep=1.2cm
%\itemsep=0.5pt

main.tex
%%
%% SIGIR-2000 WS on patent retrieval
%%
\documentstyle[11pt,twocolumn]{article}

\input{a4params.tex}

%% Title/Author/Affiliation

\title{\bf Applying a Hybrid Query Translation Method to
Japanese/English Cross-Language Patent Retrieval}

\author{\Large Masatoshi Fukui$^{\dagger}$~~Shigeto
Higuchi$^{\dagger}$~~Youichi Nakatani$^{\dagger}$~~\medskip \\
\Large Masao Tanaka$^{\dagger}$~~Atsushi Fujii$^{\ddagger}$~~Tetsuya
Ishikawa$^{\ddagger}$}

\date{$^\dagger$Japan Patent Information Organization \\
Satoh Daiya Bldg., 1-7 Toyo 4-Chome Koto-ku 135-0016, JAPAN \medskip \\
$^\ddagger$University of Library and Information Science \\
1-2 Kasuga Tsukuba 305-8550, JAPAN \\ \smallskip
\smallskip {\tt E-mail:~fujii@ulis.ac.jp}}

\newcommand{\etal}{et~al.}
\newcommand{\etaleos}{et~al}
\newcommand{\eq}[1]{(\ref{#1})}

\input{psfig.tex}

\pagestyle{empty}

\begin{document}

\maketitle\thispagestyle{empty}

\begin{abstract}
  This paper applies an existing query translation method to
  cross-language patent retrieval. In our method, multiple
  dictionaries are used to derive all possible translations for an
  input query, and collocational statistics are used to resolve
  translation ambiguity. We used Japanese/English parallel patent
  abstracts to perform comparative experiments, where our method
  outperformed a simple dictionary-based query translation method, and
  achieved 76\% of monolingual retrieval in terms of average
  precision.
\end{abstract}

\section{Introduction}
\label{sec:introduction}

Since 1978, JAPIO (Japan Patent Information Organization) has operated
PATOLIS, which is one of the first on-line patent retrieval services
in Japan, and currently provides clients (i.e., 8,000 Japanese
companies) with patent information from 62 countries and 5
international organizations. At the same time, since a patent obtained
in a single country can be protected in multiple countries
simultaneously, it is feasible that users are interested in retrieving
patent information across languages. Motivated by this background,
JAPIO manually summarizes each patent document submitted in Japan into
approximately 400 characters, and translates the summarized documents
into English, which are provided on PAJ (Patent Abstract of Japan)
CD-ROMs\footnote{Copyright by Japan Patent Office.}.

In this paper, we target cross-language information retrieval (CLIR)
in the context of patent retrieval, and evaluate its effectiveness
using Japanese/English patent abstracts on PAJ CD-ROMs.

In brief, existing CLIR systems are classified into three approaches:
(a) translating queries into the document
language~\cite{ballesteros:sigir-98,davis:sigir-97}, (b) translating
documents into the query language~\cite{mccarley:acl-99,oard:amta-98},
and (c) representing both queries and documents in a
language-independent
space~\cite{carbonell:ijcai-97,gonzalo:chum-98,littman:clir-98,salton:jasis-70}.
However, since developing a CLIR system is expensive, we used the CLIR
system proposed by Fujii and
Ishikawa~\cite{fujii:ntcir-99,fujii:emnlp-vlc-99}, which follows the
first approach.

This system has partially been developed for the NACSIS test
collection~\cite{kando:sigir-99}, which consists of 39 Japanese
queries and approximately 330,000 technical abstracts in Japanese and
English.  However, since patent information usually includes technical
terms, it is expected that this system also will perform reasonably
for patent abstracts.

\section{System Description}
\label{sec:system}

Figure~\ref{fig:system} depicts the overall design of our CLIR system,
in which we combine a query translation module and an IR engine for
monolingual retrieval.  Unlike the original system proposed by Fujii
and Ishikawa~\cite{fujii:ntcir-99,fujii:emnlp-vlc-99} targeting the
NACSIS collection, we use the JAPIO collection for the target
documents. Here, the JAPIO collection is a subset of PAJ CD-ROMs. We
will elaborate on this collection in
Section~\ref{sec:experimentation}. In this section, we briefly explain
the retrieval process based on Figure~\ref{fig:system}.

First, query translation is performed for the source language query to
output the translation. For this purpose, a hybrid method integrating
multiple resources is used. To put it more precisely, the EDR
technical/general dictionaries~\cite{edr:95} are used to derive all
possible translation candidates for words and phrases included in the
source query. In addition, for words unlisted in dictionaries,
transliteration is performed to identify phonetic equivalents in the
target language.

Then, bi-gram statistics extracted from NACSIS documents in the target
language are used to resolve the translation ambiguity. Ideally,
bi-gram statistics should be extracted from the JAPIO
collection. However, since the number of documents in this collection
is relatively small, when compared with the NACSIS collection (see
Section~\ref{sec:experimentation}), we avoided the data sparseness
problem.

Since our system is bidirectional between Japanese and English, we
tokenize documents with different methods, depending on their
language. For English documents, the tokenization involves eliminating
stopwords and identifying root forms for inflected content words. For
this purpose, we use WordNet~\cite{fellbaum:wordnet-98}, which
contains a stopword list and correspondences between inflected words
and their root form.

On the other hand, we segment Japanese documents into lexical units
using the ChaSen morphological analyzer~\cite{matsumoto:chasen-97},
which has commonly been used for much Japanese NLP research, and
extract content words based on their part-of-speech information.

Second, the IR engine searches the JAPIO collection for documents
relevant to the translated query, and sorts them according to the
degree of relevance, in descending order. Our IR engine is based on
the vector space model, in which the similarity between the query and
each document (i.e., the degree of relevance of each document) is
computed as the cosine of the angle between their associated
vectors. We use the notion of TF$\cdot$IDF for term weighting. Among a
number of variations of term weighting
methods~\cite{salton:ipm-88,zobel:sigir-forum-98}, we tentatively use
the formulae as shown in Equation~\eq{eq:tf_idf}.
\begin{equation}
  \label{eq:tf_idf}
  \begin{array}{lll}
    TF & = & 1 + \log(f_{t,d}) \\
    \noalign{\vskip 1.2ex}
    IDF & = & \log(\frac{\textstyle N}{\textstyle n_{t}})
  \end{array}
\end{equation}
Here, $f_{t,d}$ denotes the frequency that term $t$ appears in
document $d$, and $n_{t}$ denotes the number of documents containing
term $t$. $N$ is the total number of documents in the collection.

For the indexing process, we first tokenize documents as explained
above (i.e., we use WordNet and ChaSen for English and Japanese
documents, respectively), and then conduct the word-based
indexing. That is, we use each content word as a single indexing term.

Finally, since retrieved documents are not in the user's native
language, we optionally use a machine translation system to enhance
readability of retrieved documents.

\begin{figure}[t]
  \begin{center}
    \leavevmode
    \psfig{file=system.eps,height=1.3in}
  \end{center}
  \caption{The overall design of our cross-language patent retrieval
    system.}
  \label{fig:system}
\end{figure}

\section{Experimentation}
\label{sec:experimentation}

Since no test collection for Japanese/English patent retrieval is
available to the public, we produced our test collection (i.e., the
JAPIO collection), which consists of three Japanese queries and
Japanese/English comparable abstracts.

Each query, which was manually produced, consists of the description
and narrative, and corresponds to different domains, i.e., electrical
engineering, mechanical engineering and chemistry.
Figure~\ref{fig:query} shows the three query descriptions in the
second column.

\begin{figure*}[htbp]
  \begin{center}
    \leavevmode
    \small
    \begin{tabular}{llrr} \hline\hline
      {\hfill\centering IPC\hfill} & {\hfill\centering
      Description\hfill} & \#Relevant & \#Documents \\ \hline
      electronics & GPS car navigation system based on VICS & 930 &
      7,526 \\
      mechanics & eliminating dioxin in burning solid wastes & 451 &
      8,214 \\
      chemistry & antibacterial plastic combining inorganic
      materials & 473 & 5,902 \\
      \hline
    \end{tabular}
    \caption{Query descriptions in the JAPIO collection.}
    \label{fig:query}
  \end{center}
\end{figure*}

In conventional test collections, relevance assessment is usually
performed based on the pooling method~\cite{voorhees:sigir-98}, which
first pools candidates for relevant documents using multiple retrieval
systems. However, since in our case only one system described in
Section~\ref{sec:system} is currently available, a different
production method was needed.

To put it more precisely, for each query (domain), target documents
were first collected based on the IPC classification number, from PAJ
CD-ROMs in 1993-1998. Then, for each query, three professional human
searchers, who were allowed to enhance queries based on thesauri and
their introspection, searched the target documents for relevant
documents.

Thus, in practice, the JAPIO collection consists of three different
document collections corresponding to each query.  In
Figure~\ref{fig:query}, the third and fourth columns denote the number
of relevant documents and the total number of target documents for
each query.

We compared the following methods:
\begin{itemize}
\item Japanese-English CLIR, where all possible translations derived
  from EDR dictionaries and the transliteration method were used as
  query terms (JEALL),
\item Japanese-English CLIR, where disambiguation based on bi-gram
  statistics were performed, and $k$-best translations were used as
  query terms (JEDIS),
\item Japanese-Japanese monolingual IR (JJ).
\end{itemize}
Here, we empirically set \mbox{$k=1$}. Although the performance of
JEDIS did not significantly differ as long as we set a small value of
$k$ (e.g., \mbox{$k=5$}), we achieved the best performance when we set
\mbox{$k=1$}.

Figure~\ref{fig:rp} shows recall-precision curves for the above three
methods, where JEDIS generally outperformed JEALL, and JJ generally
outperformed both JEALL and JEDIS, regardless of the recall.  The
difference between JEALL and JEDIS is attributed to the fact that
JEDIS resolved translation ambiguity based on bi-gram statistics
extracted from the NACSIS collection. Thus, we can conclude that the
use of bi-gram statistics (even extracted from a collection other than
the JAPIO collection) was effective for the query translation.

Table~\ref{tab:avg_pre} shows the non-interpolated average precision
values, averaged over the three queries, for each method.  This table
shows that JJ outperformed JEALL and JEDIS, JEDIS outperformed JEALL,
and the average precision value for JEDIS was 76\% of that obtained
with JJ.

These results are also observable in existing CLIR experiments using
the TREC and NACSIS collections. Thus, we conclude that our
cross-language patent retrieval system is relatively comparable with
those for newspaper articles and technical abstracts in performance.

However, we could not conduct statistical testing, which investigates
whether the difference in average precision is meaningful or simply
due to chance~\cite{hull:sigir-93}, because the number of queries is
small. We concede that experiments using a larger number of queries
need to be further explored.

\section{Conclusion}
\label{sec:conclusion}

In this paper, we explored Japanese/English cross-language patent
retrieval. For this purpose, we used an existing cross-language IR
system relying on a hybrid query translation method, and evaluated its
effectiveness using Japanese queries and English patent abstracts.
The experimental results paralleled existing experiments. That is, we
found that resolving translation ambiguity was effective for the query
translation, and that the average precision value for cross-language
IR was approximately 76\% of that obtained with monolingual IR.
Future work will include qualitative/quantitative analyses based on a
larger number of queries.

\begin{figure}[t]
  \begin{center}
    \leavevmode
    \psfig{file=rp-curve.ps,height=3.2in}
  \end{center}
  \caption{Recall-precision curves for different methods.}
  \label{fig:rp}
\end{figure}

\begin{table}[htbp]
  \begin{center}
    \caption{Non-interpolated average precision values,
    averaged over the three queries, for different methods.}
    \medskip
    \leavevmode
    \small
    \begin{tabular}{lcc} \hline\hline
      Method & Avg. Precision & Ratio to JJ \\ \hline
      JJ & 0.4151 & -- \\
      JEDIS & 0.3156 & 0.7603 \\
      JEALL & 0.2709 & 0.6526 \\
      \hline
    \end{tabular}
    \label{tab:avg_pre}
  \end{center}
\end{table}

\small

\bibliographystyle{jplain}

\begin{thebibliography}{10}

\bibitem{ballesteros:sigir-98}
Lisa Ballesteros and W.~Bruce Croft.
\newblock Resolving ambiguity for cross-language retrieval.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp. 64--71,
  1998.

\bibitem{carbonell:ijcai-97}
Jaime~G. Carbonell, Yiming Yang, Robert~E. Frederking, Ralf~D. Brown, Yibing
  Geng, and Danny Lee.
\newblock Translingual information retrieval: A comparative evaluation.
\newblock In {\em Proceedings of the 15th International Joint Conference on
  Artificial Intelligence}, pp. 708--714, 1997.

\bibitem{davis:sigir-97}
Mark~W. Davis and William~C. Ogden.
\newblock {QUILT}: Implementing a large-scale cross-language text retrieval
  system.
\newblock In {\em Proceedings of the 20th Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp. 92--98,
  1997.

\bibitem{fellbaum:wordnet-98}
Christiane Fellbaum, editor.
\newblock {\em {WordNet}: An Electronic Lexical Database}.
\newblock MIT Press, 1998.

\bibitem{fujii:ntcir-99}
Atsushi Fujii and Tetsuya Ishikawa.
\newblock Cross-language information retrieval at {ULIS}.
\newblock In {\em Proceedings of the 1st NTCIR Workshop on Research in Japanese
  Text Retrieval and Term Recognition}, pp. 163--169, 1999.

\bibitem{fujii:emnlp-vlc-99}
Atsushi Fujii and Tetsuya Ishikawa.
\newblock Cross-language information retrieval for technical documents.
\newblock In {\em Proceedings of the Joint ACL SIGDAT Conference on Empirical
  Methods in Natural Language Processing and Very Large Corpora}, pp. 29--37,
  1999.

\bibitem{gonzalo:chum-98}
Julio Gonzalo, Felisa Verdejo, Carol Peters, and Nicoletta Calzolari.
\newblock Applying {EuroWordNet} to cross-language text retrieval.
\newblock {\em Computers and the Humanities}, Vol.~32, pp. 185--207, 1998.

\bibitem{hull:sigir-93}
David Hull.
\newblock Using statistical testing in the evaluation of retrieval experiments.
\newblock In {\em Proceedings of the 16th Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp.
  329--338, 1993.

\bibitem{edr:95}
{Japan Electronic Dictionary Research Institute}.
\newblock {EDR} electronic dictionary technical guide, 1995.
\newblock (In Japanese).

\bibitem{kando:sigir-99}
Noriko Kando, Kazuko Kuriyama, and Toshihiko Nozue.
\newblock {NACSIS} test collection workshop ({NTCIR-1}).
\newblock In {\em Proceedings of the 22nd Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp.
  299--300, 1999.

\bibitem{littman:clir-98}
Michael~L. Littman, Susan~T. Dumais, and Thomas~K. Landauer.
\newblock Automatic cross-language information retrieval using latent semantic
  indexing.
\newblock In Gregory Grefenstette, editor, {\em Cross-Language Information
  Retrieval}, chapter~5, pp. 51--62. Kluwer Academic Publishers, 1998.

\bibitem{matsumoto:chasen-97}
Yuji Matsumoto, Akira Kitauchi, Tatsuo Yamashita, Osamu Imaichi, and Tomoaki
  Imamura.
\newblock {Japanese} morphological analysis system {ChaSen} manual.
\newblock Technical Report NAIST-IS-TR97007, NAIST, 1997.
\newblock (In Japanese).

\bibitem{mccarley:acl-99}
J.~Scott McCarley.
\newblock Should we translate the documents or the queries in cross-language
  information retrieval?
\newblock In {\em Proceedings of the 37th Annual Meeting of the Association for
  Computational Linguistics}, pp. 208--214, 1999.

\bibitem{oard:amta-98}
Douglas~W. Oard.
\newblock A comparative study of query and document translation for
  cross-language information retrieval.
\newblock In {\em Proceedings of the 3rd Conference of the Association for
  Machine Translation in the Americas}, pp. 472--483, 1998.

\bibitem{salton:jasis-70}
Gerard Salton.
\newblock Automatic processing of foreign language documents.
\newblock {\em Journal of the American Society for Information Science},
  Vol.~21, No.~3, pp. 187--194, 1970.

\bibitem{salton:ipm-88}
Gerard Salton and Christopher Buckley.
\newblock Term-weighting approaches in automatic text retrieval.
\newblock {\em Information Processing \& Management}, Vol.~24, No.~5, pp.
  513--523, 1988.

\bibitem{voorhees:sigir-98}
Ellen~M. Voorhees.
\newblock Variations in relevance judgments and the measurement of retrieval
  effectiveness.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp.
  315--323, 1998.

\bibitem{zobel:sigir-forum-98}
Justin Zobel and Alistair Moffat.
\newblock Exploring the similarity space.
\newblock {\em ACM SIGIR FORUM}, Vol.~32, No.~1, pp. 18--34, 1998.

\end{thebibliography}

\end{document}

% Local Variables:
% mode: japanese-LaTeX
% TeX-master: t
% End:

psfig.tex
% Psfig/TeX Release 1.2
%
% Archive users note: this is an out-of-date version, preserved because future
% versions are backwards incompatible. Use psfig.sty for the up-to-date
% version.
%
% dvips version
%
% All software, documentation, and related files in this distribution of
% psfig/tex are Copyright 1987, 1988 Trevor J. Darrell
%
% Permission is granted for use and non-profit distribution of psfig/tex 
% providing that this notice be clearly maintained, but the right to
% distribute any portion of psfig/tex for profit or as part of any commercial
% product is specifically reserved for the author.
%
% $Header: psfig.tex,v 1.9 88/01/08 17:42:01 trevor Exp $
% $Source: $
%
% Thanks to Greg Hager (GDH) and Ned Batchelder for their contributions
% to this project.
%
\catcode`\@=11\relax
\newwrite\@unused
\def\typeout#1{{\let\protect\string\immediate\write\@unused{#1}}}
\typeout{psfig/tex 1.2-dvips}


%% Here's how you define your figure path.  Should be set up with null
%% default and a user useable definition.

\def\figurepath{./}
\def\psfigurepath#1{\edef\figurepath{#1}}

%
% @psdo control structure -- similar to Latex @for.
% I redefined these with different names so that psfig can
% be used with TeX as well as LaTeX, and so that it will not 
% be vunerable to future changes in LaTeX's internal
% control structure,
%
\def\@nnil{\@nil}
\def\@empty{}
\def\@psdonoop#1\@@#2#3{}
\def\@psdo#1:=#2\do#3{\edef\@psdotmp{#2}\ifx\@psdotmp\@empty \else
    \expandafter\@psdoloop#2,\@nil,\@nil\@@#1{#3}\fi}
\def\@psdoloop#1,#2,#3\@@#4#5{\def#4{#1}\ifx #4\@nnil \else
       #5\def#4{#2}\ifx #4\@nnil \else#5\@ipsdoloop #3\@@#4{#5}\fi\fi}
\def\@ipsdoloop#1,#2\@@#3#4{\def#3{#1}\ifx #3\@nnil 
       \let\@nextwhile=\@psdonoop \else
      #4\relax\let\@nextwhile=\@ipsdoloop\fi\@nextwhile#2\@@#3{#4}}
\def\@tpsdo#1:=#2\do#3{\xdef\@psdotmp{#2}\ifx\@psdotmp\@empty \else
    \@tpsdoloop#2\@nil\@nil\@@#1{#3}\fi}
\def\@tpsdoloop#1#2\@@#3#4{\def#3{#1}\ifx #3\@nnil 
       \let\@nextwhile=\@psdonoop \else
      #4\relax\let\@nextwhile=\@tpsdoloop\fi\@nextwhile#2\@@#3{#4}}
% 
%
\def\psdraft{
	\def\@psdraft{0}
	%\typeout{draft level now is \@psdraft \space . }
}
\def\psfull{
	\def\@psdraft{100}
	%\typeout{draft level now is \@psdraft \space . }
}
\psfull
\newif\if@prologfile
\newif\if@postlogfile
\newif\if@noisy
\def\pssilent{
	\@noisyfalse
}
\def\psnoisy{
	\@noisytrue
}
\psnoisy
%%% These are for the option list.
%%% A specification of the form a = b maps to calling \@p@@sa{b}
\newif\if@bbllx
\newif\if@bblly
\newif\if@bburx
\newif\if@bbury
\newif\if@height
\newif\if@width
\newif\if@rheight
\newif\if@rwidth
\newif\if@clip
\newif\if@verbose
\def\@p@@sclip#1{\@cliptrue}

%%% GDH 7/26/87 -- changed so that it first looks in the local directory,
%%% then in a specified global directory for the ps file.

\def\@p@@sfile#1{\def\@p@sfile{null}%
	        \openin1=#1
		\ifeof1\closein1%
		       \openin1=\figurepath#1
			\ifeof1\typeout{Error, File #1 not found}
			\else\closein1
			    \edef\@p@sfile{\figurepath#1}%
                        \fi%
		 \else\closein1%
		       \def\@p@sfile{#1}%
		 \fi}
\def\@p@@sfigure#1{\def\@p@sfile{null}%
	        \openin1=#1
		\ifeof1\closein1%
		       \openin1=\figurepath#1
			\ifeof1\typeout{Error, File #1 not found}
			\else\closein1
			    \def\@p@sfile{\figurepath#1}%
                        \fi%
		 \else\closein1%
		       \def\@p@sfile{#1}%
		 \fi}

\def\@p@@sbbllx#1{
		%\typeout{bbllx is #1}
		\@bbllxtrue
		\dimen100=#1
		\edef\@p@sbbllx{\number\dimen100}
}
\def\@p@@sbblly#1{
		%\typeout{bblly is #1}
		\@bbllytrue
		\dimen100=#1
		\edef\@p@sbblly{\number\dimen100}
}
\def\@p@@sbburx#1{
		%\typeout{bburx is #1}
		\@bburxtrue
		\dimen100=#1
		\edef\@p@sbburx{\number\dimen100}
}
\def\@p@@sbbury#1{
		%\typeout{bbury is #1}
		\@bburytrue
		\dimen100=#1
		\edef\@p@sbbury{\number\dimen100}
}
\def\@p@@sheight#1{
		\@heighttrue
		\dimen100=#1
   		\edef\@p@sheight{\number\dimen100}
		%\typeout{Height is \@p@sheight}
}
\def\@p@@swidth#1{
		%\typeout{Width is #1}
		\@widthtrue
		\dimen100=#1
		\edef\@p@swidth{\number\dimen100}
}
\def\@p@@srheight#1{
		%\typeout{Reserved height is #1}
		\@rheighttrue
		\dimen100=#1
		\edef\@p@srheight{\number\dimen100}
}
\def\@p@@srwidth#1{
		%\typeout{Reserved width is #1}
		\@rwidthtrue
		\dimen100=#1
		\edef\@p@srwidth{\number\dimen100}
}
\def\@p@@ssilent#1{ 
		\@verbosefalse
}
\def\@p@@sprolog#1{\@prologfiletrue\def\@prologfileval{#1}}
\def\@p@@spostlog#1{\@postlogfiletrue\def\@postlogfileval{#1}}
\def\@cs@name#1{\csname #1\endcsname}
\def\@setparms#1=#2,{\@cs@name{@p@@s#1}{#2}}
%
% initialize the defaults (size the size of the figure)
%
\def\ps@init@parms{
		\@bbllxfalse \@bbllyfalse
		\@bburxfalse \@bburyfalse
		\@heightfalse \@widthfalse
		\@rheightfalse \@rwidthfalse
		\def\@p@sbbllx{}\def\@p@sbblly{}
		\def\@p@sbburx{}\def\@p@sbbury{}
		\def\@p@sheight{}\def\@p@swidth{}
		\def\@p@srheight{}\def\@p@srwidth{}
		\def\@p@sfile{}
		\def\@p@scost{10}
		\def\@sc{}
		\@prologfilefalse
		\@postlogfilefalse
		\@clipfalse
		\if@noisy
			\@verbosetrue
		\else
			\@verbosefalse
		\fi
}
%
% Go through the options setting things up.
%
\def\parse@ps@parms#1{
	 	\@psdo\@psfiga:=#1\do
		   {\expandafter\@setparms\@psfiga,}}
%
% Compute bb height and width
%
\newif\ifno@bb
\newif\ifnot@eof
\newread\ps@stream
\def\bb@missing{
	\if@verbose{
		\typeout{psfig: searching \@p@sfile \space  for bounding box}
	}\fi
	\openin\ps@stream=\@p@sfile
	\no@bbtrue
	\not@eoftrue
	\catcode`\%=12
	\loop
		\read\ps@stream to \line@in
		\global\toks200=\expandafter{\line@in}
		\ifeof\ps@stream \not@eoffalse \fi
		%\typeout{ looking at :: \the\toks200 }
		\@bbtest{\toks200}
		\if@bbmatch\not@eoffalse\expandafter\bb@cull\the\toks200\fi
	\ifnot@eof \repeat
	\catcode`\%=14
}	
\catcode`\%=12
\newif\if@bbmatch
\def\@bbtest#1{\expandafter\@a@\the#1%%BoundingBox:\@bbtest\@a@}
\long\def\@a@#1%%BoundingBox:#2#3\@a@{\ifx\@bbtest#2\@bbmatchfalse\else\@bbmatchtrue\fi}
\long\def\bb@cull#1 #2 #3 #4 #5 {
	\dimen100=#2 bp\edef\@p@sbbllx{\number\dimen100}
	\dimen100=#3 bp\edef\@p@sbblly{\number\dimen100}
	\dimen100=#4 bp\edef\@p@sbburx{\number\dimen100}
	\dimen100=#5 bp\edef\@p@sbbury{\number\dimen100}
	\no@bbfalse
}
\catcode`\%=14
%
\def\compute@bb{
		\no@bbfalse
		\if@bbllx \else \no@bbtrue \fi
		\if@bblly \else \no@bbtrue \fi
		\if@bburx \else \no@bbtrue \fi
		\if@bbury \else \no@bbtrue \fi
		\ifno@bb \bb@missing \fi
		\ifno@bb \typeout{FATAL ERROR: no bb supplied or found}
			\no-bb-error
		\fi
		%
		\count203=\@p@sbburx
		\count204=\@p@sbbury
		\advance\count203 by -\@p@sbbllx
		\advance\count204 by -\@p@sbblly
		\edef\@bbw{\number\count203}
		\edef\@bbh{\number\count204}
		%\typeout{ bbh = \@bbh, bbw = \@bbw }
}
%
% \in@hundreds performs #1 * (#2 / #3) correct to the hundreds,
%	then leaves the result in @result
%
\def\in@hundreds#1#2#3{\count240=#2 \count241=#3
		     \count100=\count240	% 100 is first digit #2/#3
		     \divide\count100 by \count241
		     \count101=\count100
		     \multiply\count101 by \count241
		     \advance\count240 by -\count101
		     \multiply\count240 by 10
		     \count101=\count240	%101 is second digit of #2/#3
		     \divide\count101 by \count241
		     \count102=\count101
		     \multiply\count102 by \count241
		     \advance\count240 by -\count102
		     \multiply\count240 by 10
		     \count102=\count240	% 102 is the third digit
		     \divide\count102 by \count241
		     \count200=#1\count205=0
		     \count201=\count200
			\multiply\count201 by \count100
		 	\advance\count205 by \count201
		     \count201=\count200
			\divide\count201 by 10
			\multiply\count201 by \count101
			\advance\count205 by \count201
			%
		     \count201=\count200
			\divide\count201 by 100
			\multiply\count201 by \count102
			\advance\count205 by \count201
			%
		     \edef\@result{\number\count205}
}
\def\compute@wfromh{
		% computing : width = height * (bbw / bbh)
		\in@hundreds{\@p@sheight}{\@bbw}{\@bbh}
		%\typeout{ \@p@sheight * \@bbw / \@bbh, = \@result }
		\edef\@p@swidth{\@result}
		%\typeout{w from h: width is \@p@swidth}
}
\def\compute@hfromw{
		% computing : height = width * (bbh / bbw)
		\in@hundreds{\@p@swidth}{\@bbh}{\@bbw}
		%\typeout{ \@p@swidth * \@bbh / \@bbw = \@result }
		\edef\@p@sheight{\@result}
		%\typeout{h from w : height is \@p@sheight}
}
\def\compute@handw{
		\if@height 
			\if@width
			\else
				\compute@wfromh
			\fi
		\else 
			\if@width
				\compute@hfromw
			\else
				\edef\@p@sheight{\@bbh}
				\edef\@p@swidth{\@bbw}
			\fi
		\fi
}
\def\compute@resv{
		\if@rheight \else \edef\@p@srheight{\@p@sheight} \fi
		\if@rwidth \else \edef\@p@srwidth{\@p@swidth} \fi
}
%		
% Compute any missing values
\def\compute@sizes{
	\compute@bb
	\compute@handw
	\compute@resv
}
%
% \psfig
% usage : \psfig{file=, height=, width=, bbllx=, bblly=, bburx=, bbury=,
%			rheight=, rwidth=, clip=}
%
% "clip=" is a switch and takes no value, but the `=' must be present.
\def\psfig#1{\vbox {
	% do a zero width hard space so that a single
	% \psfig in a centering enviornment will behave nicely
	%{\setbox0=\hbox{\ }\ \hskip-\wd0}
	%
	\ps@init@parms
	\parse@ps@parms{#1}
	\compute@sizes
	%
	\ifnum\@p@scost<\@psdraft{
		\if@verbose{
			\typeout{psfig: including \@p@sfile \space }
		}\fi
		%
		\special{ps::[begin] 	\@p@swidth \space \@p@sheight \space
				\@p@sbbllx \space \@p@sbblly \space
				\@p@sbburx \space \@p@sbbury \space
				startTexFig \space }
		\if@clip{
			\if@verbose{
				\typeout{(clip)}
			}\fi
			\special{ps:: doclip \space }
		}\fi
		\if@prologfile
		    \special{ps: plotfile \@prologfileval \space } \fi
		\special{ps: plotfile \@p@sfile \space }
		\if@postlogfile
		    \special{ps: plotfile \@postlogfileval \space } \fi
		\special{ps::[end] endTexFig \space }
		% Create the vbox to reserve the space for the figure
		\vbox to \@p@srheight true sp{
			\hbox to \@p@srwidth true sp{
				\hss
			}
		\vss
		}
	}\else{
		% draft figure, just reserve the space and print the
		% path name.
		\vbox to \@p@srheight true sp{
		\vss
			\hbox to \@p@srwidth true sp{
				\hss
				\if@verbose{
					\@p@sfile
				}\fi
				\hss
			}
		\vss
		}
	}\fi
}}
\def\psglobal{\typeout{psfig: PSGLOBAL is OBSOLETE; use psprint -m instead}}
\catcode`\@=12\relax


cs0206034.extracted_bib
\begin{thebibliography}{10}

\bibitem{ballesteros:sigir-98}
Lisa Ballesteros and W.~Bruce Croft.
\newblock Resolving ambiguity for cross-language retrieval.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp. 64--71,
  1998.

\bibitem{carbonell:ijcai-97}
Jaime~G. Carbonell, Yiming Yang, Robert~E. Frederking, Ralf~D. Brown, Yibing
  Geng, and Danny Lee.
\newblock Translingual information retrieval: A comparative evaluation.
\newblock In {\em Proceedings of the 15th International Joint Conference on
  Artificial Intelligence}, pp. 708--714, 1997.

\bibitem{davis:sigir-97}
Mark~W. Davis and William~C. Ogden.
\newblock {QUILT}: Implementing a large-scale cross-language text retrieval
  system.
\newblock In {\em Proceedings of the 20th Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp. 92--98,
  1997.

\bibitem{fellbaum:wordnet-98}
Christiane Fellbaum, editor.
\newblock {\em {WordNet}: An Electronic Lexical Database}.
\newblock MIT Press, 1998.

\bibitem{fujii:ntcir-99}
Atsushi Fujii and Tetsuya Ishikawa.
\newblock Cross-language information retrieval at {ULIS}.
\newblock In {\em Proceedings of the 1st NTCIR Workshop on Research in Japanese
  Text Retrieval and Term Recognition}, pp. 163--169, 1999.

\bibitem{fujii:emnlp-vlc-99}
Atsushi Fujii and Tetsuya Ishikawa.
\newblock Cross-language information retrieval for technical documents.
\newblock In {\em Proceedings of the Joint ACL SIGDAT Conference on Empirical
  Methods in Natural Language Processing and Very Large Corpora}, pp. 29--37,
  1999.

\bibitem{gonzalo:chum-98}
Julio Gonzalo, Felisa Verdejo, Carol Peters, and Nicoletta Calzolari.
\newblock Applying {EuroWordNet} to cross-language text retrieval.
\newblock {\em Computers and the Humanities}, Vol.~32, pp. 185--207, 1998.

\bibitem{hull:sigir-93}
David Hull.
\newblock Using statistical testing in the evaluation of retrieval experiments.
\newblock In {\em Proceedings of the 16th Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp.
  329--338, 1993.

\bibitem{edr:95}
{Japan Electronic Dictionary Research Institute}.
\newblock {EDR} electronic dictionary technical guide, 1995.
\newblock (In Japanese).

\bibitem{kando:sigir-99}
Noriko Kando, Kazuko Kuriyama, and Toshihiko Nozue.
\newblock {NACSIS} test collection workshop ({NTCIR-1}).
\newblock In {\em Proceedings of the 22nd Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp.
  299--300, 1999.

\bibitem{littman:clir-98}
Michael~L. Littman, Susan~T. Dumais, and Thomas~K. Landauer.
\newblock Automatic cross-language information retrieval using latent semantic
  indexing.
\newblock In Gregory Grefenstette, editor, {\em Cross-Language Information
  Retrieval}, chapter~5, pp. 51--62. Kluwer Academic Publishers, 1998.

\bibitem{matsumoto:chasen-97}
Yuji Matsumoto, Akira Kitauchi, Tatsuo Yamashita, Osamu Imaichi, and Tomoaki
  Imamura.
\newblock {Japanese} morphological analysis system {ChaSen} manual.
\newblock Technical Report NAIST-IS-TR97007, NAIST, 1997.
\newblock (In Japanese).

\bibitem{mccarley:acl-99}
J.~Scott McCarley.
\newblock Should we translate the documents or the queries in cross-language
  information retrieval?
\newblock In {\em Proceedings of the 37th Annual Meeting of the Association for
  Computational Linguistics}, pp. 208--214, 1999.

\bibitem{oard:amta-98}
Douglas~W. Oard.
\newblock A comparative study of query and document translation for
  cross-language information retrieval.
\newblock In {\em Proceedings of the 3rd Conference of the Association for
  Machine Translation in the Americas}, pp. 472--483, 1998.

\bibitem{salton:jasis-70}
Gerard Salton.
\newblock Automatic processing of foreign language documents.
\newblock {\em Journal of the American Society for Information Science},
  Vol.~21, No.~3, pp. 187--194, 1970.

\bibitem{salton:ipm-88}
Gerard Salton and Christopher Buckley.
\newblock Term-weighting approaches in automatic text retrieval.
\newblock {\em Information Processing \& Management}, Vol.~24, No.~5, pp.
  513--523, 1988.

\bibitem{voorhees:sigir-98}
Ellen~M. Voorhees.
\newblock Variations in relevance judgments and the measurement of retrieval
  effectiveness.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pp.
  315--323, 1998.

\bibitem{zobel:sigir-forum-98}
Justin Zobel and Alistair Moffat.
\newblock Exploring the similarity space.
\newblock {\em ACM SIGIR FORUM}, Vol.~32, No.~1, pp. 18--34, 1998.

\end{thebibliography}