6.5cm
Atsushi Fujii\thanks{~~~The first and second authors are also members of CREST, Japan Science and Technology Corporation.}
Information Science
1-2 Kasuga
Tsukuba
305-8550
Japan
Advanced Industrial
Science and Technology
1-1-1 Chuuou Daini Umezono
305-8568
While recent retrieval techniques do not limit the number of index terms, out-of-vocabulary (OOV) words are crucial in speech recognition. Aiming at retrieving information with spoken queries, we fill the gap between speech recognition and text retrieval in terms of the vocabulary size. Given a spoken query, we generate a transcription and detect OOV words through speech recognition. We then correspond detected OOV words to terms indexed in a target collection to complete the transcription, and search the collection for documents relevant to the completed transcription. We show the effectiveness of our method by way of experiments.
main.tex
\documentclass[11pt]{article}
\usepackage{acl2002}
\usepackage{times}
\usepackage{latexsym}
\setlength\titlebox{6.5cm}    % Expanding the titlebox

\title{A Method for Open-Vocabulary Speech-Driven Text Retrieval}

\author{Atsushi Fujii\thanks{~~~The first and second authors
       are also members of CREST, Japan Science and Technology
       Corporation.}\\
  University of Library and\\
  Information Science\\
  1-2 Kasuga, Tsukuba\\
  305-8550, Japan\\
  {\tt fujii@ulis.ac.jp} \And
  Katunobu Itou\\
  National Institute of\\
  Advanced Industrial\\
  Science and Technology\\
  1-1-1 Chuuou Daini Umezono\\
  Tsukuba, 305-8568, Japan\\
  {\tt itou@ni.aist.go.jp} \And
  Tetsuya Ishikawa\\
  University of Library and\\
  Information Science\\
  1-2 Kasuga, Tsukuba\\
  305-8550, Japan\\
  {\tt ishikawa@ulis.ac.jp}}

\date{}

\newcommand{\etal}{et~al.}
\newcommand{\etaleos}{et~al}
\newcommand{\eq}[1]{(\ref{#1})}
\renewcommand{\nocite}[1]{\shortcite{#1}}
\input{psfig.tex}

\begin{document}
\maketitle
\begin{abstract}
  While recent retrieval techniques do not limit the number of index
  terms, out-of-vocabulary (OOV) words are crucial in speech
  recognition. Aiming at retrieving information with spoken queries,
  we fill the gap between speech recognition and text retrieval in
  terms of the vocabulary size. Given a spoken query, we generate a
  transcription and detect OOV words through speech recognition. We
  then correspond detected OOV words to terms indexed in a target
  collection to complete the transcription, and search the collection
  for documents relevant to the completed transcription. We show the
  effectiveness of our method by way of experiments.
\end{abstract}

\section{Introduction}
\label{sec:introduction}

Automatic speech recognition, which decodes human voice to generate
transcriptions, has of late become a practical technology. It is
feasible that speech recognition is used in real-world human language
applications, such as information retrieval.

Initiated partially by TREC-6, various methods have been proposed for
``spoken document retrieval (SDR),'' in which written queries are used to
search speech archives for relevant
information~\cite{garofolo:trec-97}.  State-of-the-art SDR methods,
where speech recognition error rate is 20-30\%, are comparable with
text retrieval methods in performance~\cite{jourlin:sc-2000}, and thus
are already practical. Possible rationales include that recognition
errors are overshadowed by a large number of words correctly
transcribed in target documents.

However, ``speech-driven retrieval,'' where spoken queries are used to
retrieve (textual) information, has not fully been explored, although
it is related to numerous keyboard-less applications, such as
telephone-based retrieval, car navigation systems, and user-friendly
interfaces.

Unlike spoken document retrieval, speech-driven retrieval is still a
challenging task, because recognition errors in short queries
considerably decrease retrieval accuracy.  A number of references
addressing this issue can be found in past research literature.

Barnett~\etal~\shortcite{barnett:eurospeech-97}  and
Crestani~\shortcite{crestani:fqas-2000} independently performed
comparative experiments related to speech-driven retrieval, where the
DRAGON speech recognition system was used as an input interface for
the INQUERY text retrieval system.  They used as test queries 35
topics in the TREC collection, dictated by a single male speaker.
However, these cases focused on improving text retrieval methods and
did not address problems in improving speech recognition.  As a
result, errors in recognizing spoken queries (error rate was
approximately 30\%) considerably decreased the retrieval accuracy.

Although we showed that the use of target document collections in
producing language models for speech recognition significantly
improved the performance of speech-driven
retrieval~\cite{fujii:springer-2002,itou:asru-2001}, a number of
issues still remain open questions.

Section~\ref{sec:problem} clarifies problems addressed in this paper.
Section~\ref{sec:overview} overviews our speech-driven text retrieval
system. Sections~\ref{sec:speech_recognition}-\ref{sec:query_completion}
elaborate on our methodology.  Section~\ref{sec:experimentation}
describes comparative experiments, in which an existing IR test
collection was used to evaluate the effectiveness of our
method. Section~\ref{sec:related_work} discusses related research
literature.

\section{Problem Statement}
\label{sec:problem}

One major problem in speech-driven retrieval is related to
out-of-vocabulary (OOV) words.

On the one hand, recent IR systems do not limit the vocabulary size
(i.e., the number of index terms), and can be seen as open-vocabulary
systems, which allow users to input any keywords contained in a target
collection.  It is often the case that a couple of million terms are
indexed for a single IR system.

On the other hand, state-of-the-art speech recognition systems still
need to limit the vocabulary size (i.e., the number of words in a
dictionary), due to problems in estimating statistical language
models~\cite{young:ieee-spm-1996} and constraints associated with
hardware, such as memories. In addition, computation time is crucial
for a real-time usage, including speech-driven retrieval.  In view of
these problems,  for many languages the vocabulary size is limited to
a couple of ten
thousands~\cite{itou:jas-1999,paul:darpa-ws-1992,steeneken:eurospeech-1995},
which is incomparably smaller than the size of indexes for practical
IR systems.

In addition, high-frequency words, such as functional words and common
nouns, are usually included in dictionaries and recognized with a high
accuracy. However, those words are not necessarily useful for
retrieval.  On the contrary, low-frequency words appearing in specific
documents are often effective query terms.

To sum up, the OOV problem is inherent in speech-driven retrieval, and
we need to fill the gap between speech recognition and text retrieval
in terms of the vocabulary size. In this paper, we propose a method to
resolve this problem aiming at open-vocabulary speech-driven retrieval.

\section{System Overview}
\label{sec:overview}

Figure~\ref{fig:system} depicts the overall design of our
speech-driven text retrieval system, which consists of speech
recognition, text retrieval and query completion modules.  Although
our system is currently implemented for Japanese, our methodology is
language-independent.  We explain the retrieval process based on this
figure.

Given a query spoken by a user, the speech recognition module uses a
dictionary and acoustic/language models to generate a transcription of
the user speech. During this process, OOV words, which are not listed
in the dictionary, are also detected.  For this purpose, our language
model includes both words and syllables so that OOV words are
transcribed as sequences of syllables.

For example, in the case where ``{\it kankitsu\/}~(citrus)'' is not
listed in the dictionary, this word should be transcribed as
/\verb|ka N ki tsu|/. However, it is possible that this word is mistakenly
transcribed, such as /\verb|ka N ke tsu|/ and /\verb|ka N ke tsu ke ko|/.

To improve the quality of our system, these syllable sequences have to
be transcribed as {\em words\/}, which is one of the central issues in
this paper. In the case of speech-driven retrieval, where users
usually have specific information needs, it is feasible that users
utter contents related to a target collection. In other words, there
is a great possibility that detected OOV words can be identified as
index terms that are phonetically identical or similar.

However, since a) a single sound can potentially correspond to more
than one word (i.e., homonyms) and b) searching the entire collection
for phonetically identical/similar terms is prohibitive,  we need an
efficient disambiguation method. Specifically, in the case of
Japanese,  the homonym problem is multiply crucial because words
consist of different character types, i.e., ``{\it kanji},'' ``{\it
  katakana},'' ``{\it hiragana},'' alphabets and other characters like
numerals\footnote{In Japanese, {\it kanji\/} (or Chinese character) is
  the idiogram, and {\it katakana\/} and {\it hiragana\/} are
  phonograms.}.

To resolve this problem, we use a two-stage retrieval method. In the
first stage, we delete OOV words from the transcription, and perform
text retrieval using remaining words, to obtain a specific number of
top-ranked documents according to the degree of relevance.  Even if
speech recognition is not perfect, these documents are potentially
associated with the user speech more than the entire collection.
Thus, we search only these documents for index terms corresponding to
detected OOV words.

Then, in the second stage, we replace detected OOV words with
identified index terms so as to complete the transcription, and
re-perform text retrieval to obtain final outputs. However, we do not
re-perform speech recognition in the second stage.

In the above example, let us assume that the user also utters words
related to ``{\it kankitsu\/}~(citrus),'' such as ``{\it
  orenji\/}~(orange)'' and ``{\it remon\/}~(lemon),'' and that these
words are correctly recognized as words. In this case, it is possible
that retrieved documents contain the word ``{\it
  kankitsu\/}~(citrus).'' Thus, we replace the syllable sequence
/\verb|ka N ke tsu|/ in the query with ``{\it kankitsu\/},'' which is
additionally used as a query term in the second stage.

It may be argued that our method resembles the notion of
pseudo-relevance feedback (or local feedback) for IR, where documents
obtained in the first stage are used to expand query terms, and final
outputs are refined in the second stage~\cite{kwok:sigir-98}.
However, while relevance feedback is used to improve only the
retrieval accuracy, our method improves the speech recognition and
retrieval accuracy.

\begin{figure}[htbp]
  \begin{center}
    \leavevmode \psfig{file=system.eps,height=2.3in}
  \end{center}
  \caption{The overall design of our speech-driven text retrieval system.}
  \label{fig:system}
\end{figure}

\section{Speech Recognition}
\label{sec:speech_recognition}

The speech recognition module generates word sequence $W$, given phone
sequence $X$.  In a stochastic speech recognition
framework~\cite{bahl:ieee-tpami-1983}, the task is to select the $W$
maximizing $P(W|X)$, which is transformed as in Equation~\eq{eq:bayes}
through the Bayesian theorem.
\begin{equation}
  \label{eq:bayes}
  \arg\max_{W}P(W|X) = \arg\max_{W}P(X|W)\cdot P(W)
\end{equation}
Here, $P(X|W)$ models a probability that word sequence $W$ is
transformed into phone sequence $X$, and $P(W)$ models a probability
that $W$ is linguistically acceptable. These factors are usually
called acoustic and language models, respectively.

For the speech recognition module, we use the Japanese dictation
toolkit~\cite{kawahara:icslp-2000}\footnote{http://winnie.kuis.kyoto-u.ac.jp/dictation},
which includes the ``Julius'' recognition engine and acoustic/language
models. The acoustic model was produced by way of the ASJ speech
database (ASJ-JNAS)~\cite{itou:98:a,itou:jas-1999}, which contains
approximately 20,000 sentences uttered by 132 speakers including the
both gender groups.

This toolkit also includes development softwares so that acoustic and
language models can be produced and replaced depending on the
application.  While we use the acoustic model provided in the toolkit,
we use a new language model including both words and syllables.  For
this purpose, we used the ``ChaSen'' morphological
analyzer\footnote{http://chasen.aist-nara.ac.jp} to extract words from
ten years worth of ``Mainichi Shimbun'' newspaper articles (1991-2000).

Then, we selected 20,000 high-frequency words to produce a dictionary.
At the same time, we segmented remaining lower-frequency words into
syllables based on the Japanese phonogram system. The resultant number
of syllable types was approximately 700.  Finally, we produced a
word/syllable-based trigram language model.  In other words, OOV words
were modeled as sequences of syllables. Thus, by using our language
model, OOV words can easily be detected.

In spoken document retrieval, an open-vocabulary method, which
combines recognition methods for words and syllables in target speech
documents, was also proposed~\cite{wechsler:sigir-98}.  However, this
method requires an additional computation for recognizing syllables,
and thus is expensive. In contrast, since our language model is a
regular statistical $N$-gram model, we can use the same speech
recognition framework as in Equation~\eq{eq:bayes}.

\section{Text Retrieval}
\label{sec:text_retrieval}

The text retrieval module is based on the ``Okapi'' probabilistic
retrieval method~\cite{robertson:sigir-94}, which is used to compute
the relevance score between the transcribed query and each document in
a target collection.  To produce an inverted file (i.e., an index), we
use ChaSen to extract content words from documents as terms, and
perform a word-based indexing. We also extract terms from transcribed
queries using the same method.

\section{Query Completion}
\label{sec:query_completion}

\subsection{Overview}
\label{subsec:qc_overview}

As explained in Section~\ref{sec:overview}, the basis of the query
completion module is to correspond OOV words detected by speech
recognition (Section~\ref{sec:speech_recognition}) to index terms used
for text retrieval (Section~\ref{sec:text_retrieval}). However, to
identify corresponding index terms efficiently, we limit the number of
documents in the first stage retrieval.  In principle, terms that are
indexed in top-ranked documents (those retrieved in the first stage)
and have the same sound with detected OOV words can be corresponding
terms.

However, a single sound often corresponds to multiple words.  In
addition, since speech recognition on a syllable-by-syllable basis is
not perfect, it is possible that OOV words are incorrectly
transcribed.  For example, in some cases the Japanese word ``{\it
  kankitsu\/}~(citrus)'' is transcribed as /\verb|ka N ke tsu|/.
Thus, we also need to consider index terms that are phonetically {\em
  similar\/} to OOV words.  To sum up, we need a disambiguation method
to select appropriate corresponding terms, out of a number of
candidates.

\subsection{Formalization}
\label{subsec:qc_formalization}

Intuitively, it is feasible that appropriate terms:
\begin{itemize}
\item have identical/similar sound with OOV words detected in spoken
  queries,
\item frequently appear in a top-ranked document set,
\item and appear in higher-ranked documents.
\end{itemize}
From the viewpoint of probability theory, possible representations for
the above three properties include Equation~\eq{eq:prob}, where each
property corresponds to different parameters. Our task is to select
the $t$ maximizing the value computed by this equation as the
corresponding term for OOV word $w$.
\begin{equation}
  \label{eq:prob}
  \sum_{d \in D_{q}} P(w|t) \cdot P(t|d) \cdot P(d|q)
\end{equation}
Here, $D_{q}$ is the top-ranked document set retrieved in the first
stage, given query $q$.  \mbox{$P(w|t)$} is a probability that index
term $t$ can be replaced with detected OOV word $w$, in terms of
phonetics. \mbox{$P(t|d)$} is the relative frequency of term $t$ in
document $d$. \mbox{$P(d|q)$} is a probability that document $d$ is
relevant to query $q$, which is associated with the score formalized
in the Okapi method.

However, from the viewpoint of empiricism, Equation~\eq{eq:prob} is
not necessarily effective. First, it is not easy to estimate $P(w|t)$
based on the probability theory. Second, the probability score
computed by the Okapi method is an approximation focused mainly on
{\em relative\/} superiority among retrieved documents, and thus it is
difficult to estimate $P(d|q)$ in a rigorous manner. Finally, it is
also difficult to determine the degree to which each parameter
influences in the final probability score.

In view of these problems, through preliminary experiments we
approximated Equation~\eq{eq:prob} and formalized a method to compute
the degree (not the probability) to which given index term $t$
corresponds to OOV word $w$.

First, we estimate \mbox{$P(w|t)$} by the ratio between the number of
syllables commonly included in both $w$ and $t$ and the total number
of syllables in $w$.  We use a DP matching method to identify the
number of cases related to deletion, insertion, and substitution in
$w$, on a syllable-by-syllable basis.

Second, \mbox{$P(w|t)$} should be more influential than
\mbox{$P(t|d)$} and \mbox{$P(d|q)$} in Equation~\eq{eq:prob}, although
the last two parameters are effective in the case where a large number
of candidates phonetically similar to $w$ are obtained. To decrease
the effect of \mbox{$P(t|d)$} and \mbox{$P(d|q)$}, we tentatively use
logarithms of these parameters. In addition, we use the score computed
by the Okapi method as \mbox{$P(d|q)$}.

According to the above approximation, we compute the score of $t$ as
in Equation~\eq{eq:score}.
\begin{equation}
  \label{eq:score}
  \sum_{d \in D_{q}} P(w|t) \cdot \log(P(t|d) \cdot P(d|q))
\end{equation}
It should be noted that Equation~\eq{eq:score} is independent of the
indexing method used, and therefore $t$ can be any sequences of
characters contained in $D_{q}$. In other words, any types of indexing
methods (e.g., word-based and phrase-based indexing methods) can be
used in our framework.

\subsection{Implementation}
\label{subsec:qc_implementation}

Since computation time is crucial for a real-time usage, we preprocess
documents in a target collection so as to identify candidate terms
efficiently. This process is similar to the indexing process performed
in the text retrieval module.

In the case of text retrieval, index terms are organized in an
inverted file so that documents including terms that {\em exactly\/}
match with query keywords can be retrieved efficiently.

However, in the case of query completion, terms that are included in
top-ranked documents need to be retrieved. In addition, to minimize a
score computation (for example, DP matching is time-consuming), it is
desirable to delete terms that are associated with a diminished
phonetic similarity value, \mbox{$P(w|t)$}, prior to the computation
of Equation~\eq{eq:score}.  In other words, an index file for query
completion has to be organized so that a {\em partial\/} matching
method can be used. For example, /\verb|ka N ki tsu|/ has to be
retrieved efficiently in response to /\verb|ka N ke tsu|/.

Thus, we implemented a forward/backward partial-matching method, in
which entries can be retrieved by any substrings from the first/last
characters. In addition, we index words and word-based bigrams,
because preliminary experiments showed that OOV words detected by our
speech recognition module are usually single words or short phrases,
such as ``{\it ozon-houru\/}~(ozone hole).''

\section{Experimentation}
\label{sec:experimentation}

\subsection{Methodology}
\label{subsec:ex_method}

\begin{figure*}[htbp]
  \begin{center}
    \leavevmode
    \begin{quote}
      \tt
      \footnotesize
      <TOPIC><TOPIC-ID>1001</TOPIC-ID> \\
      <DESCRIPTION>Corporate merging</DESCRIPTION> \\
      <NARRATIVE>The article describes a corporate merging and in the
      article, the name of companies have to be
      identifiable. Information
      including the field and the purpose of the merging have to be
      identifiable. Corporate merging includes corporate acquisition,
      corporate unifications and corporate buying.</NARRATIVE></TOPIC>
    \end{quote}
    \caption{An English translation for an example topic in the IREX collection.}
    \label{fig:irex_topic}
  \end{center}
\end{figure*}

To evaluate the performance of our speech-driven retrieval system, we
used the IREX
collection\footnote{http://cs.nyu.edu/cs/projects/proteus/irex/index-e.html}. This
test collection, which resembles one used in the TREC ad hoc retrieval
track, includes 30 Japanese topics (information need) and relevance
assessment (correct judgement) for each topic, along with target
documents.  The target documents are 211,853 articles collected from
two years worth of ``Mainichi Shimbun'' newspaper (1994-1995).

Each topic consists of the ID, description and narrative. While
descriptions are short phrases related to the topic, narratives
consist of one or more sentences describing the
topic. Figure~\ref{fig:irex_topic} shows an example topic in the SGML
form (translated into English by one of the organizers of the IREX
workshop).

However, since the IREX collection does not contain spoken queries, we
asked four speakers (two males/females) to dictate the narrative
field. Thus, we produced four different sets of 30 spoken queries. By
using those queries, we compared the following different methods:
\begin{enumerate}
\item text-to-text retrieval, which used written narratives as
  queries, and can be seen as a perfect speech-driven text retrieval,
\item speech-driven text retrieval, in which only words listed in the
  dictionary were modeled in the language model (in other words, the
  OOV word detection and query completion modules were not used),
\item speech-driven text retrieval, in which OOV words detected in
  spoken queries were simply deleted (in other words, the query
  completion module was not used),
\item speech-driven text retrieval, in which our method proposed in
  Section~\ref{sec:overview} was used.
\end{enumerate}
In cases of methods~2-4, queries dictated by four speakers were used
independently. Thus, in practice we compared 13 different retrieval
results.  In addition, for methods~2-4, ten years worth of {\it
  Mainichi Shimbun\/} Japanese newspaper articles (1991-2000) were
used to produce language models. However, while method~2 used only
20,000 high-frequency words for language modeling, methods~3 and 4
also used syllables extracted from lower-frequency words (see
Section~\ref{sec:speech_recognition}).

Following the IREX workshop, each method retrieved 300 top documents
in response to each query, and non-interpolated average precision
values were used to evaluate each method.

\subsection{Results}
\label{subsec:ex_results}

First, we evaluated the performance of detecting OOV words. In the 30
queries used for our evaluation, 14 word {\em tokens\/} (13 word {\em
  types\/}) were OOV words unlisted in the dictionary for speech
recognition. Table~\ref{tab:oov_evaluation} shows the results on a
speaker-by-speaker basis, where ``\#Detected'' and ``\#Correct''
denote the total number of OOV words detected by our method and the
number of OOV words correctly detected, respectively.  In addition,
``\#Completed'' denotes the number of detected OOV words that were
corresponded to correct index terms in 300 top documents.

\begin{table*}[htbp]
  \begin{center}
    \caption{Results for detecting and completing OOV words.}
    \medskip
    \leavevmode
    \small
    \begin{tabular}{lcccccc} \hline\hline
      Speaker & \#Detected & \#Correct & \#Completed & Recall &
      Precision & Accuracy \\ \hline
      Female \#1 &  51 &  9 & 18 & 0.643 & 0.176 & 0.353 \\
      Female \#2 &  56 & 10 & 18 & 0.714 & 0.179 & 0.321 \\
      Male \#1   &  33 &  9 & 12 & 0.643 & 0.273 & 0.364 \\
      Male \#2   &  37 & 12 & 16 & 0.857 & 0.324 & 0.432 \\
      \hline
      Total      & 176 & 40 & 64 & 0.714 & 0.226 & 0.362 \\
      \hline
    \end{tabular}
    \label{tab:oov_evaluation}
  \end{center}
\end{table*}

It should be noted that ``\#Completed'' was greater than ``\#Correct''
because our method often mistakenly detected words in the dictionary
as OOV words, but completed them with index terms correctly. We
estimated recall and precision for detecting OOV words, and accuracy
for query completion, as in Equation~\eq{eq:rpa}.
\begin{equation}
  \label{eq:rpa}
  \begin{array}{lll}
    recall & = & \frac{\textstyle \#Correct}{\textstyle 14} \\
    \noalign{\vskip 1.2ex}
    precision & = & \frac{\textstyle \#Correct}{\textstyle \#Detect} \\
    \noalign{\vskip 1.2ex}
    accuracy & = & \frac{\textstyle \#Completed}{\textstyle \#Detect}
  \end{array}
\end{equation}
Looking at Table~\ref{tab:oov_evaluation}, one can see that recall was
generally greater than precision. In other words, our method tended to
detect as many OOV words as possible. In addition, accuracy of
query completion was relatively low.

Figure~\ref{fig:examples} shows example words in spoken queries,
detected as OOV words and correctly completed with index terms. In
this figure, OOV words are transcribed with syllables, where
``\verb|:|'' denotes a long vowel. Hyphens are inserted between
Japanese words, which inherently lack lexical segmentation.

\begin{figure*}[htbp]
  \begin{center}
    \small
    \begin{tabular}{lll} \hline\hline
      {\hfill\centering OOV words\hfill} &
      {\hfill\centering Index terms (syllables)\hfill} &
      {\hfill\centering English gloss\hfill} \\ \hline
      /\verb|gu re : pu ra chi na ga no|/
      & {\it gureepu-furuutsu\/}~/\verb|gu re : pu fu ru : tsu|/
      & grapefruit \\
      /\verb|ya yo i chi ta|/
      & {\it Yayoi-jidai\/}~/\verb|ya yo i ji da i|/
      & the {\it Yayoi\/} period \\
      /\verb|ni ku ku ra i su|/
      & {\it nikku-puraisu\/}~/\verb|ni q ku pu ra i su|/
      & Nick Price \\
      /\verb|be N pi|/
      & {\it benpi\/}~/\verb|be N pi|/
      & constipation \\
      \hline
    \end{tabular}
    \caption{Example words detected as OOV words and completed
      correctly by our method.}
    \label{fig:examples}
  \end{center}
\end{figure*}

Second, to evaluate the effectiveness of our query completion method
more carefully, we compared retrieval accuracy for methods~1-4 (see
Section~\ref{subsec:ex_method}).  Table~\ref{tab:avg_pre} shows
average precision values, averaged over the 30 queries, for each
method\footnote{Average precision is often used to evaluate IR
  systems, which should not be confused with evaluation measures in
  Equation~\eq{eq:rpa}.}. The average precision values of our method
(i.e., method~4) was approximately 87\% of that for text-to-text
retrieval.

By comparing methods~2-4, one can see that our method improved average
precision values of the other methods irrespective of the speaker.  To
put it more precisely, by comparing methods~3 and 4, one can see the
effectiveness of the query completion method. In addition, by
comparing methods~2 and 4, one can see that a combination of the OOV
word detection and query completion methods was effective.

It may be argued that the improvement was relatively small. However,
since the number of OOV words inherent in 30 queries was only 14, the
effect of our method was overshadowed by a large number of other
words. In fact, the number of words used as query terms for our
method, averaged over the four speakers, was 421. Since existing test
collections for IR research were not produced to explore the OOV
problem, it is difficult to derive conclusions that are statistically
valid. Experiments using larger-scale test collections where the OOV
problem is more crucial need to be further explored.

Finally, we investigated the time efficiency of our method, and found
that CPU time required for the query completion process per detected
OOV word was 3.5 seconds (AMD Athlon MP 1900+). However, an additional
CPU time for detecting OOV words, which can be performed in a
conventional speech recognition process, was not crucial.

\subsection{Analyzing Errors}
\label{subsec:error_analysis}

We manually analyzed seven cases where the average precision value of
our method was significantly lower than that obtained with method~2
(the total number of cases was the product of numbers of queries and
speakers).

Among these seven cases, in five cases our query completion method
selected incorrect index terms, although correct index terms were
included in top-ranked documents obtained with the first stage.  For
example, in the case of the query 1021 dictated by a female speaker,
the word ``{\it seido\/}~(institution)'' was mistakenly transcribed as
/\verb|se N do|/. As a result, the word ``{\it sendo\/}~(freshness),''
which is associated with the same syllable sequences, was selected as
the index term. The word ``{\it seido\/}~(institution)'' was the third
candidate based on the score computed by Equation~\eq{eq:score}. To
reduce these errors, we need to enhance the score computation.

In another case, our speech recognition module did not correctly
recognize words in the dictionary, and decreased the retrieval
accuracy.

In the final case, a fragment of a narrative sentence consisting of
ten words was detected as a single OOV word. As a result, our method,
which can complete up to two word sequences, mistakenly processed that
word, and decreased the retrieval accuracy.  However, this case was
exceptional. In most cases, functional words, which were recognized
with a high accuracy, segmented OOV words into shorter fragments.

\begin{table}[htbp]
  \begin{center}
    \caption{Non-interpolated average precision values, averaged over
      30 queries, for different methods.}
    \medskip
    \leavevmode
    \small
    \tabcolsep=4pt
    \begin{tabular}{lcccc} \hline\hline
      Speaker$\backslash$Method & 1 & 2 & 3 & 4 \\ \hline
      Female \#1 & -- & 0.2831 & 0.2834 & 0.3195 \\
      Female \#2 & -- & 0.2745 & 0.2443 & 0.2846 \\
      Male \#1   & -- & 0.3005 & 0.2987 & 0.3179 \\
      Male \#2   & -- & 0.2787 & 0.2675 & 0.2957 \\
      \hline
      Total      & 0.3486 & 0.2842 & 0.2734 & 0.3044 \\
      \hline
    \end{tabular}
    \label{tab:avg_pre}
  \end{center}
\end{table}

\section{Related Work}
\label{sec:related_work}

The method proposed by Kupiec~\etal~\shortcite{kupiec:arpa-hlt-94} and
our method are similar in the sense that both methods use target
collections as language models for speech recognition to realize
open-vocabulary speech-driven retrieval.

Kupiec~\etaleos's method, which is based on word recognition and
accepts only short queries, derives multiple transcription candidates
(i.e., possible word combinations), and searches a target collection
for the most plausible word combination. However, in the case of
longer queries, the number of candidates increases, and thus the
searching cost is prohibitive.  This is a reason why operational
speech recognition systems have to limit the vocabulary size.

In contrast, our method, which is based on a recent {\em continuous\/}
speech recognition framework, can accept longer
sentences. Additionally, our method uses a two-stage retrieval
principle to limit a search space in a target collection, and
disambiguates only detected OOV words. Thus, the computation cost can
be minimized.

\section{Conclusion}
\label{sec:conclusion}

To facilitate retrieving information by spoken queries, the
out-of-vocabulary problem in speech recognition needs to be
resolved. In our proposed method, out-of-vocabulary words in a query
are detected by speech recognition, and completed with terms indexed
for text retrieval, so as to improve the recognition accuracy. In
addition, the completed query is used to improve the retrieval
accuracy. We showed the effectiveness of our method by using dictated
queries in the IREX collection.  Future work would include experiments
using larger-scale test collections in various domains.

\bibliographystyle{acl.bst}

\begin{thebibliography}{}

\bibitem[\protect\citename{Bahl \bgroup et al.\egroup
  }1983]{bahl:ieee-tpami-1983}
Lalit.~R. Bahl, Frederick Jelinek, and Robert~L. Mercer.
\newblock 1983.
\newblock A maximum likelihood approach to continuous speech recognition.
\newblock {\em IEEE Transactions on Pattern Analysis and Machine Intelligence},
  5(2):179--190.

\bibitem[\protect\citename{Barnett \bgroup et al.\egroup
  }1997]{barnett:eurospeech-97}
J.~Barnett, S.~Anderson, J.~Broglio, M.~Singh, R.~Hudson, and S.~W. Kuo.
\newblock 1997.
\newblock Experiments in spoken queries for document retrieval.
\newblock In {\em Proceedings of Eurospeech97}, pages 1323--1326.

\bibitem[\protect\citename{Crestani}2000]{crestani:fqas-2000}
Fabio Crestani.
\newblock 2000.
\newblock Word recognition errors and relevance feedback in spoken query
  processing.
\newblock In {\em Proceedings of the Fourth International Conference on
  Flexible Query Answering Systems}, pages 267--281.

\bibitem[\protect\citename{Fujii \bgroup et al.\egroup
  }2002]{fujii:springer-2002}
Atsushi Fujii, Katunobu Itou, and Tetsuya Ishikawa.
\newblock 2002.
\newblock Speech-driven text retrieval: Using target {IR} collections for
  statistical language model adaptation in speech recognition.
\newblock In Anni~R. Coden, Eric~W. Brown, and Savitha Srinivasan, editors,
  {\em Information Retrieval Techniques for Speech Applications (LNCS 2273)},
  pages 94--104. Springer.

\bibitem[\protect\citename{Garofolo \bgroup et al.\egroup
  }1997]{garofolo:trec-97}
John~S. Garofolo, Ellen~M. Voorhees, Vincent~M. Stanford, and Karen~Sparck
  Jones.
\newblock 1997.
\newblock {TREC-6} 1997 spoken document retrieval track overview and results.
\newblock In {\em Proceedings of the 6th Text REtrieval Conference}, pages
  83--91.

\bibitem[\protect\citename{Itou \bgroup et al.\egroup }1998]{itou:98:a}
K.~Itou, M.~Yamamoto, K.~Takeda, T.~Takezawa, T.~Matsuoka, T.~Kobayashi,
  K.~Shikano, and S.~Itahashi.
\newblock 1998.
\newblock The design of the newspaper-based {Japanese} large vocabulary
  continuous speech recognition corpus.
\newblock In {\em Proceedings of the 5th International Conference on Spoken
  Language Processing}, pages 3261--3264.

\bibitem[\protect\citename{Itou \bgroup et al.\egroup }1999]{itou:jas-1999}
Katunobu Itou, Mikio Yamamoto, Kazuya Takeda, Toshiyuki Takezawa, Tatsuo
  Matsuoka, Tetsunori Kobayashi, and Kiyohiro Shikano.
\newblock 1999.
\newblock {JNAS}: {Japanese} speech corpus for large vocabulary continuous
  speech recognition research.
\newblock {\em Journal of Acoustic Society of Japan}, 20(3):199--206.

\bibitem[\protect\citename{Itou \bgroup et al.\egroup }2001]{itou:asru-2001}
Katunobu Itou, Atsushi Fujii, and Tetsuya Ishikawa.
\newblock 2001.
\newblock Language modeling for multi-domain speech-driven text retrieval.
\newblock In {\em IEEE Automatic Speech Recognition and Understanding
  Workshop}.

\bibitem[\protect\citename{Jourlin \bgroup et al.\egroup
  }2000]{jourlin:sc-2000}
Pierre Jourlin, Sue~E. Johnson, Karen~Sp\"{a}rck Jones, and Philip~C. Woodland.
\newblock 2000.
\newblock Spoken document representations for probabilistic retrieval.
\newblock {\em Speech Communication}, 32:21--36.

\bibitem[\protect\citename{Kawahara \bgroup et al.\egroup
  }2000]{kawahara:icslp-2000}
T.~Kawahara, A.~Lee, T.~Kobayashi, K.~Takeda, N.~Minematsu, S.~Sagayama,
  K.~Itou, A.~Ito, M.~Yamamoto, A.~Yamada, T.~Utsuro, and K.~Shikano.
\newblock 2000.
\newblock Free software toolkit for {Japanese} large vocabulary continuous
  speech recognition.
\newblock In {\em Proceedings of the 6th International Conference on Spoken
  Language Processing}, pages 476--479.

\bibitem[\protect\citename{Kupiec \bgroup et al.\egroup
  }1994]{kupiec:arpa-hlt-94}
Julian Kupiec, Don Kimber, and Vijay Balasubramanian.
\newblock 1994.
\newblock Speech-based retrieval using semantic co-occurrence filtering.
\newblock In {\em Proceedings of the ARPA Human Language Technology Workshop},
  pages 373--377.

\bibitem[\protect\citename{Kwok and Chan}1998]{kwok:sigir-98}
K.L. Kwok and M.~Chan.
\newblock 1998.
\newblock Improving two-stage ad-hoc retrieval for short queries.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pages
  250--256.

\bibitem[\protect\citename{Paul and Baker}1992]{paul:darpa-ws-1992}
Douglas~B. Paul and Janet~M. Baker.
\newblock 1992.
\newblock The design for the {Wall} {Street} {Journal}-based {CSR} corpus.
\newblock In {\em Proceedings of DARPA Speech \& Natural Language Workshop},
  pages 357--362.

\bibitem[\protect\citename{Robertson and Walker}1994]{robertson:sigir-94}
S.E. Robertson and S.~Walker.
\newblock 1994.
\newblock Some simple effective approximations to the 2-poisson model for
  probabilistic weighted retrieval.
\newblock In {\em Proceedings of the 17th Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pages
  232--241.

\bibitem[\protect\citename{Steeneken and van
  Leeuwen}1995]{steeneken:eurospeech-1995}
Herman J.~M. Steeneken and David~A. van Leeuwen.
\newblock 1995.
\newblock Multi-lingual assessment of speaker independent large vocabulary
  speech-recognition systems: The {SQALE}-project.
\newblock In {\em Proceedings of Eurospeech95}, pages 1271--1274.

\bibitem[\protect\citename{Wechsler \bgroup et al.\egroup
  }1998]{wechsler:sigir-98}
Martin Wechsler, Eugen Munteanu, and Peter Sch\"{a}uble.
\newblock 1998.
\newblock New techniques for open-vocabulary spoken document retrieval.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pages
  20--27.

\bibitem[\protect\citename{Young}1996]{young:ieee-spm-1996}
Steve Young.
\newblock 1996.
\newblock A review of large-vocabulary continuous-speech recognition.
\newblock {\em IEEE Signal Processing Magazine}, pages 45--57, September.

\end{thebibliography}

\end{document}

psfig.tex
% Psfig/TeX Release 1.2
%
% Archive users note: this is an out-of-date version, preserved because future
% versions are backwards incompatible. Use psfig.sty for the up-to-date
% version.
%
% dvips version
%
% All software, documentation, and related files in this distribution of
% psfig/tex are Copyright 1987, 1988 Trevor J. Darrell
%
% Permission is granted for use and non-profit distribution of psfig/tex 
% providing that this notice be clearly maintained, but the right to
% distribute any portion of psfig/tex for profit or as part of any commercial
% product is specifically reserved for the author.
%
% $Header: psfig.tex,v 1.9 88/01/08 17:42:01 trevor Exp $
% $Source: $
%
% Thanks to Greg Hager (GDH) and Ned Batchelder for their contributions
% to this project.
%
\catcode`\@=11\relax
\newwrite\@unused
\def\typeout#1{{\let\protect\string\immediate\write\@unused{#1}}}
\typeout{psfig/tex 1.2-dvips}


%% Here's how you define your figure path.  Should be set up with null
%% default and a user useable definition.

\def\figurepath{./}
\def\psfigurepath#1{\edef\figurepath{#1}}

%
% @psdo control structure -- similar to Latex @for.
% I redefined these with different names so that psfig can
% be used with TeX as well as LaTeX, and so that it will not 
% be vunerable to future changes in LaTeX's internal
% control structure,
%
\def\@nnil{\@nil}
\def\@empty{}
\def\@psdonoop#1\@@#2#3{}
\def\@psdo#1:=#2\do#3{\edef\@psdotmp{#2}\ifx\@psdotmp\@empty \else
    \expandafter\@psdoloop#2,\@nil,\@nil\@@#1{#3}\fi}
\def\@psdoloop#1,#2,#3\@@#4#5{\def#4{#1}\ifx #4\@nnil \else
       #5\def#4{#2}\ifx #4\@nnil \else#5\@ipsdoloop #3\@@#4{#5}\fi\fi}
\def\@ipsdoloop#1,#2\@@#3#4{\def#3{#1}\ifx #3\@nnil 
       \let\@nextwhile=\@psdonoop \else
      #4\relax\let\@nextwhile=\@ipsdoloop\fi\@nextwhile#2\@@#3{#4}}
\def\@tpsdo#1:=#2\do#3{\xdef\@psdotmp{#2}\ifx\@psdotmp\@empty \else
    \@tpsdoloop#2\@nil\@nil\@@#1{#3}\fi}
\def\@tpsdoloop#1#2\@@#3#4{\def#3{#1}\ifx #3\@nnil 
       \let\@nextwhile=\@psdonoop \else
      #4\relax\let\@nextwhile=\@tpsdoloop\fi\@nextwhile#2\@@#3{#4}}
% 
%
\def\psdraft{
	\def\@psdraft{0}
	%\typeout{draft level now is \@psdraft \space . }
}
\def\psfull{
	\def\@psdraft{100}
	%\typeout{draft level now is \@psdraft \space . }
}
\psfull
\newif\if@prologfile
\newif\if@postlogfile
\newif\if@noisy
\def\pssilent{
	\@noisyfalse
}
\def\psnoisy{
	\@noisytrue
}
\psnoisy
%%% These are for the option list.
%%% A specification of the form a = b maps to calling \@p@@sa{b}
\newif\if@bbllx
\newif\if@bblly
\newif\if@bburx
\newif\if@bbury
\newif\if@height
\newif\if@width
\newif\if@rheight
\newif\if@rwidth
\newif\if@clip
\newif\if@verbose
\def\@p@@sclip#1{\@cliptrue}

%%% GDH 7/26/87 -- changed so that it first looks in the local directory,
%%% then in a specified global directory for the ps file.

\def\@p@@sfile#1{\def\@p@sfile{null}%
	        \openin1=#1
		\ifeof1\closein1%
		       \openin1=\figurepath#1
			\ifeof1\typeout{Error, File #1 not found}
			\else\closein1
			    \edef\@p@sfile{\figurepath#1}%
                        \fi%
		 \else\closein1%
		       \def\@p@sfile{#1}%
		 \fi}
\def\@p@@sfigure#1{\def\@p@sfile{null}%
	        \openin1=#1
		\ifeof1\closein1%
		       \openin1=\figurepath#1
			\ifeof1\typeout{Error, File #1 not found}
			\else\closein1
			    \def\@p@sfile{\figurepath#1}%
                        \fi%
		 \else\closein1%
		       \def\@p@sfile{#1}%
		 \fi}

\def\@p@@sbbllx#1{
		%\typeout{bbllx is #1}
		\@bbllxtrue
		\dimen100=#1
		\edef\@p@sbbllx{\number\dimen100}
}
\def\@p@@sbblly#1{
		%\typeout{bblly is #1}
		\@bbllytrue
		\dimen100=#1
		\edef\@p@sbblly{\number\dimen100}
}
\def\@p@@sbburx#1{
		%\typeout{bburx is #1}
		\@bburxtrue
		\dimen100=#1
		\edef\@p@sbburx{\number\dimen100}
}
\def\@p@@sbbury#1{
		%\typeout{bbury is #1}
		\@bburytrue
		\dimen100=#1
		\edef\@p@sbbury{\number\dimen100}
}
\def\@p@@sheight#1{
		\@heighttrue
		\dimen100=#1
   		\edef\@p@sheight{\number\dimen100}
		%\typeout{Height is \@p@sheight}
}
\def\@p@@swidth#1{
		%\typeout{Width is #1}
		\@widthtrue
		\dimen100=#1
		\edef\@p@swidth{\number\dimen100}
}
\def\@p@@srheight#1{
		%\typeout{Reserved height is #1}
		\@rheighttrue
		\dimen100=#1
		\edef\@p@srheight{\number\dimen100}
}
\def\@p@@srwidth#1{
		%\typeout{Reserved width is #1}
		\@rwidthtrue
		\dimen100=#1
		\edef\@p@srwidth{\number\dimen100}
}
\def\@p@@ssilent#1{ 
		\@verbosefalse
}
\def\@p@@sprolog#1{\@prologfiletrue\def\@prologfileval{#1}}
\def\@p@@spostlog#1{\@postlogfiletrue\def\@postlogfileval{#1}}
\def\@cs@name#1{\csname #1\endcsname}
\def\@setparms#1=#2,{\@cs@name{@p@@s#1}{#2}}
%
% initialize the defaults (size the size of the figure)
%
\def\ps@init@parms{
		\@bbllxfalse \@bbllyfalse
		\@bburxfalse \@bburyfalse
		\@heightfalse \@widthfalse
		\@rheightfalse \@rwidthfalse
		\def\@p@sbbllx{}\def\@p@sbblly{}
		\def\@p@sbburx{}\def\@p@sbbury{}
		\def\@p@sheight{}\def\@p@swidth{}
		\def\@p@srheight{}\def\@p@srwidth{}
		\def\@p@sfile{}
		\def\@p@scost{10}
		\def\@sc{}
		\@prologfilefalse
		\@postlogfilefalse
		\@clipfalse
		\if@noisy
			\@verbosetrue
		\else
			\@verbosefalse
		\fi
}
%
% Go through the options setting things up.
%
\def\parse@ps@parms#1{
	 	\@psdo\@psfiga:=#1\do
		   {\expandafter\@setparms\@psfiga,}}
%
% Compute bb height and width
%
\newif\ifno@bb
\newif\ifnot@eof
\newread\ps@stream
\def\bb@missing{
	\if@verbose{
		\typeout{psfig: searching \@p@sfile \space  for bounding box}
	}\fi
	\openin\ps@stream=\@p@sfile
	\no@bbtrue
	\not@eoftrue
	\catcode`\%=12
	\loop
		\read\ps@stream to \line@in
		\global\toks200=\expandafter{\line@in}
		\ifeof\ps@stream \not@eoffalse \fi
		%\typeout{ looking at :: \the\toks200 }
		\@bbtest{\toks200}
		\if@bbmatch\not@eoffalse\expandafter\bb@cull\the\toks200\fi
	\ifnot@eof \repeat
	\catcode`\%=14
}	
\catcode`\%=12
\newif\if@bbmatch
\def\@bbtest#1{\expandafter\@a@\the#1%%BoundingBox:\@bbtest\@a@}
\long\def\@a@#1%%BoundingBox:#2#3\@a@{\ifx\@bbtest#2\@bbmatchfalse\else\@bbmatchtrue\fi}
\long\def\bb@cull#1 #2 #3 #4 #5 {
	\dimen100=#2 bp\edef\@p@sbbllx{\number\dimen100}
	\dimen100=#3 bp\edef\@p@sbblly{\number\dimen100}
	\dimen100=#4 bp\edef\@p@sbburx{\number\dimen100}
	\dimen100=#5 bp\edef\@p@sbbury{\number\dimen100}
	\no@bbfalse
}
\catcode`\%=14
%
\def\compute@bb{
		\no@bbfalse
		\if@bbllx \else \no@bbtrue \fi
		\if@bblly \else \no@bbtrue \fi
		\if@bburx \else \no@bbtrue \fi
		\if@bbury \else \no@bbtrue \fi
		\ifno@bb \bb@missing \fi
		\ifno@bb \typeout{FATAL ERROR: no bb supplied or found}
			\no-bb-error
		\fi
		%
		\count203=\@p@sbburx
		\count204=\@p@sbbury
		\advance\count203 by -\@p@sbbllx
		\advance\count204 by -\@p@sbblly
		\edef\@bbw{\number\count203}
		\edef\@bbh{\number\count204}
		%\typeout{ bbh = \@bbh, bbw = \@bbw }
}
%
% \in@hundreds performs #1 * (#2 / #3) correct to the hundreds,
%	then leaves the result in @result
%
\def\in@hundreds#1#2#3{\count240=#2 \count241=#3
		     \count100=\count240	% 100 is first digit #2/#3
		     \divide\count100 by \count241
		     \count101=\count100
		     \multiply\count101 by \count241
		     \advance\count240 by -\count101
		     \multiply\count240 by 10
		     \count101=\count240	%101 is second digit of #2/#3
		     \divide\count101 by \count241
		     \count102=\count101
		     \multiply\count102 by \count241
		     \advance\count240 by -\count102
		     \multiply\count240 by 10
		     \count102=\count240	% 102 is the third digit
		     \divide\count102 by \count241
		     \count200=#1\count205=0
		     \count201=\count200
			\multiply\count201 by \count100
		 	\advance\count205 by \count201
		     \count201=\count200
			\divide\count201 by 10
			\multiply\count201 by \count101
			\advance\count205 by \count201
			%
		     \count201=\count200
			\divide\count201 by 100
			\multiply\count201 by \count102
			\advance\count205 by \count201
			%
		     \edef\@result{\number\count205}
}
\def\compute@wfromh{
		% computing : width = height * (bbw / bbh)
		\in@hundreds{\@p@sheight}{\@bbw}{\@bbh}
		%\typeout{ \@p@sheight * \@bbw / \@bbh, = \@result }
		\edef\@p@swidth{\@result}
		%\typeout{w from h: width is \@p@swidth}
}
\def\compute@hfromw{
		% computing : height = width * (bbh / bbw)
		\in@hundreds{\@p@swidth}{\@bbh}{\@bbw}
		%\typeout{ \@p@swidth * \@bbh / \@bbw = \@result }
		\edef\@p@sheight{\@result}
		%\typeout{h from w : height is \@p@sheight}
}
\def\compute@handw{
		\if@height 
			\if@width
			\else
				\compute@wfromh
			\fi
		\else 
			\if@width
				\compute@hfromw
			\else
				\edef\@p@sheight{\@bbh}
				\edef\@p@swidth{\@bbw}
			\fi
		\fi
}
\def\compute@resv{
		\if@rheight \else \edef\@p@srheight{\@p@sheight} \fi
		\if@rwidth \else \edef\@p@srwidth{\@p@swidth} \fi
}
%		
% Compute any missing values
\def\compute@sizes{
	\compute@bb
	\compute@handw
	\compute@resv
}
%
% \psfig
% usage : \psfig{file=, height=, width=, bbllx=, bblly=, bburx=, bbury=,
%			rheight=, rwidth=, clip=}
%
% "clip=" is a switch and takes no value, but the `=' must be present.
\def\psfig#1{\vbox {
	% do a zero width hard space so that a single
	% \psfig in a centering enviornment will behave nicely
	%{\setbox0=\hbox{\ }\ \hskip-\wd0}
	%
	\ps@init@parms
	\parse@ps@parms{#1}
	\compute@sizes
	%
	\ifnum\@p@scost<\@psdraft{
		\if@verbose{
			\typeout{psfig: including \@p@sfile \space }
		}\fi
		%
		\special{ps::[begin] 	\@p@swidth \space \@p@sheight \space
				\@p@sbbllx \space \@p@sbblly \space
				\@p@sbburx \space \@p@sbbury \space
				startTexFig \space }
		\if@clip{
			\if@verbose{
				\typeout{(clip)}
			}\fi
			\special{ps:: doclip \space }
		}\fi
		\if@prologfile
		    \special{ps: plotfile \@prologfileval \space } \fi
		\special{ps: plotfile \@p@sfile \space }
		\if@postlogfile
		    \special{ps: plotfile \@postlogfileval \space } \fi
		\special{ps::[end] endTexFig \space }
		% Create the vbox to reserve the space for the figure
		\vbox to \@p@srheight true sp{
			\hbox to \@p@srwidth true sp{
				\hss
			}
		\vss
		}
	}\else{
		% draft figure, just reserve the space and print the
		% path name.
		\vbox to \@p@srheight true sp{
		\vss
			\hbox to \@p@srwidth true sp{
				\hss
				\if@verbose{
					\@p@sfile
				}\fi
				\hss
			}
		\vss
		}
	}\fi
}}
\def\psglobal{\typeout{psfig: PSGLOBAL is OBSOLETE; use psprint -m instead}}
\catcode`\@=12\relax


cs0206014.extracted_bib
\begin{thebibliography}{}

\bibitem[\protect\citename{Bahl \bgroup et al.\egroup
  }1983]{bahl:ieee-tpami-1983}
Lalit.~R. Bahl, Frederick Jelinek, and Robert~L. Mercer.
\newblock 1983.
\newblock A maximum likelihood approach to continuous speech recognition.
\newblock {\em IEEE Transactions on Pattern Analysis and Machine Intelligence},
  5(2):179--190.

\bibitem[\protect\citename{Barnett \bgroup et al.\egroup
  }1997]{barnett:eurospeech-97}
J.~Barnett, S.~Anderson, J.~Broglio, M.~Singh, R.~Hudson, and S.~W. Kuo.
\newblock 1997.
\newblock Experiments in spoken queries for document retrieval.
\newblock In {\em Proceedings of Eurospeech97}, pages 1323--1326.

\bibitem[\protect\citename{Crestani}2000]{crestani:fqas-2000}
Fabio Crestani.
\newblock 2000.
\newblock Word recognition errors and relevance feedback in spoken query
  processing.
\newblock In {\em Proceedings of the Fourth International Conference on
  Flexible Query Answering Systems}, pages 267--281.

\bibitem[\protect\citename{Fujii \bgroup et al.\egroup
  }2002]{fujii:springer-2002}
Atsushi Fujii, Katunobu Itou, and Tetsuya Ishikawa.
\newblock 2002.
\newblock Speech-driven text retrieval: Using target {IR} collections for
  statistical language model adaptation in speech recognition.
\newblock In Anni~R. Coden, Eric~W. Brown, and Savitha Srinivasan, editors,
  {\em Information Retrieval Techniques for Speech Applications (LNCS 2273)},
  pages 94--104. Springer.

\bibitem[\protect\citename{Garofolo \bgroup et al.\egroup
  }1997]{garofolo:trec-97}
John~S. Garofolo, Ellen~M. Voorhees, Vincent~M. Stanford, and Karen~Sparck
  Jones.
\newblock 1997.
\newblock {TREC-6} 1997 spoken document retrieval track overview and results.
\newblock In {\em Proceedings of the 6th Text REtrieval Conference}, pages
  83--91.

\bibitem[\protect\citename{Itou \bgroup et al.\egroup }1998]{itou:98:a}
K.~Itou, M.~Yamamoto, K.~Takeda, T.~Takezawa, T.~Matsuoka, T.~Kobayashi,
  K.~Shikano, and S.~Itahashi.
\newblock 1998.
\newblock The design of the newspaper-based {Japanese} large vocabulary
  continuous speech recognition corpus.
\newblock In {\em Proceedings of the 5th International Conference on Spoken
  Language Processing}, pages 3261--3264.

\bibitem[\protect\citename{Itou \bgroup et al.\egroup }1999]{itou:jas-1999}
Katunobu Itou, Mikio Yamamoto, Kazuya Takeda, Toshiyuki Takezawa, Tatsuo
  Matsuoka, Tetsunori Kobayashi, and Kiyohiro Shikano.
\newblock 1999.
\newblock {JNAS}: {Japanese} speech corpus for large vocabulary continuous
  speech recognition research.
\newblock {\em Journal of Acoustic Society of Japan}, 20(3):199--206.

\bibitem[\protect\citename{Itou \bgroup et al.\egroup }2001]{itou:asru-2001}
Katunobu Itou, Atsushi Fujii, and Tetsuya Ishikawa.
\newblock 2001.
\newblock Language modeling for multi-domain speech-driven text retrieval.
\newblock In {\em IEEE Automatic Speech Recognition and Understanding
  Workshop}.

\bibitem[\protect\citename{Jourlin \bgroup et al.\egroup
  }2000]{jourlin:sc-2000}
Pierre Jourlin, Sue~E. Johnson, Karen~Sp\"{a}rck Jones, and Philip~C. Woodland.
\newblock 2000.
\newblock Spoken document representations for probabilistic retrieval.
\newblock {\em Speech Communication}, 32:21--36.

\bibitem[\protect\citename{Kawahara \bgroup et al.\egroup
  }2000]{kawahara:icslp-2000}
T.~Kawahara, A.~Lee, T.~Kobayashi, K.~Takeda, N.~Minematsu, S.~Sagayama,
  K.~Itou, A.~Ito, M.~Yamamoto, A.~Yamada, T.~Utsuro, and K.~Shikano.
\newblock 2000.
\newblock Free software toolkit for {Japanese} large vocabulary continuous
  speech recognition.
\newblock In {\em Proceedings of the 6th International Conference on Spoken
  Language Processing}, pages 476--479.

\bibitem[\protect\citename{Kupiec \bgroup et al.\egroup
  }1994]{kupiec:arpa-hlt-94}
Julian Kupiec, Don Kimber, and Vijay Balasubramanian.
\newblock 1994.
\newblock Speech-based retrieval using semantic co-occurrence filtering.
\newblock In {\em Proceedings of the ARPA Human Language Technology Workshop},
  pages 373--377.

\bibitem[\protect\citename{Kwok and Chan}1998]{kwok:sigir-98}
K.L. Kwok and M.~Chan.
\newblock 1998.
\newblock Improving two-stage ad-hoc retrieval for short queries.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pages
  250--256.

\bibitem[\protect\citename{Paul and Baker}1992]{paul:darpa-ws-1992}
Douglas~B. Paul and Janet~M. Baker.
\newblock 1992.
\newblock The design for the {Wall} {Street} {Journal}-based {CSR} corpus.
\newblock In {\em Proceedings of DARPA Speech \& Natural Language Workshop},
  pages 357--362.

\bibitem[\protect\citename{Robertson and Walker}1994]{robertson:sigir-94}
S.E. Robertson and S.~Walker.
\newblock 1994.
\newblock Some simple effective approximations to the 2-poisson model for
  probabilistic weighted retrieval.
\newblock In {\em Proceedings of the 17th Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pages
  232--241.

\bibitem[\protect\citename{Steeneken and van
  Leeuwen}1995]{steeneken:eurospeech-1995}
Herman J.~M. Steeneken and David~A. van Leeuwen.
\newblock 1995.
\newblock Multi-lingual assessment of speaker independent large vocabulary
  speech-recognition systems: The {SQALE}-project.
\newblock In {\em Proceedings of Eurospeech95}, pages 1271--1274.

\bibitem[\protect\citename{Wechsler \bgroup et al.\egroup
  }1998]{wechsler:sigir-98}
Martin Wechsler, Eugen Munteanu, and Peter Sch\"{a}uble.
\newblock 1998.
\newblock New techniques for open-vocabulary spoken document retrieval.
\newblock In {\em Proceedings of the 21st Annual International ACM SIGIR
  Conference on Research and Development in Information Retrieval}, pages
  20--27.

\bibitem[\protect\citename{Young}1996]{young:ieee-spm-1996}
Steve Young.
\newblock 1996.
\newblock A review of large-vocabulary continuous-speech recognition.
\newblock {\em IEEE Signal Processing Magazine}, pages 45--57, September.

\end{thebibliography}