Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: SimulEval: An Evaluation Toolkit for Simultaneous Translation   Simultaneous translation on both text and speech focuses on a real-time and
low-latency scenario where the model starts translating before reading the
complete source input. Evaluating simultaneous translation models is more
complex than offline models because the latency is another factor to consider
in addition to translation quality. The research community, despite its growing
focus on novel modeling approaches to simultaneous translation, currently lacks
a universal evaluation procedure. Therefore, we present SimulEval, an
easy-to-use and general evaluation toolkit for both simultaneous text and
speech translation. A server-client scheme is introduced to create a
simultaneous translation scenario, where the server sends source input and
receives predictions for evaluation and the client executes customized
policies. Given a policy, it automatically performs simultaneous decoding and
collectively reports several popular latency metrics. We also adapt latency
metrics from text simultaneous translation to the speech task. Additionally,
SimulEval is equipped with a visualization interface to provide better
understanding of the simultaneous decoding process of a system. SimulEval has
already been extensively used for the IWSLT 2020 shared task on simultaneous
speech translation. Code will be released upon publication.

31
The categories of this paper are: ao-sci
23
The categories of this paper are: solv-int
13
The categories of this paper are: quant-ph
10
The categories of this paper are: q-bio
18
The categories of this paper are: econ
11
The categories of this paper are: hep-lat
6
The categories of this paper are: astro-ph
2
The categories of this paper are: math
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
9
The categories of this paper are: nlin
15
The categories of this paper are: nucl-ex
29
The categories of this paper are: funct-an
36
The categories of this paper are: cmp-lg
12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess