Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: End-to-End Multi-View Lipreading   Non-frontal lip views contain useful information which can be used to enhance
the performance of frontal view lipreading. However, the vast majority of
recent lipreading works, including the deep learning approaches which
significantly outperform traditional approaches, have focused on frontal mouth
images. As a consequence, research on joint learning of visual features and
speech classification from multiple views is limited. In this work, we present
an end-to-end multi-view lipreading system based on Bidirectional Long-Short
Memory (BLSTM) networks. To the best of our knowledge, this is the first model
which simultaneously learns to extract features directly from the pixels and
performs visual speech classification from multiple views and also achieves
state-of-the-art performance. The model consists of multiple identical streams,
one for each view, which extract features directly from different poses of
mouth images. The temporal dynamics in each stream/view are modelled by a BLSTM
and the fusion of multiple streams/views takes place via another BLSTM. An
absolute average improvement of 3% and 3.8% over the frontal view performance
is reported on the OuluVS2 database when the best two (frontal and profile) and
three views (frontal, profile, 45) are combined, respectively. The best
three-view model results in a 10.5% absolute improvement over the current
multi-view state-of-the-art performance on OuluVS2, without using external
databases for training, achieving a maximum classification accuracy of 96.9%.

30
The categories of this paper are: q-alg
33
The categories of this paper are: plasm-ph
27
The categories of this paper are: comp-gas
29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
9
The categories of this paper are: nlin
13
The categories of this paper are: quant-ph
6
The categories of this paper are: astro-ph
8
The categories of this paper are: hep-ex
31
The categories of this paper are: ao-sci
10
The categories of this paper are: q-bio
20
The categories of this paper are: acc-phys
36
The categories of this paper are: cmp-lg
32
The categories of this paper are: atom-ph
15
The categories of this paper are: nucl-ex
26
The categories of this paper are: bayes-an