Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Focus Your Attention: A Bidirectional Focal Attention Network for
  Image-Text Matching   Learning semantic correspondence between image and text is significant as it
bridges the semantic gap between vision and language. The key challenge is to
accurately find and correlate shared semantics in image and text. Most existing
methods achieve this goal by representing the shared semantic as a weighted
combination of all the fragments (image regions or text words), where fragments
relevant to the shared semantic obtain more attention, otherwise less. However,
despite relevant ones contribute more to the shared semantic, irrelevant ones
will more or less disturb it, and thus will lead to semantic misalignment in
the correlation phase. To address this issue, we present a novel Bidirectional
Focal Attention Network (BFAN), which not only allows to attend to relevant
fragments but also diverts all the attention into these relevant fragments to
concentrate on them. The main difference with existing works is they mostly
focus on learning attention weight while our BFAN focus on eliminating
irrelevant fragments from the shared semantic. The focal attention is achieved
by pre-assigning attention based on inter-modality relation, identifying
relevant fragments based on intra-modality relation and reassigning attention.
Furthermore, the focal attention is jointly applied in both image-to-text and
text-to-image directions, which enables to avoid preference to long text or
complex image. Experiments show our simple but effective framework
significantly outperforms state-of-the-art, with relative Recall@1 gains of
2.2% on both Flicr30K and MSCOCO benchmarks.

12
The categories of this paper are: nucl-th
31
The categories of this paper are: ao-sci
27
The categories of this paper are: comp-gas
20
The categories of this paper are: acc-phys
13
The categories of this paper are: quant-ph
25
The categories of this paper are: patt-sol
10
The categories of this paper are: q-bio
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
5
The categories of this paper are: gr-qc
3
The categories of this paper are: physics
29
The categories of this paper are: funct-an
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
4
The categories of this paper are: cond-mat
14
The categories of this paper are: math-ph