Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Joint Embedding with Modality Alignments for Cross-Modal
  Retrieval of Recipes and Food Images   This paper presents a three-tier modality alignment approach to learning
text-image joint embedding, coined as JEMA, for cross-modal retrieval of
cooking recipes and food images. The first tier improves recipe text embedding
by optimizing the LSTM networks with term extraction and ranking enhanced
sequence patterns, and optimizes the image embedding by combining the
ResNeXt-101 image encoder with the category embedding using wideResNet-50 with
word2vec. The second tier modality alignment optimizes the textual-visual joint
embedding loss function using a double batch-hard triplet loss with soft-margin
optimization. The third modality alignment incorporates two types of
cross-modality alignments as the auxiliary loss regularizations to further
reduce the alignment errors in the joint learning of the two modality-specific
embedding functions. The category-based cross-modal alignment aims to align the
image category with the recipe category as a loss regularization to the joint
embedding. The cross-modal discriminator-based alignment aims to add the
visual-textual embedding distribution alignment to further regularize the joint
embedding loss. Extensive experiments with the one-million recipes benchmark
dataset Recipe1M demonstrate that the proposed JEMA approach outperforms the
state-of-the-art cross-modal embedding methods for both image-to-recipe and
recipe-to-image retrievals.

10
The categories of this paper are: q-bio
19
The categories of this paper are: eess
26
The categories of this paper are: bayes-an
33
The categories of this paper are: plasm-ph
28
The categories of this paper are: alg-geom
29
The categories of this paper are: funct-an
3
The categories of this paper are: physics
30
The categories of this paper are: q-alg
16
The categories of this paper are: stat
31
The categories of this paper are: ao-sci
12
The categories of this paper are: nucl-th
17
The categories of this paper are: q-fin
22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
2
The categories of this paper are: math
11
The categories of this paper are: hep-lat