Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Image Captioning with Visual Object Representations Grounded in the
  Textual Modality   We present our work in progress exploring the possibilities of a shared
embedding space between textual and visual modality. Leveraging the textual
nature of object detection labels and the hypothetical expressiveness of
extracted visual object representations, we propose an approach opposite to the
current trend, grounding of the representations in the word embedding space of
the captioning system instead of grounding words or sentences in their
associated images. Based on the previous work, we apply additional grounding
losses to the image captioning training objective aiming to force visual object
representations to create more heterogeneous clusters based on their class
label and copy a semantic structure of the word embedding space. In addition,
we provide an analysis of the learned object vector space projection and its
impact on the IC system performance. With only slight change in performance,
grounded models reach the stopping criterion during training faster than the
unconstrained model, needing about two to three times less training updates.
Additionally, an improvement in structural correlation between the word
embeddings and both original and projected object vectors suggests that the
grounding is actually mutual.

29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
34
The categories of this paper are: chem-ph
8
The categories of this paper are: hep-ex
32
The categories of this paper are: atom-ph
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
18
The categories of this paper are: econ
37
The categories of this paper are: supr-con
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom
3
The categories of this paper are: physics
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess