  Current word embedding models despite their success, still suffer from their
lack of grounding in the real world. In this line of research, Gunther et al.
2022 proposed a behavioral experiment to investigate the relationship between
words and images. In their setup, participants were presented with a target
noun and a pair of images, one chosen by their model and another chosen
randomly. Participants were asked to select the image that best matched the
target noun. In most cases, participants preferred the image selected by the
model. Gunther et al., therefore, concluded the possibility of a direct link
between words and embodied experience. We took their experiment as a point of
departure and addressed the following questions. 1. Apart from utilizing
visually embodied simulation of given images, what other strategies might
subjects have used to solve this task? To what extent does this setup rely on
visual information from images? Can it be solved using purely textual
representations? 2. Do current visually grounded embeddings explain subjects'
selection behavior better than textual embeddings? 3. Does visual grounding
improve the semantic representations of both concrete and abstract words? To
address these questions, we designed novel experiments by using pre-trained
textual and visually grounded word embeddings. Our experiments reveal that
subjects' selection behavior is explained to a large extent based on purely
text-based embeddings and word-based similarities, suggesting a minor
involvement of active embodied experiences. Visually grounded embeddings
offered modest advantages over textual embeddings only in certain cases. These
findings indicate that the experiment by Gunther et al. may not be well suited
for tapping into the perceptual experience of participants, and therefore the
extent to which it measures visually grounded knowledge is unclear.
