  The contextual information of Web images is investigated to address the issue
of enriching their index characterizations with semantic descriptors and
therefore bridge the semantic gap (i.e. the gap between the low-level
content-based description of images and their semantic interpretation).
Although we are highly motivated by the availability of rich knowledge on the
Web and the relative success achieved by commercial search engines in indexing
images using surrounding text-based information in webpages, we are aware that
the unpredictable quality of the surrounding text is a major limiting factor.
In order to improve its quality, we highlight contextual information which is
relevant for the semantic characterization of Web images and study its
statistical properties in terms of its location and nature considering a
classification into five semantic concept classes: signal, object, scene,
abstract and relational. A user study is conducted to validate the results. The
results suggest that there are several locations that consistently contain
relevant textual information with respect to the image. The importance of each
location is influenced by the type of webpage as the results show the different
distribution of relevant contextual information across the locations for
different webpage types. The frequently found semantic concept classes are
object and abstract. Another important outcome of the user study shows that a
webpage is not an atomic unit and can be further partitioned into smaller
segments. Segments containing images are of interest and termed as image
segments. We observe that users typically single out textual information which
they consider relevant to the image from the textual information bounded within
the image segment.
