# Fusion For Evaluation Of Image Classification In Uncertain Environments

A. Martin E3I2 EA3876 ENSIETA
2 rue Fran¸cois Verny, 29806 Brest Cedex 09, France Arnaud.Martin@ensieta.fr Abstract - **We present in this article a new evaluation method for classification and segmentation**
of textured images in uncertain environments. In uncertain environments, real classes and boundaries are known with only a partial certainty given by the experts. Most of the time, in many presented papers, only classification or only segmentation are considered and evaluated. Here, we propose to take into account both the classification and segmentation results according to the certainty given by the experts.

We present the results of this method on a fusion of classifiers of sonar images for a seabed characterization.

Keywords: Image classification, Image segmentation, Uncertainty environment, Sonar image, Fusion of experts, Fusion of classifiers.

## 1 Introduction

Textured image classification is a difficult problem in image processing and it is fundamental for a lot of applications. Many features can be extracted from the images to classify, and many classification algorithms can be used [1]. Hence, it is really necessary to evaluate their performance in order to compare them and choose the most adapted to the application.

For instance, with satellite or sonar images, human experts must be able to classify the types of soils present in the images. Many types of soils can be encountered in a single image, and classification must be done on a local part of the image (pixel-wise, or often on small tiles of e.g. 16× 16 or 32× **32 pixels) taken**
as unit for the classification algorithm. Hence, after the image classification, an implicit image segmentation is obtained according to the size of the tiles. One image will be segmented into several patches, each one corresponding to a class (e.g. **a specific type of soil).**
The image classification methods are currently evaluated by the confusion matrix. Good-classification rates and error rates are usually calculated from this matrix. We must know the real class of the considered units of the images in order to establish the confusion matrix. Confusion matrix does not give an evaluation of the produced segmentation.

In order to evaluate the segmentation, we can not only consider visual comparison between the initial image and the segmented image. The image segmentation evaluation is still a studied problem [2, 3, 4, 5].

We can consider two cases: we do not have anya priori **knowledge of the correct segmentation, or we**
have an a priori **knowledge of the correct segmentation. Here we are in the second case because of the**
confusion matrix for which we need to get referenced images. In order to obtain these referenced images, experts must manually provide the image segmentation, for example via **a visual inspection. Zhang in [2] gives**
a review of usual discrepancy measures based on different distances between the segmented-pixel and the referenced-pixel. Most of the time, only one measure of mis-segmented pixel is given. We will propose on the contrary, in this article a linked study of one wellsegmented pixel measure and a mis-segmented pixel measure. Indeed, in general case, if a pixel is not missegmented, it is not necessary well-segmented. So we can have few mis-segmented pixels but also few wellsegmented pixels: the segmentation is not good.

We think that global image classification evaluation must be made by evaluating both the classification on considered units (with the confusion matrix) and in the same time by the evaluation of the produced segmentation (well-segmented pixel measure and a missegmented pixel measure) [6].

In real applications, it is really hard for one human expert to provide a certain information on the class and on the boundaries between the classes. For instance, the seabed characterization with sonar images cannot be made by human expert with a sufficient certainty. These images, illustrating this paper, are obtained with many imperfections [7]. Figure 1 exhibits the differences between the interpretation and the certainty of two sonar experts trying to differentiate the type of sediment (rock, cobbles, sand, ripple, silt) or shadow when the information is invisible (each color correspond to a kind of sediment and the associated certainty of the expert for this sediment expressed in terms of sure, moderately sure and not sure).

We propose here a new approach for textured image classification and segmentation taking into account the information given by multiple experts and their certainty. In section 2, we show how to integrate the expert certainty in confusion matrix and then deduce