  Neural saturation in Deep Neural Networks (DNNs) has been studied
extensively, but remains relatively unexplored in Convolutional Neural Networks
(CNNs). Understanding and alleviating the effects of convolutional kernel
saturation is critical for enhancing CNN models classification accuracies. In
this paper, we analyze the effect of convolutional kernel saturation in CNNs
and propose a simple data augmentation technique to mitigate saturation and
increase classification accuracy, by supplementing negative images to the
training dataset. We hypothesize that greater semantic feature information can
be extracted using negative images since they have the same structural
information as standard images but differ in their data representations. Varied
data representations decrease the probability of kernel saturation and thus
increase the effectiveness of kernel weight updates. The two datasets selected
to evaluate our hypothesis were CIFAR- 10 and STL-10 as they have similar image
classes but differ in image resolutions thus making for a better understanding
of the saturation phenomenon. MNIST dataset was used to highlight the
ineffectiveness of the technique for linearly separable data. The ResNet CNN
architecture was chosen since the skip connections in the network ensure the
most important features contributing the most to classification accuracy are
retained. Our results show that CNNs are indeed susceptible to convolutional
kernel saturation and that supplementing negative images to the training
dataset can offer a statistically significant increase in classification
accuracies when compared against models trained on the original datasets. Our
results present accuracy increases of 6.98% and 3.16% on the STL-10 and
CIFAR-10 datasets respectively.
