  In recent years, there have been many popular Convolutional Neural Networks
(CNNs), such as Google's Inception-V4, that have performed very well for
various image classification problems. These commonly used CNN models usually
use the same activation function, such as RELU, for all neurons in the
convolutional layers; they are "Single-function CNNs." However, SCNNs may not
always be optimal. Thus, a "Multi-function CNN" (MCNN), which uses different
activation functions for different neurons, has been shown to outperform a
SCNN. Also, CNNs typically have very large architectures that use a lot of
memory and need a lot of data in order to be trained well. As a result, they
tend to have very high training and prediction times too. An important research
problem is how to automatically and efficiently find the best CNN with both
high classification performance and compact architecture with high training and
prediction speeds, small power usage, and small memory size for any image
classification problem. It is very useful to intelligently find an effective,
fast, energy-efficient, and memory-efficient "Compressed Multi-function CNN"
(CMCNN) from a large number of candidate MCNNs. A new compensatory algorithm
using a new genetic algorithm (GA) is created to find the best CMCNN with an
ideal compensation between performance and architecture size. The optimal CMCNN
has the best performance and the smallest architecture size. Simulations using
the CIFAR10 dataset showed that the new compensatory algorithm could find
CMCNNs that could outperform non-compressed MCNNs in terms of classification
performance (F1-score), speed, power usage, and memory usage. Other effective,
fast, power-efficient, and memory-efficient CMCNNs based on popular CNN
architectures will be developed for image classification problems in important
real-world applications, such as brain informatics and biomedical imaging.
