  The events of recent years have highlighted the importance of telemedicine
solutions which could potentially allow remote treatment and diagnosis.
Relatedly, Computational Paralinguistics, a unique subfield of Speech
Processing, aims to extract information about the speaker and form an important
part of telemedicine applications. In this work, we focus on two paralinguistic
problems: mask detection and breathing state prediction. Solutions developed
for these tasks could be invaluable and have the potential to help monitor and
limit the spread of a virus like COVID-19. The current state-of-the-art methods
proposed for these tasks are ensembles based on deep neural networks like
ResNets in conjunction with feature engineering. Although these ensembles can
achieve high accuracy, they also have a large footprint and require substantial
computational power reducing portability to devices with limited resources.
These drawbacks also mean that the previously proposed solutions are infeasible
to be used in a telemedicine system due to their size and speed. On the other
hand, employing lighter feature-engineered systems can be laborious and add
further complexity making them difficult to create a deployable system quickly.
This work proposes an ensemble-based automatic feature selection method to
enable the development of fast and memory-efficient systems. In particular, we
propose an output-gradient-based method to discover essential features using
large, well-performing ensembles before training a smaller one. In our
experiments, we observed considerable (25-32%) reductions in inference times
using neural network ensembles based on output-gradient-based features. Our
method offers a simple way to increase the speed of the system and enable
real-time usage while maintaining competitive results with larger-footprint
ensemble using all spectral features.
