  Speech enhancement concerns the processes required to remove unwanted
background sounds from the target speech to improve its quality and
intelligibility. In this paper, a novel approach for single-channel speech
enhancement is presented, using colored spectrograms. We propose the use of a
deep neural network (DNN) architecture adapted from the pix2pix generative
adversarial network (GAN) and train it over colored spectrograms of speech to
denoise them. After denoising, the colors of spectrograms are translated to
magnitudes of short-time Fourier transform (STFT) using a shallow regression
neural network. These estimated STFT magnitudes are later combined with the
noisy phases to obtain an enhanced speech. The results show an improvement of
almost 0.84 points in the perceptual evaluation of speech quality (PESQ) and 1%
in the short-term objective intelligibility (STOI) over the unprocessed noisy
data. The gain in quality and intelligibility over the unprocessed signal is
almost equal to the gain achieved by the baseline methods used for comparison
with the proposed model, but at a much reduced computational cost. The proposed
solution offers a comparative PESQ score at almost 10 times reduced
computational cost than a similar baseline model that has generated the highest
PESQ score trained on grayscaled spectrograms, while it provides only a 1%
deficit in STOI at 28 times reduced computational cost when compared to another
baseline system based on convolutional neural network-GAN (CNN-GAN) that
produces the most intelligible speech.
