  Recently, Conformer as a backbone network for end-to-end automatic speech
recognition achieved state-of-the-art performance. The Conformer block
leverages a self-attention mechanism to capture global information, along with
a convolutional neural network to capture local information, resulting in
improved performance. However, the Conformer-based model encounters an issue
with the self-attention mechanism, as computational complexity grows
quadratically with the length of the input sequence. Inspired by previous
Connectionist Temporal Classification (CTC) guided blank skipping during
decoding, we introduce intermediate CTC outputs as guidance into the
downsampling procedure of the Conformer encoder. We define the frame with
non-blank output as key frame. Specifically, we introduce the key frame-based
self-attention (KFSA) mechanism, a novel method to reduce the computation of
the self-attention mechanism using key frames. The structure of our proposed
approach comprises two encoders. Following the initial encoder, we introduce an
intermediate CTC loss function to compute the label frame, enabling us to
extract the key frames and blank frames for KFSA. Furthermore, we introduce the
key frame-based downsampling (KFDS) mechanism to operate on high-dimensional
acoustic features directly and drop the frames corresponding to blank labels,
which results in new acoustic feature sequences as input to the second encoder.
By using the proposed method, which achieves comparable or higher performance
than vanilla Conformer and other similar work such as Efficient Conformer.
Meantime, our proposed method can discard more than 60\% useless frames during
model training and inference, which will accelerate the inference speed
significantly. This work code is available in
{https://github.com/scufan1990/Key-Frame-Mechanism-For-Efficient-Conformer}
