  Whisper is a powerful automatic speech recognition (ASR) model. Nevertheless,
its zero-shot performance on low-resource speech requires further improvement.
Child speech, as a representative type of low-resource speech, is leveraged for
adaptation. Recently, parameter-efficient fine-tuning (PEFT) in NLP was shown
to be comparable and even better than full fine-tuning, while only needing to
tune a small set of trainable parameters. However, current PEFT methods have
not been well examined for their effectiveness on Whisper. In this paper, only
parameter composition types of PEFT approaches such as LoRA and Bitfit are
investigated as they do not bring extra inference costs. Different popular PEFT
methods are examined. Particularly, we compare LoRA and AdaLoRA and figure out
the learnable rank coefficient is a good design. Inspired by the sparse rank
distribution allocated by AdaLoRA, a novel PEFT approach Sparsely Shared LoRA
(S2-LoRA) is proposed. The two low-rank decomposed matrices are globally
shared. Each weight matrix only has to maintain its specific rank coefficients
that are constrained to be sparse. Experiments on low-resource Chinese child
speech show that with much fewer trainable parameters, S2-LoRA can achieve
comparable in-domain adaptation performance to AdaLoRA and exhibit better
generalization ability on out-of-domain data. In addition, the rank
distribution automatically learned by S2-LoRA is found to have similar patterns
to AdaLoRA's allocation.
