  The excellent performance of recent self-supervised learning methods on
various downstream tasks has attracted great attention from academia and
industry. Some recent research efforts have been devoted to self-supervised
music representation learning. Nevertheless, most of them learn to represent
equally-sized music clips in the waveform or a spectrogram. Despite being
effective in some tasks, learning music representations in such a manner
largely neglect the inherent part-whole hierarchies of music. Due to the
hierarchical nature of the auditory cortex [24], understanding the bottom-up
structure of music, i.e., how different parts constitute the whole at different
levels, is essential for music understanding and representation learning. This
work pursues hierarchical music representation learning and introduces the
Music-PAW framework, which enables feature interactions of cropped music clips
with part-whole hierarchies. From a technical perspective, we propose a
transformer-based part-whole interaction module to progressively reason the
structural relationships between part-whole music clips at adjacent levels.
Besides, to create a multi-hierarchy representation space, we devise a
hierarchical contrastive learning objective to align part-whole music
representations in adjacent hierarchies. The merits of audio representation
learning from part-whole hierarchies have been validated on various downstream
tasks, including music classification (single-label and multi-label), cover
song identification and acoustic scene classification.
