  Despite the recent increase in research on artificial intelligence for music,
prominent correlations between key components of lyrics and rhythm such as
keywords, stressed syllables, and strong beats are not frequently studied. This
is likely due to challenges such as audio misalignment, inaccuracies in
syllabic identification, and most importantly, the need for cross-disciplinary
knowledge. To address this lack of research, we propose a novel multimodal
lyrics-rhythm matching approach in this paper that specifically matches key
components of lyrics and music with each other without any language
limitations. We use audio instead of sheet music with readily available
metadata, which creates more challenges yet increases the application
flexibility of our method. Furthermore, our approach creatively generates
several patterns involving various multimodalities, including music strong
beats, lyrical syllables, auditory changes in a singer's pronunciation, and
especially lyrical keywords, which are utilized for matching key lyrical
elements with key rhythmic elements. This advantageous approach not only
provides a unique way to study auditory lyrics-rhythm correlations including
efficient rhythm-based audio alignment algorithms, but also bridges
computational linguistics with music as well as music cognition. Our
experimental results reveal an 0.81 probability of matching on average, and
around 30% of the songs have a probability of 0.9 or higher of keywords landing
on strong beats, including 12% of the songs with a perfect landing. Also, the
similarity metrics are used to evaluate the correlation between lyrics and
rhythm. It shows that nearly 50% of the songs have 0.70 similarity or higher.
In conclusion, our approach contributes significantly to the lyrics-rhythm
relationship by computationally unveiling insightful correlations.
