  We introduce a dataset for facilitating audio-visual analysis of music
performances. The dataset comprises 44 simple multi-instrument classical music
pieces assembled from coordinated but separately recorded performances of
individual tracks. For each piece, we provide the musical score in MIDI format,
the audio recordings of the individual tracks, the audio and video recording of
the assembled mixture, and ground-truth annotation files including frame-level
and note-level transcriptions. We describe our methodology for the creation of
the dataset, particularly highlighting our approaches for addressing the
challenges involved in maintaining synchronization and expressiveness. We
demonstrate the high quality of synchronization achieved with our proposed
approach by comparing the dataset with existing widely-used music audio
datasets.
  We anticipate that the dataset will be useful for the development and
evaluation of existing music information retrieval (MIR) tasks, as well as for
novel multi-modal tasks. We benchmark two existing MIR tasks (multi-pitch
analysis and score-informed source separation) on the dataset and compare with
other existing music audio datasets. Additionally, we consider two novel
multi-modal MIR tasks (visually informed multi-pitch analysis and polyphonic
vibrato analysis) enabled by the dataset and provide evaluation measures and
baseline systems for future comparisons (from our recent work). Finally, we
propose several emerging research directions that the dataset enables.
