text: Music generation has generally been focused on either creating scores or interpreting them. We discuss differences between these two problems and propose that, in fact, it may be valuable to work in the space of direct performance generation: jointly predicting the notes and also their expressive timing and dynamics. We consider the significance and qualities of the dataset needed for this. Having identified both a problem domain and characteristics of an appropriate dataset, we show an LSTM-based recurrent network model that subjectively performs quite well on this task. Critically, we provide generated examples. We also include feedback from professional composers and musicians about some of these examples.
  @cite_30: AbstractThe author recently described elsewhere a computer program which would transcribe classical melodies played on an organ console into the equivalent of standard musical notation. This program was the fruit of a prolonged effort to understand how Western musicians succeed in making sense of music, in discerning rhythm and the tonal relation hip between notes. The interest of the problem arise from the fact that no two performance of the same piece of music are ever identical so that the listener has to discriminate between those variation of timing and pitch which are structurally significant and those which are merely expressive. In order to under land the ability of some musician to reproduce the cores of music that they hear it is necessary to develop a formally precise theory of rhythm and tonality – a theory which is couched in terms reminiscent of Chomskyan linguistics. But it is also necessary to consider how the listener builds in his mind a representation of the rhythm and tonality of perfo...
  @cite_22: Abstract In algorithmic music composition, a simple technique involves selecting notes sequentially according to a transition table that specifies the probability of the next note as a function of the previous context. An extension of this transition-table approach is described, using a recurrent autopredictive connectionist network called CONCERT. CONCERT is trained on a set of pieces with the aim of extracting stylistic regularities. CONCERT can then be used to compose new pieces. A central ingredient of CONCERT is the incorporation of psychologically grounded representations of pitch, duration and harmonic structure. CONCERT was tested on sets of examples artificially generated according to simple rules and was shown to learn the underlying structure, even where other approaches failed. In larger experiments, CONCERT was trained on sets of J. S. Bach pieces and traditional European folk melodies and was then allowed to compose novel melodies. Although the compositions are occasionally pleasant, and are...
  @cite_41: Generating music with long-term structure is one of the main challenges in the field of automatic composition. This article describes MorpheuS, a music generation system. MorpheuS uses state-of-the-art pattern detection techniques to find repeated patterns in a template piece. These patterns are then used to constrain the generation process for a new polyphonic composition. The music generation process is guided by an efficient optimization algorithm, variable neighborhood search, which uses a mathematical model of tonal tension to derive its objective function. The ability to generate music according to a tension profile could be useful in a game or film music context. Pieces generated by MorpheuS have been performed in live concerts.
  @cite_42: This volume presents the most up-to-date collection of neural network models of music and creativity gathered together in one place. Chapters by leaders in the field cover new connectionist models of pitch perception, tonality, musical streaming, sequential and hierarchical melodic structure, composition, harmonization, rhythmic analysis, sound generation, and creative evolution. The collection combines journal papers on connectionist modeling, cognitive science, and music perception with new papers solicited for this volume. It also contains an extensive bibliography of related work. Contributors: Shumeet Baluja, M.I. Bellgard, Michael A. Casey, Garrison W. Cottrell, Peter Desain, Robert O. Gjerdingen, Mike Greenhough, Niall Griffith, Stephen Grossberg, Henkjan Honing, Todd Jochem, Bruce F. Katz, John F. Kolen, Edward W. Large, Michael C. Mozer, Michael P.A. Page, Caroline Palmer, Jordan B. Pollack, Dean Pomerleau, Stephen W. Smoliar, Ian Taylor, Peter M. Todd, C.P. Tsang, Gregory M. Werner.
  @cite_3: Automatic music generation systems have gained in popularity and sophistication as advances in cloud computing have enabled large-scale complex computations such as deep models and optimization algorithms on personal devices. Yet, they still face an important challenge, that of long-term structure, which is key to conveying a sense of musical coherence. We present the MorpheuS music generation system designed to tackle this problem. MorpheuS' novel framework has the ability to generate polyphonic pieces with a given tension profile and long- and short-term repeated pattern structures. A mathematical model for tonal tension quantifies the tension profile and state-of-the-art pattern detection algorithms extract repeated patterns in a template piece. An efficient optimization metaheuristic, variable neighborhood search, generates music by assigning pitches that best fit the prescribed tension profile to the template rhythm while hard constraining long-term structure through the detected patterns. This ability to generate affective music with specific tension profile and long-term structure is particularly useful in a game or film music context. Music generated by the MorpheuS system has been performed live in concerts.
  @cite_43: Digital advances have transformed the face of automatic music generation since its beginnings at the dawn of computing. Despite the many breakthroughs, issues such as the musical tasks targeted by different machines and the degree to which they succeed remain open questions. We present a functional taxonomy for music generation systems with reference to existing systems. The taxonomy organizes systems according to the purposes for which they were designed. It also reveals the inter-relatedness amongst the systems. This design-centered approach contrasts with predominant methods-based surveys and facilitates the identification of grand challenges to set the stage for new breakthroughs.
  @cite_2: We consider the problem of extracting essential ingredients of music signals, such as a well-defined global temporal structure in the form of nested periodicities (or meter). We investigate whether we can construct an adaptive signal processing device that learns by example how to generate new instances of a given musical style. Because recurrent neural networks (RNNs) can, in principle, learn the temporal structure of a signal, they are good candidates for such a task. Unfortunately, music composed by standard RNNs often lacks global coherence. The reason for this failure seems to be that RNNs cannot keep track of temporally distant events that indicate global music structure. Long short-term memory (LSTM) has succeeded in similar domains where other RNNs have failed, such as timing and counting and the learning of context sensitive languages. We show that LSTM is also a good mechanism for learning to compose music. We present experimental results showing that LSTM successfully learns a form of blues music and is able to compose novel (and we believe pleasing) melodies in that style. Remarkably, once the network has found the relevant structure, it does not drift from it: LSTM is able to play the blues with good timing and proper structure as long as one is willing to listen.
  @cite_5: We apply deep learning methods, specifically long short-term memory (LSTM) networks, to music transcription modelling and composition. We build and train LSTM networks using approximately 23,000 music transcriptions expressed with a high-level vocabulary (ABC notation), and use them to generate new transcriptions. Our practical aim is to create music transcription models useful in particular contexts of music composition. We present results from three perspectives: 1) at the population level, comparing descriptive statistics of the set of training transcriptions and generated transcriptions; 2) at the individual level, examining how a generated transcription reflects the conventions of a music practice in the training transcriptions (Celtic folk); 3) at the application level, using the system for idea generation in music composition. We make our datasets, software and sound examples open and available: this https URL .
  @cite_20: This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content. We propose a methodology based on five dimensions for our analysis: Objective - What musical content is to be generated? Examples are: melody, polyphony, accompaniment or counterpoint. - For what destination and for what use? To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file). Representation - What are the concepts to be manipulated? Examples are: waveform, spectrogram, note, chord, meter and beat. - What format is to be used? Examples are: MIDI, piano roll or text. - How will the representation be encoded? Examples are: scalar, one-hot or many-hot. Architecture - What type(s) of deep neural network is (are) to be used? Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks. Challenge - What are the limitations and open challenges? Examples are: variability, interactivity and creativity. Strategy - How do we model and control the process of generation? Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation. For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described and are used to exemplify the various choices of objective, representation, architecture, challenge and strategy. The last section includes some discussion and some prospects.
  @cite_38: We introduce a method for imposing higher-level structure on generated, polyphonic music. A Convolutional Restricted Boltzmann Machine (C-RBM) as a generative model is combined with gradient des- cent constraint optimisation to provide further control over the genera- tion process. Among other things, this allows for the use of a “template” piece, from which some structural properties can be extracted, and trans- ferred as constraints to the newly generated material. The sampling pro- cess is guided with Simulated Annealing to avoid local optima, and to find solutions that both satisfy the constraints, and are relatively stable with respect to the C-RBM. Results show that with this approach it is possible to control the higher-level self-similarity structure, the meter, and the tonal properties of the resulting musical piece, while preserving its local musical coherence.
  @cite_18: As one of our highest expressions of thought and creativity, music has always been a difficult realm to capture, model, and understand. The connectionist paradigm, now beginning to provide insights into many realms of human behavior, offers a new and unified viewpoint from which to investigate the subtleties of musical experience. Music and Connectionism provides a fresh approach to both fields, using the techniques of connectionism and parallel distributed processing to look at a wide range of topics in music research, from pitch perception to chord fingering to composition.The contributors, leading researchers in both music psychology and neural networks, address the challenges and opportunities of musical applications of network models. The result is a current and thorough survey of the field that advances understanding of musical phenomena encompassing perception, cognition, composition, and performance, and in methods for network design and analysis.Peter M. Todd is a doctoral candidate in the PDP Research Group of the Psychology Department at Stanford University. Gareth Loy is an award-winning composer, a lecturer in the Music Department of the University of California, San Diego, and a member of the technical staff of Frox Inc.Contributors. Jamshed J. Bharucha. Peter Desain. Mark Dolson. Robert Gjerclingen. Henkjan Honing. B. Keith Jenkins. Jacqueline Jons. Douglas H. Keefe. Tuevo Kohonen. Bernice Laden. Pauli Laine. Otto Laske. Marc Leman. J. P. Lewis. Christoph Lischka. D. Gareth Loy. Ben Miller. Michael Mozer. Samir I. Sayegh. Hajime Sano. Todd Soukup. Don Scarborough. Kalev Tiits. Peter M. Todd. Kari Torkkola.
  @cite_4: 
  @cite_17: HARMONET, a system employing connectionist networks for music processing, is presented. After being trained on some dozen Bach chorales using error backpropagation, the system is capable of producing four-part chorales in the style of J.S. Bach, given a one-part melody. Our system solves a musical real-world problem on a performance level appropriate for musical practice. HARMONET's power is based on (a) a new coding scheme capturing musically relevant information and (b) the integration of backpropagation and symbolic algorithms in a hierarchical system, combining the advantages of both.
  @cite_28: 
  @cite_32: We investigate the problem of modeling symbolic sequences of polyphonic music in a completely general piano-roll representation. We introduce a probabilistic model based on distribution estimators conditioned on a recurrent neural network that is able to discover temporal dependencies in high-dimensional sequences. Our approach outperforms many traditional models of polyphonic music on a variety of realistic datasets. We show how our musical language model can serve as a symbolic prior to improve the accuracy of polyphonic transcription.
  @cite_6: 
  @cite_19: 
  @cite_12: 
  @cite_0: 
  @cite_11: We propose a system, the Continuator, that bridges the gap between two classes of traditionally incompatible musical systems: (1) interactive musical systems, limited in their ability to generate stylistically consistent material, and (2) music imitation systems, which are fundamentally not interactive. Our purpose is to allow musicians to extend their technical ability with stylistically consistent, automatically learnt material. This goal requires the ability for the system to build operational representations of musical styles in a real time context. Our approach is based on a Markov model of musical styles augmented to account for musical issues such as management of rhythm, beat, harmony, and imprecision. The resulting system is able to learn and generate music in any style, either in standalone mode, as continuations of musician’s input, or as interactive improvisation back up. Lastly, the very design of the system makes possible new modes of musical collaborative playing. We describe the architectu...
summary: Two centuries later, as the foundations of AI were being set, the notion of automatically understanding (and therefore generating) music was among the earliest applications to capture the imagination of researchers, with papers on computational approaches to perception, interpretation and generation of music by Simon, Longuet-Higgins and others @cite_4 @cite_12 @cite_30 @cite_28 @cite_6 . Since then, many interesting efforts were made @cite_42 @cite_18 @cite_22 @cite_2 @cite_11 @cite_17 , and it is clear that in recent years both interest and progress in score generation has continued to advance, e.g. @cite_38 , Boulanger- @cite_32 , @cite_0 , @cite_41 , @cite_19 , Sturm @cite_5 , to name only a few. @cite_20 provide a survey of generative music models that involve machine learning. @cite_43 provide a comprehensive survey and satisfying taxonomy of music generation systems. McDonald @cite_3 gives an overview highlighting some key examples of such work.