  Generative Pre-trained Transformer (GPT) models have achieved remarkable
performance on various natural language processing tasks. However, there has
been limited research on applying similar frameworks to audio tasks. Previously
proposed large language models for audio tasks either lack sufficient
quantitative evaluations, or are limited to tasks for recognizing and
understanding audio content, or significantly underperform existing
state-of-the-art (SOTA) models. In this paper, we propose LauraGPT, a unified
GPT model for audio recognition, understanding, and generation. LauraGPT is a
versatile language model that can process both audio and text inputs and
generate outputs in either modalities. It can perform a wide range of tasks
related to content, semantics, paralinguistics, and audio-signal analysis. Some
of its noteworthy tasks include automatic speech recognition, speech-to-text
translation, text-to-speech synthesis, machine translation, speech enhancement,
automated audio captioning, speech emotion recognition, and spoken language
understanding. To achieve this goal, we use a combination of continuous and
discrete features for audio. We encode input audio into continuous
representations using an audio encoder and decode output audio from discrete
codec codes. We then fine-tune a large decoder-only Transformer-based language
model on multiple audio-to-text, text-to-audio, audio-to-audio, and
text-to-text tasks using a supervised multitask learning approach. Extensive
experiments show that LauraGPT achieves competitive or superior performance
compared to existing SOTA models on various audio processing benchmarks.
