<jupyter_start><jupyter_text>Tacotron 2 inference code Edit the variables **checkpoint_path** and **text** to match yours and run the entire code to generate plots of mel outputs, alignments and audio synthesis from the generated mel-spectrogram using Griffin-Lim.  Import libraries and setup matplotlib<jupyter_code>import matplotlib

import matplotlib.pylab as plt

import IPython.display as ipd

import sys
sys.path.append('waveglow/')
import numpy as np
import torch

from hparams import create_hparams
from model import Tacotron2
from layers import TacotronSTFT, STFT
from audio_processing import griffin_lim
from train import load_model
from text import text_to_sequence
from denoiser import Denoiser
def plot_data(data, figsize=(16, 4)):
    fig, axes = plt.subplots(1, len(data), figsize=figsize)
    for i in range(len(data)):
        axes[i].imshow(data[i], aspect='auto', origin='lower', 
                       interpolation='none')
    plt.show()<jupyter_output><empty_output><jupyter_text>Setup hparams<jupyter_code>from hydra import compose, initialize
from omegaconf import OmegaConf

initialize(config_path="conf", job_name="test_app")
cfg = compose(config_name="config")<jupyter_output><empty_output><jupyter_text>Load model from checkpoint<jupyter_code>checkpoint_path = "outputs/2022-03-24/00-58-40/output_dir/checkpoint_36000"
model = load_model(cfg)
model.load_state_dict(torch.load(checkpoint_path)['state_dict'])
_ = model.cuda().eval().half()<jupyter_output><empty_output><jupyter_text>Load WaveGlow for mel2audio synthesis and denoiser<jupyter_code>waveglow_path = 'waveglow/waveglow_256channels_universal_v5.pt'
waveglow = torch.load(waveglow_path)['model']
waveglow.cuda().eval().half()
for k in waveglow.convinv:
    k.float()
denoiser = Denoiser(waveglow)<jupyter_output>/home/nakata/.local/share/virtualenvs/tacotron2-YLMXJfpO/lib/python3.9/site-packages/torch/serialization.py:786: SourceChangeWarning: source code of class 'torch.nn.modules.conv.ConvTranspose1d' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/nakata/.local/share/virtualenvs/tacotron2-YLMXJfpO/lib/python3.9/site-packages/torch/serialization.py:786: SourceChangeWarning: source code of class 'torch.nn.modules.container.ModuleList' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.
  warnings.warn(msg, SourceChangeWarning)
/home/nakata/.local/share/virtualenvs/tacotron2-YLMXJfpO/lib/python3.9/site-packages/torch/serialization.py:786: SourceChangeWarning: source code of[...]<jupyter_text>Prepare text input<jupyter_code>text = "東京特許許可局"
sequence = np.array(text_to_sequence(text, ['english_cleaners']))[None, :]
sequence = torch.autograd.Variable(
    torch.from_numpy(sequence)).cuda().long()<jupyter_output><empty_output><jupyter_text>Decode text input and plot results<jupyter_code>mel_outputs, mel_outputs_postnet, _, alignments = model.inference(sequence)
plot_data((mel_outputs.float().data.cpu().numpy()[0],
           mel_outputs_postnet.float().data.cpu().numpy()[0],
           alignments.float().data.cpu().numpy()[0].T))<jupyter_output>/tmp/ipykernel_29961/3574190110.py:6: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()<jupyter_text>Synthesize audio from spectrogram using WaveGlow<jupyter_code>with torch.no_grad():
    audio = waveglow.infer(mel_outputs_postnet, sigma=0.666)
ipd.Audio(audio[0].data.cpu().numpy(), rate=cfg.sampling_rate)<jupyter_output><empty_output><jupyter_text>(Optional) Remove WaveGlow bias<jupyter_code>audio_denoised = denoiser(audio, strength=0.01)[:, 0]
ipd.Audio(audio_denoised.cpu().numpy(), rate=cfg.sampling_rate) 
import soundfile as sf
sf.write('test.wav', audio_denoised.cpu().numpy().T, cfg.sampling_rate)
audio_denoised.cpu().numpy().shape<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup