espnet-tts-sample
ljspeech.tacotron2.v2
Creator
- Tomoki Hayashi (Nagoya University) <kan-bayashi>
Abstract
This is tts demo of The LJ Speech Dataset [0].
tts1 recipe
tts1 recipe is based on Tacotron2 [1] (spectrogram prediction network) w/o WaveNet.
Tacotron2 generates log mel-filter bank from text and then converts it to linear spectrogram using inverse mel-basis.
Finally, phase components are recovered with Griffin-Lim.
(2019/06/16) we also support TTS-Transformer [3].
(2019/06/17) we also support Feed-forward Transformer [4].
tts2 recipe
tts2 recipe is based on Tacotron2’s spectrogram prediction network [1] and Tacotron’s CBHG module [2].
Instead of using inverse mel-basis, CBHG module is used to convert log mel-filter bank to linear spectrogram.
The recovery of the phase components is the same as tts1.
Model
v.0.4.0: tacotron2.v2
- 1024 pt window
- 256 pt shift
- GL 1000 iters
- R=1
- forward with transition agent
Environments
- date:
Fri Jun 14 10:51:01 JST 2019 - system information:
Linux huracan.sp.m.is.nagoya-u.ac.jp 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux - python version:
Python 3.7.3 - espnet version:
espnet 0.3.1 - chainer version:
chainer 5.0.0 - pytorch version:
pytorch 1.0.1.post2 - Git hash:
c86e9311641f59fa397912d0bd2b9c0c599a1127
Model files
- model link: https://drive.google.com/open?id=11T9qw8rJlYzUdXvFjkjQjYrp3iGfQ15h
- training config file:
conf/tuning/train_pytorch_tacotron2.v2.yaml - decoding config file:
conf/decode.yaml - cmvn file:
data/train_no_dev/cmvn.ark - e2e file:
exp/train_no_dev_pytorch_train_pytorch_tacotron2.v2/results/model.last1.avg.best - e2e JSON file:
exp/train_no_dev_pytorch_train_pytorch_tacotron2.v2/results/model.json - dict file:
data/lang_1char/train_no_dev_units.txt
Audio samples
- ground_truth: Recorded speech
- tacotron2.v2_GL: Synthesized speech (Feature generetion:tacotron2.v2, Waveform synthesis: Griffin-Lim algorithm)
- tacotron2.v2_WNV: Synthesized speech (Feature generetion:tacotron2.v2, Waveform synthesis: WaveNet vocoder)
* The recommended browser for Audio player: Google Chrome
Sample1
LJ050-0029 “THAT IS REFLECTED IN DEFINITE AND COMPREHENSIVE OPERATING PROCEDURES. “
| ground_truth | tacotron2.v2_GL | tacotron2.v2_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample2
LJ050-0030 “THE COMMISSION ALSO RECOMMENDS “
| ground_truth | tacotron2.v2_GL | tacotron2.v2_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample3
LJ050-0031 “THAT THE SECRET SERVICE CONSCIOUSLY SET ABOUT THE TASK OF INCULCATING AND MAINTAINING THE HIGHEST STANDARD OF EXCELLENCE AND ESPRIT, FOR ALL OF ITS PERSONNEL. “
| ground_truth | tacotron2.v2_GL | tacotron2.v2_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample4
LJ050-0032 “THIS INVOLVES TIGHT AND UNSWERVING DISCIPLINE AS WELL AS THE PROMOTION OF AN OUTSTANDING DEGREE OF DEDICATION AND LOYALTY TO DUTY. “
| ground_truth | tacotron2.v2_GL | tacotron2.v2_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample5
LJ050-0033 “THE COMMISSION EMPHASIZES THAT IT FINDS NO CAUSAL CONNECTION BETWEEN THE ASSASSINATION “
| ground_truth | tacotron2.v2_GL | tacotron2.v2_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Other samples
https://drive.google.com/open?id=1cKPDQjLGs7OD8xopSK3YWIGGth37GRSm
Synthesize speech by arbitrary text
- Go to Google colab (created by Github)
- Run “0. Installation”
- Run “3. Demonstration of the use of pretrained models”
* The recommended browser for Google colab: Google Chrome
Please modify the option about tts model
Before: !../../../utils/synth_wav.sh --models ljspeech.fastspeech.v1 example.txt
After: !../../../utils/synth_wav.sh --models ljspeech.tacotron2.v2 example.txt
References
- [0] “The LJ Speech Dataset,” https://keithito.com/LJ-Speech-Dataset/.
- [1] Shen, Jonathan, et al. “Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions.” arXiv preprint arXiv:1712.05884 (2017).
- [2] Wang, Yuxuan, et al. “Tacotron: Towards end-to-end speech synthesis.” arXiv preprint arXiv:1703.10135 (2017).
- [3] Li, Naihan, et al. “Close to human quality TTS with transformer.” arXiv preprint arXiv:1809.08895 (2018).
- [4] Ren, Yi, et al. “FastSpeech: Fast, Robust and Controllable Text to Speech.” arXiv preprint arXiv:1905.09263 (2019).