espnet-tts-sample
ljspeech.transformer.v1
Creator
- Tomoki Hayashi (Nagoya University) <kan-bayashi>
Abstract
This is tts demo of The LJ Speech Dataset [0].
tts1 recipe
tts1 recipe is based on Tacotron2 [1] (spectrogram prediction network) w/o WaveNet.
Tacotron2 generates log mel-filter bank from text and then converts it to linear spectrogram using inverse mel-basis.
Finally, phase components are recovered with Griffin-Lim.
(2019/06/16) we also support TTS-Transformer [3].
(2019/06/17) we also support Feed-forward Transformer [4].
tts2 recipe
tts2 recipe is based on Tacotron2’s spectrogram prediction network [1] and Tacotron’s CBHG module [2].
Instead of using inverse mel-basis, CBHG module is used to convert log mel-filter bank to linear spectrogram.
The recovery of the phase components is the same as tts1.
Model
v.0.4.0: transformer.v1
- 1024 pt window
- 256 pt shift
- GL 1000 iters
- R=1
- Large
Environments
- date:
Sun Jun 16 10:03:47 JST 2019 - system information:
Linux huracan.sp.m.is.nagoya-u.ac.jp 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux - python version:
Python 3.7.3 - espnet version:
espnet 0.3.1 - chainer version:
chainer 5.0.0 - pytorch version:
pytorch 1.0.1.post2 - Git hash:
267da3161cefeae72e9a44bd15e74c0d18591fb6
Model files
- model link: https://drive.google.com/open?id=13DR-RB5wrbMqBGx_MC655VZlsEq52DyS
- training config file:
conf/tuning/train_pytorch_transformer.v1.yaml - decoding config file:
conf/decode.yaml - cmvn file:
data/train_no_dev/cmvn.ark - e2e file:
exp/train_no_dev_pytorch_train_pytorch_transformer.v1/results/model.last1.avg.best - e2e JSON file:
exp/train_no_dev_pytorch_train_pytorch_transformer.v1/results/model.json - dict file:
data/lang_1char/train_no_dev_units.txt
Audio samples
- ground_truth: Recorded speech
- transformer.v1_GL: Synthesized speech (Feature generetion:transformer.v1, Waveform synthesis: Griffin-Lim algorithm)
- transformer.v1_WNV: Synthesized speech (Feature generetion:transformer.v1, Waveform synthesis: WaveNet vocoder)
* The recommended browser for Audio player: Google Chrome
Sample1
LJ050-0029 “THAT IS REFLECTED IN DEFINITE AND COMPREHENSIVE OPERATING PROCEDURES. “
| ground_truth | transformer.v1_GL | transformer.v1_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample2
LJ050-0030 “THE COMMISSION ALSO RECOMMENDS “
| ground_truth | transformer.v1_GL | transformer.v1_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample3
LJ050-0031 “THAT THE SECRET SERVICE CONSCIOUSLY SET ABOUT THE TASK OF INCULCATING AND MAINTAINING THE HIGHEST STANDARD OF EXCELLENCE AND ESPRIT, FOR ALL OF ITS PERSONNEL. “
| ground_truth | transformer.v1_GL | transformer.v1_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample4
LJ050-0032 “THIS INVOLVES TIGHT AND UNSWERVING DISCIPLINE AS WELL AS THE PROMOTION OF AN OUTSTANDING DEGREE OF DEDICATION AND LOYALTY TO DUTY. “
| ground_truth | transformer.v1_GL | transformer.v1_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Sample5
LJ050-0033 “THE COMMISSION EMPHASIZES THAT IT FINDS NO CAUSAL CONNECTION BETWEEN THE ASSASSINATION “
| ground_truth | transformer.v1_GL | transformer.v1_WNV |
|---|---|---|
| Attention wight | Probility |
|---|---|
 |
 |
Other samples
https://drive.google.com/open?id=14EboYVsMVcAq__dFP1p6lyoZtdobIL1X
Synthesize speech by arbitrary text
- Go to Google colab (created by Github)
- Run “0. Installation”
- Run “3. Demonstration of the use of pretrained models”
* The recommended browser for Google colab: Google Chrome
Please modify the option about tts model
Before: !../../../utils/synth_wav.sh --models ljspeech.fastspeech.v1 example.txt
After: !../../../utils/synth_wav.sh --models ljspeech.transformer.v1 example.txt
References
- [0] “The LJ Speech Dataset,” https://keithito.com/LJ-Speech-Dataset/.
- [1] Shen, Jonathan, et al. “Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions.” arXiv preprint arXiv:1712.05884 (2017).
- [2] Wang, Yuxuan, et al. “Tacotron: Towards end-to-end speech synthesis.” arXiv preprint arXiv:1703.10135 (2017).
- [3] Li, Naihan, et al. “Close to human quality TTS with transformer.” arXiv preprint arXiv:1809.08895 (2018).
- [4] Ren, Yi, et al. “FastSpeech: Fast, Robust and Controllable Text to Speech.” arXiv preprint arXiv:1905.09263 (2019).