espnet2.gan_tts.espnet_model.ESPnetGANTTSModel

source

class espnet2.gan_tts.espnet_model.ESPnetGANTTSModel(feats_extract: AbsFeatsExtract | None, normalize: InversibleInterface | None, pitch_extract: AbsFeatsExtract | None, pitch_normalize: InversibleInterface | None, energy_extract: AbsFeatsExtract | None, energy_normalize: InversibleInterface | None, tts: AbsGANTTS)

Bases: AbsGANESPnetModel

ESPnet model for GAN-based text-to-speech task.

Initialize ESPnetGANTTSModel module.

collect_feats(text: Tensor, text_lengths: Tensor, speech: Tensor, speech_lengths: Tensor, durations: Tensor | None = None, durations_lengths: Tensor | None = None, pitch: Tensor | None = None, pitch_lengths: Tensor | None = None, energy: Tensor | None = None, energy_lengths: Tensor | None = None, spembs: Tensor | None = None, sids: Tensor | None = None, lids: Tensor | None = None, **kwargs) → Dict[str, Tensor]

Calculate features and return them as a dict.

• Parameters:
  • text (Tensor) – Text index tensor (B, T_text).
  • text_lengths (Tensor) – Text length tensor (B,).
  • speech (Tensor) – Speech waveform tensor (B, T_wav).
  • speech_lengths (Tensor) – Speech length tensor (B, 1).
  • durations (Optional *[*Tensor) – Duration tensor.
  • durations_lengths (Optional *[*Tensor) – Duration length tensor (B,).
  • pitch (Optional *[*Tensor) – Pitch tensor.
  • pitch_lengths (Optional *[*Tensor) – Pitch length tensor (B,).
  • energy (Optional *[*Tensor) – Energy tensor.
  • energy_lengths (Optional *[*Tensor) – Energy length tensor (B,).
  • spembs (Optional *[*Tensor ]) – Speaker embedding tensor (B, D).
  • sids (Optional *[*Tensor ]) – Speaker index tensor (B, 1).
  • lids (Optional *[*Tensor ]) – Language ID tensor (B, 1).
• Returns: Dict of features.
• Return type: Dict[str, Tensor]

forward(text: Tensor, text_lengths: Tensor, speech: Tensor, speech_lengths: Tensor, durations: Tensor | None = None, durations_lengths: Tensor | None = None, pitch: Tensor | None = None, pitch_lengths: Tensor | None = None, energy: Tensor | None = None, energy_lengths: Tensor | None = None, spembs: Tensor | None = None, sids: Tensor | None = None, lids: Tensor | None = None, forward_generator: bool = True, **kwargs) → Dict[str, Any]

Return generator or discriminator loss with dict format.

• Parameters:
  • text (Tensor) – Text index tensor (B, T_text).
  • text_lengths (Tensor) – Text length tensor (B,).
  • speech (Tensor) – Speech waveform tensor (B, T_wav).
  • speech_lengths (Tensor) – Speech length tensor (B,).
  • duration (Optional *[*Tensor ]) – Duration tensor.
  • duration_lengths (Optional *[*Tensor ]) – Duration length tensor (B,).
  • pitch (Optional *[*Tensor ]) – Pitch tensor.
  • pitch_lengths (Optional *[*Tensor ]) – Pitch length tensor (B,).
  • energy (Optional *[*Tensor ]) – Energy tensor.
  • energy_lengths (Optional *[*Tensor ]) – Energy length tensor (B,).
  • spembs (Optional *[*Tensor ]) – Speaker embedding tensor (B, D).
  • sids (Optional *[*Tensor ]) – Speaker ID tensor (B, 1).
  • lids (Optional *[*Tensor ]) – Language ID tensor (B, 1).
  • forward_generator (bool) – Whether to forward generator.
  • kwargs – “utt_id” is among the input.
• Returns:
  • loss (Tensor): Loss scalar tensor.
  • stats (Dict[str, float]): Statistics to be monitored.
  • weight (Tensor): Weight tensor to summarize losses.
  • optim_idx (int): Optimizer index (0 for G and 1 for D).
• Return type: Dict[str, Any]