espnet2.tok.hifigan_loss.HiFiGANReconstructionLoss

source

class espnet2.tok.hifigan_loss.HiFiGANReconstructionLoss(num_tokens: int, token_embed_dim: int = 512, speaker_embed_dim: int = 512, segment_size: int = 32, generator_params: Dict[str, Any] | None = None, discriminator_params: Dict[str, Any] | None = None, generator_adv_loss_params: Dict[str, Any] | None = None, discriminator_adv_loss_params: Dict[str, Any] | None = None, feat_match_loss_params: Dict[str, Any] | None = None, mel_loss_params: Dict[str, Any] | None = None, use_feat_match_loss: bool = True, use_mel_loss: bool = True, lambda_adv: float = 1.0, lambda_feat_match: float = 2.0, lambda_mel: float = 45.0, cache_generator_outputs: bool = True)

Bases: Module

Reconstruct speech from differentiable tokens with HiFi-GAN.

The loss owns its token embedding, speaker-conditioned generator, discriminators, reconstruction losses, and reusable generator-output cache.

Token assignments are embedded differentiably. A pretrained speaker embedding is repeated over token frames and concatenated with the token embedding before the standard ESPnet HiFi-GAN generator. Generator, discriminator, and mel-loss parameters are supplied by the recipe so that sampling-rate and frontend-specific settings remain in the configuration.

When generator outputs are cached during training, call forward() followed by forward_discriminator() with the same speech tensor. The cache is single-use, and an invalid call order or a different batch raises an error instead of silently reusing stale generated waveforms.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(tokenizer_output: SpeechTokenizerOutput, speech: Tensor, spembs: Tensor, **batch: Tensor) → Tuple[Tensor, Dict[str, Tensor], Tensor]

Compute the generator-side vocoder loss.

forward_discriminator(tokenizer_output: SpeechTokenizerOutput | None, speech: Tensor, spembs: Tensor, **batch: Tensor) → Tuple[Tensor, Dict[str, Tensor], Tensor]

Compute discriminator loss, reusing cached generated waveform.

property has_cached_generator_outputs : bool

Return whether a discriminator turn can reuse generated waveforms.

property is_adversarial : bool

Return that this loss requires a discriminator optimizer.

synthesize(tokenizer_output: SpeechTokenizerOutput, spembs: Tensor, **batch: Tensor) → Tuple[Tensor, Tensor]

Synthesize full waveforms from tokens and speaker embeddings.

property upsample_factor : int

Return the number of waveform samples generated per token frame.