espnet3.systems.f5tts.src.vocoder_mel.VocoderMelSpec

source

class espnet3.systems.f5tts.src.vocoder_mel.VocoderMelSpec(fs: int = 24000, n_fft: int = 1024, hop_length: int = 256, win_length: int = 1024, n_mels: int = 100)

Bases: AbsFeatsExtract

F5-TTS mel spectrogram (Vocos target) as an AbsFeatsExtract.

Configure the mel front end.

• Parameters:
  • fs – Sampling rate of the input waveform.
  • n_fft – FFT size.
  • hop_length – STFT hop in samples.
  • win_length – STFT window length in samples.
  • n_mels – Number of mel bins, which becomes the model’s mel dimension.

Example

feats_extract_conf:
  fs: 24000
  n_fft: 1024
  hop_length: 256
  win_length: 1024
  n_mels: 100

######## NOTE The mel must match the vocoder used at inference, so these values are fixed by the vocoder rather than freely tunable. n_mels becomes the model’s mel dimension via output_size().

forward(input: Tensor, input_lengths: Tensor | None = None) → Tuple[Tensor, Tensor]

Turn waveform [B, T_wav] into mel [B, T, n_mels] and lengths.

feats_lengths follows the frame count of the centre-padded STFT, T_wav // hop + 1 (the same formula espnet2’s Stft uses).

• Parameters:
  • input – Waveform batch [B, T_wav].
  • input_lengths – Valid sample count per utterance, [B]. When omitted every utterance is treated as full length.
• Returns: Tuple of mel [B, T, n_mels] and frame counts [B].

Example

>>> feats, lengths = VocoderMelSpec()(torch.randn(2, 24000))
>>> feats.shape, lengths.tolist()
(torch.Size([2, 94, 100]), [94, 94])

######## NOTE Output is time-first [B, T, n_mels], which is what the model consumes, not the channel-first layout F5’s own MelSpec returns.

get_parameters() → Dict[str, Any]

Parameters a vocoder needs to reconstruct waveforms from these feats.

• Returns: Dict with fs, n_fft, n_shift, win_length and n_mels.

Example

>>> sorted(VocoderMelSpec().get_parameters())
['fs', 'n_fft', 'n_mels', 'n_shift', 'win_length']

######## NOTE The hop is reported as n_shift, espnet2’s name for it, not as hop_length. Downstream espnet2 vocoder tooling expects that key.

output_size() → int

Return the mel dimension the model generates.

• Returns:n_mels.

Example

>>> VocoderMelSpec(n_mels=100).output_size()
100

######## NOTE A method, not a property, because AbsFeatsExtract declares it as one and espnet2 calls it as feats_extract.output_size(). F5TTS reads it to size its backbone, which is why the recipe states the mel dimension only once, in feats_extract_config.