espnet3.systems.f5tts.inference.F5TTSInference

source

class espnet3.systems.f5tts.inference.F5TTSInference(train_config: str, checkpoint_path: str, device: str = 'cpu', use_ema: bool = True, vocoder_path: str | None = None, target_sample_rate: int = 24000, ode_solver_steps: int = 32, guidance_strength: float = 2.0, sway_sampling_coefficient: float = -1.0, speed: float = 1.0, target_rms: float = 0.1, cross_fade_duration: float = 0.15, seed: int | None = None)

Bases: object

Load a trained F5-TTS model + vocoder and synthesize waveforms.

Build the model, tokenizer and vocoder for inference.

• Parameters:
  • train_config – Path to the training YAML (provides the model block + preprocessor tokenization settings, single source of truth).
  • checkpoint_path – Lightning checkpoint (.ckpt) from training.
  • device – Torch device string.
  • use_ema – Load EMA-averaged weights (ema_model_state_dict) when present; otherwise the raw state_dict.
  • vocoder_path – Local directory holding the Vocos config.yaml and pytorch_model.bin. When unset the default checkpoint is fetched from the Hugging Face Hub.
  • target_sample_rate – Output/vocoder sample rate.
  • ode_solver_steps – Number of ODE solver steps, upstream’s nfe_step (number of function evaluations).
  • guidance_strength – Classifier-free guidance scale, upstream’s cfg_strength.
  • seed (sway_sampling_coefficient / speed /) – Remaining sampling hyperparameters forwarded to CFM.sample.
• Raises:ValueError – If train_config has no model._target_ or if its dataset.preprocessor.token_list is missing.

Example

inference:
  _target_: espnet3.systems.f5tts.src.f5tts.F5TTS
  train_config: ${recipe_dir}/conf/training_f5_tts_small.yaml
  checkpoint_path: ${exp_dir}/last.ckpt
  device: cuda
  ode_solver_steps: 32
  guidance_strength: 2.0

NOTE

train_config is the recipe’s own training YAML, not the config.yaml written into exp_dir: the model is rebuilt from the model: block via Hydra, so architecture and tokenizer settings always come from one source of truth. Construction is eager, loading the checkpoint and the vocoder up front.

infer_one(target_text: str, reference_audio: ndarray, reference_text: str | None = None) → ndarray

Synthesize target_text in the voice of reference_audio.

Mirrors upstream F5-TTS infer_process: RMS-normalize the reference, split the target text into reference-length-dependent chunks, sample each chunk, vocode, then cross-fade the pieces back together.

• Parameters:
  • target_text – Target text to speak.
  • reference_audio – Reference waveform at target_sample_rate. Multi- channel input is averaged down to mono.
  • reference_text – Transcript of reference_audio. Defaults to target_text, treating the reference as self-referential.
• Returns: Mono waveform as float32 at target_sample_rate. Returns a single zero sample when the text yields no synthesizable chunk.

Example

>>> wav = tts.infer_one(
...     "hello world", reference_audio, reference_text="hi"
... )
>>> wav.dtype, wav.ndim
(dtype('float32'), 1)

NOTE

Output length is governed by the reference: the per-chunk duration is extrapolated from the reference audio-to-text ratio, so a mismatched reference_text skews it. The reference loudness is restored after vocoding, so the result matches the input level rather than target_rms.