espnet3.components.data.collect_stats.collect_stats

source

espnet3.components.data.collect_stats.collect_stats(model_config, dataset_config, dataloader_config, mode: str, output_dir: Path, task: str | None = None, parallel_config: DictConfig | None = None, write_collected_feats: bool = False, batch_size: int = 4, train_mode: bool = False)

Entry point for collecting dataset statistics used for feature normalization.

Runs the runner-based collection once, optionally configuring parallel execution via espnet3.parallel.set_parallel() when parallel_config is provided.

• Parameters:
  • model_config – Configuration object used to instantiate the model that extracts features from the input examples.
  • dataset_config – Configuration of the dataset organizer providing the split specified by mode.
  • dataloader_config – Dataloader configuration.
  • mode – Name of the dataset split to process (train or valid).
  • output_dir – Directory where aggregated statistics and optionally collected features are written.
  • task – Name of the ESPnet task. If None, model_config should be directly instantiable.
  • parallel_config – Configuration for parallel execution.
  • write_collected_feats – Whether to persist the raw collected features.
  • batch_size – Number of dataset items processed per batch.
  • train_mode – Build the train split’s ESPnet preprocessor in training mode, so its random augmentation is applied. False by default, as in ESPnet2: the statistics and shape files are then the same on every run and match the unaugmented input the model sees at inference.
• Returns: Aggregated statistics are saved under output_dir / mode.
• Return type: None