espnet3.systems.esp2_cls.src.audio_conversion_runner.AudioConversionRunner
espnet3.systems.esp2_cls.src.audio_conversion_runner.AudioConversionRunner
class espnet3.systems.esp2_cls.src.audio_conversion_runner.AudioConversionRunner(provider: EnvironmentProvider, batch_size: int | None = None, output_dir: str | Path | None = None, shard_subdir: str = '', resume: bool = True)
Bases: BaseRunner
Runner for converting source clips to WAV in parallel.
Conversion dominates create_dataset on corpora distributed as video, where converting clips one at a time is slow. Each clip is independent, so the work shards cleanly.
Each shard appends its status dicts to a shard-local results.jsonl file, and merge() reads every shard file back and re-sorts by idx, so callers receive results in job order regardless of shard completion order.
Initialize BaseRunner object.
static forward(idx: int | Iterable[int], jobs: List[Tuple[str, str]], ffmpeg: str, sampling_rate: int, channels: int, **env) → Dict[str, Any] | list
Convert the clip at the given index or batch of indices.
- Parameters:
- idx – Single index or iterable of indices into
jobs. - jobs – List of
(source_path, destination_path)pairs. - ffmpeg – Path to the
ffmpegexecutable. - sampling_rate – Target sampling rate in Hz.
- channels – Target channel count.
- **env – Additional environment entries.
- idx – Single index or iterable of indices into
- Returns: A status dict for an int index, or a list of status dicts for an iterable. Each entry is
{"idx": int, "path": str, "converted": bool}, whereconvertedisFalsefor a destination that already existed. - Raises:subprocess.CalledProcessError – If
ffmpegfails on a clip.
merge(shard_dirs: List[Path]) → List[Dict[str, Any]]
Concatenate shard results and restore job (idx) order.
Each shard’s results.jsonl holds one JSON object per line, e.g.:
{"idx": 0, "path": "data/wav/train/a.wav", "converted": true}
{"idx": 1, "path": "data/wav/train/b.wav", "converted": false}static open_writers(shard_dir: Path | None, **env) → Dict[str, Any]
Open the shard-local JSONL results file.
static write_record(writers: Dict[str, Any], result: Any, state: Dict[str, Any], **env) → None
Append one forward result (or batch of results) to the shard file.
