espnet2.fileio.omniio_audio.OmniIOEntryReader

source

class espnet2.fileio.omniio_audio.OmniIOEntryReader(always_2d: bool = False, max_cache_fd: int = 32)

Bases: object

Read single <bin path>:<start_byte>:<end_byte> entries.

Examples

>>> read = OmniIOEntryReader()
>>> rate, array = read('/data/format.1/wav/blob_0.bin:0:31052')

Like kaldi_ark with max_cache_fd, the bin files are kept open (up to max_cache_fd of them, least recently used closed first), so reading an entry is a single positional read rather than an open/seek/read/close: every open is a metadata request, which is what is slow on shared file systems. Reads use os.pread, which does not move a shared file offset.

The descriptors belong to the process that opened them: a pickled copy (e.g. a dataset sent to spawned DataLoader workers) starts with an empty cache, and a forked process opens its own instead of using, or closing, the inherited ones.

FLAC, WAV and OGG entries are decoded here exactly as omniio.interface.audio_read decodes them (soundfile, float32, (frames, channels)); WebM/Opus entries are handed to audio_read.

• Parameters:
  • always_2d – Keep the channel axis even for single-channel audio.
  • max_cache_fd – Bin files kept open; 0 opens and closes per read.

Set the output shape and how many bin files are kept open.

close()

Close the bin file descriptors this process opened.

read_segment(entry: str, start: float, end: float) → Tuple[int, ndarray]

Return (rate, signal) of seconds start..`end` of entry.

Cut as SegmentsExtractor cuts a Kaldi segment (end == -1: to the end). FLAC/WAV/OGG are seeked: only the segment is read and decoded. WebM is decoded whole and sliced.