espnet3.components.data.base_dataset.BaseDataset

source

class espnet3.components.data.base_dataset.BaseDataset

Bases: Dataset, ABC

What a recipe’s dataset promises: the keys and kinds of one item.

A subclass declares fields - a class attribute when every instance’s item shape is the same, or set on self in __init__ when it depends on the dataset’s own arguments - and implements __getitem__. The first call to __getitem__ on each instance is checked against the declaration; later calls are not, so checking adds no cost once a dataset is known good.

Examples

>>> class MiniDataset(BaseDataset):
...     fields = (Field("utt_id", "text"), Field("text", "text"))
...     def __init__(self):
...         self.items = [{"utt_id": "u1", "text": "hi"}]
...     def __len__(self):
...         return len(self.items)
...     def __getitem__(self, index):
...         return self.items[index]
>>> MiniDataset()[0]["text"]
'hi'

fields : ClassVar[Tuple[Field, ...]]