espnet.utils.training.iterators.ToggleableShufflingMultiprocessIterator

source

class espnet.utils.training.iterators.ToggleableShufflingMultiprocessIterator(dataset, batch_size, repeat=True, shuffle=True, n_processes=None, n_prefetch=1, shared_mem=None, maxtasksperchild=20)

Bases: MultiprocessIterator

MultiprocessIterator having its shuffling property activated during training.

Init the iterator.

• Parameters:
  • dataset (torch.nn.Tensor) – The dataset to take batches from
  • batch_size (int) – The batch size
  • repeat (bool) – Whether to repeat batches or not (enables multiple epochs)
  • shuffle (bool) – Whether to shuffle the order of the batches
  • n_processes (int) – How many processes to use
  • n_prefetch (int) – The number of prefetch to use
  • shared_mem (int) – How many memory to share between processes
  • maxtasksperchild (int) – Maximum number of tasks per child

start_shuffle()

Start shuffling (or reshuffles) the batches.