espnet.nets.scorers.ngram.Ngrambase

source

class espnet.nets.scorers.ngram.Ngrambase(ngram_model, token_list)

Bases: ABC

Ngram base implemented through ScorerInterface.

Initialize Ngrambase.

• Parameters:
  • ngram_model – ngram model path
  • token_list – token list from dict or model.json

init_state(x)

Initialize tmp state.

score_partial_(y, next_token, state, x)

Score interface for both full and partial scorer.

• Parameters:
  • y – previous char
  • next_token – next token need to be score
  • state – previous state
  • x – encoded feature
• Returns: Tuple of : batchfied scores for next token with shape of (n_batch, n_vocab) and next state list for ys.
• Return type: tuple[torch.Tensor, List[Any]]