espnet3.systems.esp2_cls.metrics.macro_f1.MacroF1
espnet3.systems.esp2_cls.metrics.macro_f1.MacroF1
class espnet3.systems.esp2_cls.metrics.macro_f1.MacroF1(token_list: str | None = None)
Bases: BaseMetric
Compute the macro-averaged F1 score for a classification dataset.
The F1 score of each class is computed independently and then averaged without weighting by support, so rare classes are not drowned out.
Declare it in conf/metrics.yaml:
- metric:
_target_: espnet3.systems.esp2_cls.metrics.macro_f1.MacroF1
token_list: data/token_list # optional; fixes the class set
inputs:
ref: dataset:label
hyp: hypInitialize the macro F1 metric.
- Parameters:token_list – Optional path to the recipe’s token list. It fixes the class set so unseen labels are caught; without it the classes are taken from the references.
compute() → Dict[str, float]
Return the macro-averaged F1 score of every utterance added so far.
- Returns:
{"MacroF1": <percentage>}, rounded to 2 places. - Return type: Dict[str, float]
- Raises:
- RuntimeError – If
scikit-learnis not installed. - ValueError – If no utterance was added, an utterance carries multiple labels, or a reference label is missing from
token_list.
- RuntimeError – If
####### Examples
>>> metric = MacroF1()
>>> metric.update(ref=["joy", "anger"], hyp=["joy", "joy"])
>>> metric.compute()
{'MacroF1': 33.33}inputs : ClassVar[Tuple[Field, ...]] = (Field(name='ref', kind='text', label='Ref', optional=False, channels=None), Field(name='hyp', kind='text', label='Hyp', optional=False, channels=None))
one Field per SCP input. The name is the key of data (and the default SCP file name); the kind says what the SCP values hold (text: the value itself, audio: a path). A class attribute for a fixed contract; set on self instead, before calling super().__init__(), for one built from the instance’s own configuration.
- Type: What this metric reads
outputs : ClassVar[Tuple[Field, ...]] = (Field(name='MacroF1', kind='number', label='Macrof1', optional=False, channels=None),)
one Field per key of the result dict, all of kind number. Same class-attribute-or-self choice as inputs.
- Type: What this metric returns
update(, ref: Sequence[str], hyp: Sequence[str]) → None
Keep the labels of one batch; the score needs every utterance.
- Parameters:
- ref – Reference labels, one per utterance.
- hyp – Predicted labels, aligned with
ref.
####### Examples
>>> metric = MacroF1()
>>> metric.update(ref=["joy", "anger"], hyp=["joy", "joy"])