espnet3.systems.esp2_cls.metrics.ua.UA
espnet3.systems.esp2_cls.metrics.ua.UA
class espnet3.systems.esp2_cls.metrics.ua.UA(token_list: str | None = None)
Bases: BaseMetric
Compute unweighted accuracy (UA) for a classification dataset.
UA is the macro average of the per-class recall, so every class counts equally regardless of how many examples it has. The emotion recognition literature also calls it unweighted average recall (UAR), and scikit-learn calls it balanced accuracy. On the imbalanced label distributions typical of the task it is the more informative companion to espnet3.systems.esp2_cls.metrics.wa.WA.
Declare it in conf/metrics.yaml:
- metric:
_target_: espnet3.systems.esp2_cls.metrics.ua.UA
token_list: data/token_list # optional; fixes the class set
inputs:
ref: dataset:label
hyp: hypInitialize the UA metric.
- Parameters:token_list – Optional path to the recipe’s token list. It fixes the class set so unseen labels are caught; without it the classes are taken from the references.
compute() → Dict[str, float]
Return the macro-averaged recall of every utterance added so far.
- Returns:
{"UA": <percentage>}, rounded to 2 places. - Return type: Dict[str, float]
- Raises:
- RuntimeError – If
scikit-learnis not installed. - ValueError – If no utterance was added, an utterance carries multiple labels, or a reference label is missing from
token_list.
- RuntimeError – If
####### Examples
>>> metric = UA()
>>> metric.update(ref=["joy", "anger"], hyp=["joy", "joy"])
>>> metric.compute()
{'UA': 50.0}inputs : ClassVar[Tuple[Field, ...]] = (Field(name='ref', kind='text', label='Ref', optional=False, channels=None), Field(name='hyp', kind='text', label='Hyp', optional=False, channels=None))
one Field per SCP input. The name is the key of data (and the default SCP file name); the kind says what the SCP values hold (text: the value itself, audio: a path). A class attribute for a fixed contract; set on self instead, before calling super().__init__(), for one built from the instance’s own configuration.
- Type: What this metric reads
outputs : ClassVar[Tuple[Field, ...]] = (Field(name='UA', kind='number', label='Ua', optional=False, channels=None),)
one Field per key of the result dict, all of kind number. Same class-attribute-or-self choice as inputs.
- Type: What this metric returns
update(, ref: Sequence[str], hyp: Sequence[str]) → None
Keep the labels of one batch; the score needs every utterance.
- Parameters:
- ref – Reference labels, one per utterance.
- hyp – Predicted labels, aligned with
ref.
####### Examples
>>> metric = UA()
>>> metric.update(ref=["joy", "anger"], hyp=["joy", "joy"])