espnet3.systems.esp2_cls.metrics.mean_ap.MAP
espnet3.systems.esp2_cls.metrics.mean_ap.MAP
class espnet3.systems.esp2_cls.metrics.mean_ap.MAP(token_list: str)
Bases: BaseMetric
Compute mean average precision (mAP) from class probabilities.
Average precision is computed per class in a one-vs-rest fashion and then macro averaged, matching the mAP column of espnet2’s cls_score.py. Because it consumes ranked probabilities rather than the arg-max label, it is sensitive to how well the model orders its classes.
It reads the per-class scores in addition to the reference labels. Declare it in conf/metrics.yaml:
- metric:
_target_: espnet3.systems.esp2_cls.metrics.mean_ap.MAP
token_list: data/token_list
inputs:
ref: dataset:label
score: scoreInitialize the mAP metric.
- Parameters:token_list – Path to the recipe’s token list. It defines the order of the columns in
score.scp.
compute() → Dict[str, float]
Return the mean average precision of every utterance added so far.
- Returns:
{"mAP": <percentage>}, rounded to 2 places. - Return type: Dict[str, float]
- Raises:
- RuntimeError – If
scikit-learnis not installed. - ValueError – If a reference label is missing from
token_list, a score row has the wrong length, or no class has a positive reference.
- RuntimeError – If
inputs : ClassVar[Tuple[Field, ...]] = (Field(name='ref', kind='text', label='Ref', optional=False, channels=None), Field(name='score', kind='text', label='Score', optional=False, channels=None))
one Field per SCP input. The name is the key of data (and the default SCP file name); the kind says what the SCP values hold (text: the value itself, audio: a path). A class attribute for a fixed contract; set on self instead, before calling super().__init__(), for one built from the instance’s own configuration.
- Type: What this metric reads
outputs : ClassVar[Tuple[Field, ...]] = (Field(name='mAP', kind='number', label='Map', optional=False, channels=None),)
one Field per key of the result dict, all of kind number. Same class-attribute-or-self choice as inputs.
- Type: What this metric returns
update(, ref: Sequence[str], score: Sequence[str]) → None
Keep the labels and scores of one batch; the score needs them all.
- Parameters:
- ref – Reference labels, one per utterance; space-separated labels are multiple positives.
- score – Space-separated per-class scores, aligned with
ref.
Examples
>>> metric = MAP("data/token_list")
>>> metric.update(ref=["joy"], score=["0.1 0.9"])