RangeBasedRecall

class dtaianomaly.evaluation.RangeBasedRecall(alpha: float = 0.5, delta: Literal['flat', 'front', 'back', 'middle'] = 'flat', gamma: Literal['one', 'reciprocal'] = 'reciprocal')[source]

Computes the range-based recall score [34].

The range-based recall computes a recall-score for each ground truth anomalous range and then takes the average over all ranges. This recall-score consists of three parts: (1) the amount of overlap between the ground truth range and the predicted ranges, (2) whether the ground truth range overlaps with only one or multiple predicted ranges, and (3) whether the final ground truth range is detected at all. Components (1) and (2) are computed independently and multiplied, of which the result is combined with component (3) through a convex combination to get a final recall-score for the ground truth range.

Parameters:

alphafloat, default=0.5

The importance of detecting the events (even if it is only a single detected point) compared to detecting a large portion of the ground truth events. Should be at least 0 and at most 1.

deltastr, default=’flat’

Bias for the position of the predicted anomaly in the ground truth anomalous range. Valid options are:

'flat': Equal bias towards all positions in the ground truth anomalous range.
'front': Predictions that are near the front of the ground truth anomaly (i.e. early detection) have a higher weight.
'back': Predictions that are near the end of the ground truth anomaly (i.e. late detection) have a higher weight.
'middle': Predictions that are near the center of the ground truth anomaly have a higher weight.

gammastr, default=’reciprocal’

Penalization approach for detecting multiple ranges with a single range. Valid options are:

'one': Fragmented detection should not be penalized.
'reciprocal': Weight fragmented detection of :math:´N´ ranges with as single range by a factor of :math:´1/N´.

Warning

Note that, while tuning a metric to some domain is beneficial in practical applications, this flexibility makes it difficult for a large-scale, general-purpose evaluation of multiple anomaly detectors, as you can optimize the metric for a specific application

See also

RangeBasedPrecision: Compute the range-based precision score.
RangeBasedFBeta: Compute the range-based \(F_\beta\) score.

Examples

>>> from dtaianomaly.evaluation import RangeBasedRecall
>>> metric = RangeBasedRecall()
>>> y_true = [0, 0, 0, 1, 1, 0, 0, 0]
>>> y_pred = [1, 0, 0, 1, 1, 1, 0, 0]
>>> metric.compute(y_true, y_pred)
1.0

compute(y_true: ndarray, y_pred: ndarray, **kwargs) → float

Compute the performance score.

Evaluate how closely the given anomaly scores align to the ground truth anomaly scores.

Parameters:

y_truearray-like of shape (n_samples): Ground-truth labels.
y_predarray-like of shape (n_samples): Predicted anomaly scores.
**kwargs: Additional arguments used for computing the evaluation metric.

Returns:

float: The alignment score of the given ground truth and prediction, according to this score.

Raises:

ValueError: When inputs are not numeric “array-like”s
ValueError: If shapes of y_true and y_pred are not of identical shape
ValueError: If y_true is non-binary.
ValueError: If y_pred is non-binary.