RangeBasedPrecision

class dtaianomaly.evaluation.RangeBasedPrecision(delta: Literal['flat', 'front', 'back', 'middle'] = 'flat', gamma: Literal['one', 'reciprocal'] = 'reciprocal')[source]

Computes the range-based precision score [34].

The range-based precision computes a precision-score for each predicted anomalous range and then takes the average over all ranges. This precision-score consists of two parts: (1) the amount of overlap between the predicted range and the ground truth ranges, and (2) whether the predicted range overlaps with only one or multiple ground truth ranges. These components can be computed independently, and are multiplied to get a final precision-score for the range.

Parameters:

deltastr, default=’flat’

Bias for the position of the predicted anomaly in the ground truth anomalous range. Valid options are:

'flat': Equal bias towards all positions in the ground truth anomalous range.
'front': Predictions that are near the front of the ground truth anomaly (i.e. early detection) have a higher weight.
'back': Predictions that are near the end of the ground truth anomaly (i.e. late detection) have a higher weight.
'middle': Predictions that are near the center of the ground truth anomaly have a higher weight.

gammastr, default=’reciprocal’

Penalization approach for detecting multiple ranges with a single range. Valid options are:

'one': Fragmented detection should not be penalized.
'reciprocal': Weight fragmented detection of :math:´N´ ranges with as single range by a factor of :math:´1/N´.

Warning

Note that, while tuning a metric to some domain is beneficial in practical applications, this flexibility makes it difficult for a large-scale, general-purpose evaluation of multiple anomaly detectors, as you can optimize the metric for a specific application.

See also

RangeBasedPrecision: Compute the range-based precision score.
RangeBasedRecall: Compute the range-based recall score.

Examples

>>> from dtaianomaly.evaluation import RangeBasedPrecision
>>> metric = RangeBasedPrecision()
>>> y_true = [0, 0, 0, 1, 1, 0, 0, 0]
>>> y_pred = [1, 0, 0, 1, 1, 1, 0, 0]
>>> metric.compute(y_true, y_pred)
0.333...

compute(y_true: ndarray, y_pred: ndarray, **kwargs) → float

Compute the performance score.

Evaluate how closely the given anomaly scores align to the ground truth anomaly scores.

Parameters:

y_truearray-like of shape (n_samples): Ground-truth labels.
y_predarray-like of shape (n_samples): Predicted anomaly scores.
**kwargs: Additional arguments used for computing the evaluation metric.

Returns:

float: The alignment score of the given ground truth and prediction, according to this score.

Raises:

ValueError: When inputs are not numeric “array-like”s
ValueError: If shapes of y_true and y_pred are not of identical shape
ValueError: If y_true is non-binary.
ValueError: If y_pred is non-binary.