RangeAreaUnderROC

class dtaianomaly.evaluation.RangeAreaUnderROC(buffer_size: int = None, compatibility_mode: bool = False, max_samples: int = 250)[source]

Computes the area under the range-based ROC-curve [24].

A slope of length buffer_size // 2 is added at the beginning and end of each anomalous event. Next, the false positive rate and true positive rate is computed, taking into account the slopes in ground truth labels to allow for some small misalignment in the predicted and actual anomalous events. Then, max_samples thresholds are sampled uniformly from the anomaly scores to compute the new FPR and TPR, after which the area under the curve can be computed as final evaluation score.

Parameters:

buffer_sizeint, default=None: Size of the buffer region around an anomaly. We add an increasing slope of size buffer_size//2 to the beginning of anomalies and a decreasing slope of size buffer_size//2 to the end of anomalies. Per default (when buffer_size==None), buffer_size is the median length of the anomalies within the time series. However, you can also set it to the period size of the dominant frequency or any other desired value.
compatibility_modebool, default=False: When set to True, produces exactly the same output as the metric implementation by the original authors. Otherwise, TimeEval uses a slightly improved implementation that fixes some bugs and uses linear slopes.
max_samplesint, default= 250: Calculating precision and recall for many thresholds is quite slow. We, therefore, uniformly sample thresholds from the available score space. This parameter controls the maximum number of thresholds; too low numbers degrade the metrics’ quality.

Warning

Implementation of the Volume Under the Surface (VUS) metrics proposed by [24]. This implementation is adopted from [38], who slightly modified the original implementations:

For the recall (FPR) existence reward, anomalies are counted as separate events, even if the added slopes overlap;
Overlapping slopes don’t sum up in their anomaly weight, the anomaly weight for each point in the ground truth is maximized;
The original slopes are asymmetric: the slopes at the end of anomalies are a single point shorter than the ones at the beginning of anomalies. Symmetric slopes are used, with the same size for the beginning and end of anomalies;
A linear approximation of the slopes is used instead of the convex slope shape presented in the paper.

By default, the adjusted versions of each metric are used. To use the original implementations, you can set compatibility_mode=True when initializing the metrics.

In addition, we numbafied the most expensive part of the code (i.e., computing the recalls, precisions and false positive rates for every threshold), which leads to a more than 25x speedup on the demonstration time series.

See also

RangeAreaUnderPR: Compute the area under the range-based PR-curve.
VolumeUnderROC: Compute the volume under the range-based ROC-surface.
VolumeUnderPR: Compute the volume under the range-based PR-surface.

Examples

>>> from dtaianomaly.evaluation import RangeAreaUnderROC
>>> metric = RangeAreaUnderROC()
>>> y_true = [0, 0, 0, 1, 1, 0, 0, 0]
>>> y_pred = [1, 0, 0, 1, 1, 1, 0, 0]
>>> metric.compute(y_true, y_pred)
0.877...

compute(y_true: ndarray, y_pred: ndarray, **kwargs) → float

Compute the performance score.

Evaluate how closely the given anomaly scores align to the ground truth anomaly scores.

Parameters:

y_truearray-like of shape (n_samples): Ground-truth labels.
y_predarray-like of shape (n_samples): Predicted anomaly scores.
**kwargs: Additional arguments used for computing the evaluation metric.

Returns:

float: The alignment score of the given ground truth and prediction, according to this score.

Raises:

ValueError: When inputs are not numeric “array-like”s
ValueError: If shapes of y_true and y_pred are not of identical shape
ValueError: If y_true is non-binary.