VolumeUnderPR

class dtaianomaly.evaluation.VolumeUnderPR(max_buffer_size: int = 500, compatibility_mode: bool = False, max_samples: int = 250)[source]

Computes the volume under the range-based precision-recall-curve [24].

Create a buffer around the anomalous event (similar as for RangeAreaUnderPR) for each buffer size in the range [0, max_buffer_size]. Then, max_samples thresholds are sampled uniformly from the anomaly scores to compute the new precision and recall for each buffer size. Also varying the buffer size results in a volume (instead of a curve), and the final evaluation score is computed as the volume under this surface.

Parameters:

max_buffer_sizeint, default=500: Maximum size of the buffer region around an anomaly. We iterate over all buffer sizes from 0 to may_buffer_size to create the surface.
compatibility_modebool, default=False: When set to True, produces exactly the same output as the metric implementation by the original authors. Otherwise, TimeEval uses a slightly improved implementation that fixes some bugs and uses linear slopes.
max_samplesint, default=250: Calculating precision and recall for many thresholds is quite slow. We, therefore, uniformly sample thresholds from the available score space. This parameter controls the maximum number of thresholds; too low numbers degrade the metrics’ quality.

Warning

Implementation of the Volume Under the Surface (VUS) metrics proposed by [24]. This implementation is adopted from [38], who slightly modified the original implementations:

For the recall (FPR) existence reward, anomalies are counted as separate events, even if the added slopes overlap;
Overlapping slopes don’t sum up in their anomaly weight, the anomaly weight for each point in the ground truth is maximized;
The original slopes are asymmetric: the slopes at the end of anomalies are a single point shorter than the ones at the beginning of anomalies. Symmetric slopes are used, with the same size for the beginning and end of anomalies;
A linear approximation of the slopes is used instead of the convex slope shape presented in the paper.

By default, the adjusted versions of each metric are used. To use the original implementations, you can set compatibility_mode=True when initializing the metrics.

In addition, we numbafied the most expensive part of the code (i.e., computing the recalls, precisions and false positive rates for every threshold), which leads to a more than 25x speedup on the demonstration time series.

See also

AreaUnderROC: Compute the area under the range-based ROC-curve.
RangeAreaUnderPR: Compute the area under the range-based PR-curve.
VolumeUnderROC: Compute the volume under the range-based ROC-surface.

Examples

>>> from dtaianomaly.evaluation import VolumeUnderPR
>>> metric = VolumeUnderPR()
>>> y_true = [0, 0, 0, 1, 1, 0, 0, 0]
>>> y_pred = [1, 0, 0, 1, 1, 1, 0, 0]
>>> metric.compute(y_true, y_pred)
0.994...

compute(y_true: ndarray, y_pred: ndarray, **kwargs) → float

Compute the performance score.

Evaluate how closely the given anomaly scores align to the ground truth anomaly scores.

Parameters:

y_truearray-like of shape (n_samples): Ground-truth labels.
y_predarray-like of shape (n_samples): Predicted anomaly scores.
**kwargs: Additional arguments used for computing the evaluation metric.

Returns:

float: The alignment score of the given ground truth and prediction, according to this score.

Raises:

ValueError: When inputs are not numeric “array-like”s
ValueError: If shapes of y_true and y_pred are not of identical shape
ValueError: If y_true is non-binary.