Evaluation module

This module contains functionality to evaluate performance of an anomaly detector. It can be imported as follows:

>>> from dtaianomaly import evaluation

Custom evaluation metrics can be implemented by extending BinaryMetric or ProbaMetric. The former expects predicted “decisions” (anomaly or not), the latter predicted “scores” (more or less anomalous). This distinction is important for later use in a Worfklow.

Base Objects

Metric()	Base class for metrics.
BinaryMetric()	A metric that takes as input binary anomaly labels.
ProbaMetric()	Base class for probabilistic metrics.
FBetaMixin([beta])	Base class for F-Beta metrics.

Affiliation-based

AffiliationPrecision()	Compute the affiliation-based precision score [17].
AffiliationRecall()	Compute the affiliation-based recall score [17].
AffiliationFBeta([beta])	Compute the affiliation-based \(F_\beta\) score [17].

Area Under the Curve (AUC)

AreaUnderPR()	Computes the Area Under the Precision-Recall Curve (AUC-PR) score.
AreaUnderROC()	Compute the Area Under the Receiver Operating Characteristic Curve (AUC-ROC) score.

Classification-based

Precision()	Compute the Precision score.
Recall()	Computes the Recall score.
FBeta([beta])	Computes the \(F_\beta\) score.

Compound metrics

ThresholdMetric(thresholder, metric)	Apply a Thresholding and BinaryMetric in sequence.
BestThresholdMetric(metric[, max_nb_thresholds])	Compute the maximum score across all thresholds.

Event-wise

EventWisePrecision()	Compute the Event-Wise Precision score [9].
EventWiseRecall()	Compute the Event-Wise Recall score [9].
EventWiseFBeta([beta])	Compute the Event-Wise \(F_\beta\) score [9].

Point-adjusted

PointAdjustedPrecision()	Compute the point-adjusted precision.
PointAdjustedRecall()	Compute the point-adjusted recall.
PointAdjustedFBeta([beta])	Compute the point-adjusted \(F_\beta\).

Range-based

RangeBasedPrecision([delta, gamma])	Computes the range-based precision score [34].
RangeBasedRecall([alpha, delta, gamma])	Computes the range-based recall score [34].
RangeBasedFBeta([beta, alpha, delta, gamma])	Computes the range-based \(F_\beta\) score [34].

UCR Score

UCRScore([tolerance])

Compute the UCR-Score to evaluate the predicted anomaly scores [39].

Volume Under the Surface (VUS)

RangeAreaUnderPR([buffer_size, ...])	Computes the area under the range-based PR-curve [24].
RangeAreaUnderROC([buffer_size, ...])	Computes the area under the range-based ROC-curve [24].
VolumeUnderPR([max_buffer_size, ...])	Computes the volume under the range-based precision-recall-curve [24].
VolumeUnderROC([max_buffer_size, ...])	Computes the volume under the range-based ROC-curve [24].