BestThresholdMetric

class dtaianomaly.evaluation.BestThresholdMetric(metric: BinaryMetric, max_nb_thresholds: int = None)[source]

Compute the maximum score across all thresholds.

Compute the maximum score of a BinaryMetric over all thresholds. This method will iterate over the possible thresholds for given predicted anomaly scores, compute the BinaryMetric for each threshold, and then return the score for the highest threshold.

Parameters:

metricBinaryMetric: Instance of the desired BinaryMetric class.
max_nb_thresholdsint, default=None: The maximum number of thresholds to use for computing the best threshold. If max_nb_thresholds = None, all thresholds will be used. Otherwise, the value indicates the subsample of all possible thresholds that should be used. This subset is created by first sorting the possible unique thresholds, and then selecting the threshold at regular intervals (i.e., the 3rd, 6th, 9th, …). We recommend using the default value (use all thresholds), but can be used for reducing the resource requirements.

Attributes:

threshold_float: The threshold resulting in the best performance.
thresholds_array-like of floats: The thresholds used for evaluating the performance.
scores_array-like of floats: The evaluation scores corresponding to each threshold in thresholds_.

Examples

>>> from dtaianomaly.evaluation import BestThresholdMetric, Precision
>>> metric = BestThresholdMetric(Precision())
>>> y_true = [   0,   0,   0,   1,   1,   0,   0,   0]
>>> y_pred = [0.95, 0.5, 0.4, 0.8, 1.0, 0.7, 0.2, 0.1]
>>> metric.compute(y_true, y_pred)
1.0

compute(y_true: ndarray, y_pred: ndarray, **kwargs) → float

Compute the performance score.

Evaluate how closely the given anomaly scores align to the ground truth anomaly scores.

Parameters:

y_truearray-like of shape (n_samples): Ground-truth labels.
y_predarray-like of shape (n_samples): Predicted anomaly scores.
**kwargs: Additional arguments used for computing the evaluation metric.

Returns:

float: The alignment score of the given ground truth and prediction, according to this score.

Raises:

ValueError: When inputs are not numeric “array-like”s
ValueError: If shapes of y_true and y_pred are not of identical shape
ValueError: If y_true is non-binary.