RangeBasedFBeta

class dtaianomaly.evaluation.RangeBasedFBeta(beta: int | float = 1.0, alpha: float = 0.5, delta: Literal['flat', 'front', 'back', 'middle'] = 'flat', gamma: Literal['one', 'reciprocal'] = 'reciprocal')[source]

Computes the range-based \(F_\beta\) score [34].

The range-based \(F_\beta\)-score equals the harmonic mean of the range-based precision and range-based recall. The metrics take into account three parts: (1) the amount of overlap between the ground truth ranges and the predicted ranges, (2) whether there is fragmented detection or not, and (3) whether the ground truth ranges are detected at all.

Parameters:

betaint, float, default=1

Desired beta parameter.

alphafloat, default=0.5

The importance of detecting the events (even if it is only a single detected point) compared to detecting a large portion of the ground truth events. Should be at least 0 and at most 1.

deltastr, default=’flat’

Bias for the position of the predicted anomaly in the ground truth anomalous range. Valid options are:

'flat': Equal bias towards all positions in the ground truth anomalous range.
'front': Predictions that are near the front of the ground truth anomaly (i.e. early detection) have a higher weight.
'back': Predictions that are near the end of the ground truth anomaly (i.e. late detection) have a higher weight.
'middle': Predictions that are near the center of the ground truth anomaly have a higher weight.

gammastr, default=’reciprocal’

Penalization approach for detecting multiple ranges with a single range. Valid options are:

'one': Fragmented detection should not be penalized.
'reciprocal': Weight fragmented detection of :math:´N´ ranges with as single range by a factor of :math:´1/N´.

Warning

Note that, while tuning a metric to some domain is beneficial in practical applications, this flexibility makes it difficult for a large-scale, general-purpose evaluation of multiple anomaly detectors, as you can optimize the metric for a specific application.

See also

RangeBasedPrecision: Compute the range-based precision score.
RangeBasedRecall: Compute the range-based recall score.

Examples

>>> from dtaianomaly.evaluation import RangeBasedFBeta
>>> metric = RangeBasedFBeta()
>>> y_true = [0, 0, 0, 1, 1, 0, 0, 0]
>>> y_pred = [1, 0, 0, 1, 1, 1, 0, 0]
>>> metric.compute(y_true, y_pred)
0.5

compute(y_true: ndarray, y_pred: ndarray, **kwargs) → float

Compute the performance score.

Evaluate how closely the given anomaly scores align to the ground truth anomaly scores.

Parameters:

y_truearray-like of shape (n_samples): Ground-truth labels.
y_predarray-like of shape (n_samples): Predicted anomaly scores.
**kwargs: Additional arguments used for computing the evaluation metric.

Returns:

float: The alignment score of the given ground truth and prediction, according to this score.

Raises:

ValueError: When inputs are not numeric “array-like”s
ValueError: If shapes of y_true and y_pred are not of identical shape
ValueError: If y_true is non-binary.
ValueError: If y_pred is non-binary.