Event-wise metrics

Implementations of the event wise metrics proposed by [7]. These metrics will not look at the labeling of the individual points, but rather look at the event-level capabilities of the models.

class dtaianomaly.evaluation.EventWisePrecision[source]

Computes the Event-Wise Precision score [7].

Precision measures how accurately the model identifies anomalies. For the Event-Wise Precision, the true and false positives are considered at the event-level:

\(TP_e\): the number of ground truth anomalous events that fully or partially overlap with a detected segment.
\(FP_e\): the number of detected segments that do not overlap with any ground truth anomalous event.

The precision is corrected by the false-alarm rate (FAR) to avoid a model which predicts all observations as anomalous to have a high score. The FAR is computed on the point-level:

\(FP\): the number of detected anomalous points that are not actually anomalous.

We then compute the Event-Wise Precision as (with \(N\): the total number of normal points):

\[\text{Event-Wise Precision} = \frac{TP_e}{TP_e + FP_e} \times (1 - \frac{FP}{N})\]

class dtaianomaly.evaluation.EventWiseRecall[source]

Computes the Event-Wise Recall score [7].

Recall measures the model’s ability to correctly identify all actual anomalies. For the Event-Wise Recall, the true positives and false negatives are considered at the event-level:

\(TP_e\): the number of ground truth anomalous events that fully or partially overlap with a detected segment.
\(FN_e\): the number of ground truth anomalous events that do not overlap with a detected segment.

We then compute the Event-Wise Recall as:

\[\text{Event-Wise Recall} = \frac{TP_e}{TP_e + FN_e}\]

class dtaianomaly.evaluation.EventWiseFBeta(beta: float = 1.0)[source]

Computes the Event-Wise \(F_\beta\) score [7].

The \(F_\beta\) combines both precision and recall into a single value. It provides a balanced evaluation of a model’s performance, especially in anomaly detection, where there is often a trade-off between catching all anomalies (high recall) and minimizing false alarms (high precision). The parameter \(\beta\) controls the balance between precision and recall. A \(\beta > 1\) gives more weight to recall, useful when missing anomalies is costly, while \(\beta < 1\) emphasizes precision, reducing false positives.

The \(F_\beta\) score is the harmonic mean of the Event-Wise Precision and Event-Wise Recall.

Parameters:: beta (int, float, default=1) – Desired beta parameter.