Robust Principal Component Analysis

class dtaianomaly.anomaly_detection.RobustPrincipalComponentAnalysis(window_size: str | int, stride: int = 1, max_iter: int = 1000, **kwargs)

Anomaly detection based on Robust Principal Component Analysis (Robust PCA).

Assume that the data matrix is a superposition of a low-rank component and a s parse component. Robust PCA [Candes2011robust] will solve this decomposition as a convex optimization problem. The superposition offers a principeled manner to robust PCA, since the methodology can recover the principal components (first component) of a data matrix even though a positive fraction of the entries are arbitrarly corrupted or anomalous (second component).

Warning

During testing, we found that there are some deviations in the predicted decision scores, depending on if the method was run on windows or linux. The difference in the absolute value is of around the order of 2%, but the general trend of the anomaly scores remains consistent. The only randomness in this implementation of Robust PCA is the PCA solver of scikit-learn, but even setting a random state did not resolve the issue.

Notes

In most existing implementations, Robust PCA only takes one observation at a time into account (i.e., does not look at windows). However, Robust PCA can not be applied to a single variable, which is the case for univariate data. Therefore, we added a parameter window_size to apply Robust PCA in windows of a univariate time series, to make it applicable. Common behavior on multivariate time series can be obtained by setting window_size = 1.

Parameters:

• window_size (int or str) – The window size to use for extracting sliding windows from the time series. This value will be passed to compute_window_size().

• stride (int, default=1) – The stride, i.e., the step size for extracting sliding windows from the time series.

• max_iter (int, default=1000) – The maximum number of iterations allowed to optimize the low rank approximation.

• kwargs (dict) – Additional parameters to be passed PCA of Sklearn.

window_size_

The effectively used window size for this anomaly detector

Type:

int

pca_

The PCA-object used to project the data in a lower dimension.

Type:

PCA

Examples

>>> from dtaianomaly.anomaly_detection import RobustPrincipalComponentAnalysis
>>> from dtaianomaly.data import demonstration_time_series
>>> x, y = demonstration_time_series()
>>> rpca = RobustPrincipalComponentAnalysis(2).fit(x)
>>> rpca.decision_function(x)
array([1.28436687, 1.29156655, 1.33793287, ..., 1.35563558, 1.25948662,
       1.2923824 ])

References

[Candes2011robust]

Emmanuel J. Candès, Xiaodong Li, Yi Ma, and John Wright. 2011. Robust principal component analysis? J. ACM 58, 3, Article 11 (June 2011), 37 pages. doi: 10.1145/1970392.1970395

decision_function(X: ndarray) → ndarray

Abstract method, compute anomaly scores.

Parameters:

X (array-like of shape (n_samples, n_attributes)) – Input time series.

Returns:

decision_scores – The computed anomaly scores.

Return type:

array-like of shape (n_samples)

fit(X: ndarray, y: ndarray | None = None, **kwargs) → RobustPrincipalComponentAnalysis

Fit this Robust PCA to the given data

Parameters:

• X (array-like of shape (n_samples, n_attributes)) – Input time series.

• y (ignored) – Not used, present for API consistency by convention.

• kwargs – Additional parameters to be passed to compute_window_size().

Returns:

self – Returns the instance itself

Return type:

RobustPrincipleComponentAnalysis

Raises:

ValueError – If X is not a valid array.

predict_proba(X: ndarray) → ndarray

Predict anomaly probabilities

Estimate the probability of a sample of X being anomalous, based on the anomaly scores obtained from decision_function by rescaling them to the range of [0, 1] via min-max scaling.

Parameters:

X (array-like of shape (n_samples, n_attributes)) – Input time series.

Returns:

anomaly_scores – 1D array with the same length as X, with values in the interval [0, 1], in which a higher value implies that the instance is more likely to be anomalous.

Return type:

array-like of shape (n_samples)

Raises:

• ValueError – If scores is not a valid array.

• ValueError – If the prediction scores from ‘decision_function’ are constant, but not in the interval [0, 1], because these values can not unambiguously be transformed to an anomaly probability.

save(path: str | Path) → None

Save detector to disk as a pickle file with extension .dtai. If the given path consists of multiple subdirectories, then the not existing subdirectories are created.

Parameters:

path (str or Path) – Location where to store the detector.