Anomaly Detection
Single out the odd few among masses of normal data
WHAT THIS CAPABILITY MEANS
Takes data records or time series and outputs how far each deviates from the norm, or a flag. It usually works with few or no labelled anomalies, modelling what normal looks like and calling deviations anomalous. Unlike text classification the anomaly classes are not fixed, and in training you often do not know what an anomaly looks like at all.
How it is done
Unsupervised routes model normal data: autoencoders flag large reconstruction errors, isolation forests treat easily isolated points as anomalous, and statistical methods fit a distribution and score low-probability points. Time series often use a predictive framing, taking the residual between actual and forecast values as the score. With a few labels, semi-supervised or contrastive learning takes over, and cost-sensitive thresholds balance misses against false alarms.
Representative products
3Organizations involved
Typical uses
- Early warning for equipment and production-line faults
- Spotting oddity in transactions and money laundering
- Intrusion and abnormal-traffic monitoring
- Quality sampling and data-entry error catching
How it is evaluated
- AUROC
- Threshold-free ranking quality, usable on heavily imbalanced data
- PR-AUC
- More informative than AUROC when anomalies are extremely rare
- Detection delay
- Time from onset to alert, key in real-time settings
Limits and hard parts
- With so few anomalies a small threshold change spikes false alarms, and operators quickly go numb to them
- Concept drift marks normal behaviour as anomalous, and seasonality or business change demands continual recalibration
- In high dimensions with correlated features distance-based measures break down, diluting the gaps between normal points
Concepts behind it
Unsupervised Learning
With no answers given, structure must emerge from the data itself — and “good” has to be redefined
Probability & Distributions
A model never hands you an answer; it hands you a degree of belief over every possible answer
Model Evaluation & Cross-Validation
Accuracy is the easiest metric to fool you — get evaluation wrong and everything else follows