Lilly Tech Systems
Intermediate
Failure Prediction
Build machine learning models that predict device failures, link degradation, and service outages hours or days before they occur.
Problem Formulation
Failure prediction can be framed as several ML problems:
- Binary classification: Will this device fail within the next N hours? (yes/no)
- Time-to-event (survival analysis): How long until this device is likely to fail?
- Regression: What is the predicted remaining useful life (RUL) of this component?
- Anomaly detection: Is this device exhibiting pre-failure behavior patterns?
Feature Engineering
Key features for failure prediction models:
| Category | Features | Signal |
|---|---|---|
| Device age | Uptime, installation date, firmware version | Older devices fail more often |
| Error trends | CRC errors, memory errors, log error rate | Increasing errors precede failures |
| Environmental | Temperature, fan speed, power draw | Overheating causes component failure |
| Utilization | CPU, memory, bandwidth usage patterns | Sustained high utilization accelerates wear |
| Historical | Past failures, maintenance events, peer failures | Prior issues predict future problems |
Class imbalance: Failures are rare events compared to normal operation. Use techniques like SMOTE, class weighting, or focal loss to handle the severe class imbalance typical in failure prediction datasets.
Model Selection
- Random Forest / XGBoost: Excellent for tabular device telemetry data with strong interpretability
- LSTM networks: Capture temporal patterns in time-series metrics leading up to failures
- Survival models (Cox, DeepSurv): Naturally handle censored data (devices still running)
- Ensemble methods: Combine multiple models for robust predictions across failure types
Evaluation Metrics
- Precision: When we predict failure, how often are we right? (avoid unnecessary maintenance)
- Recall: Of all actual failures, how many did we predict? (avoid missed failures)
- Lead time: How far in advance can we predict failures? (enough time to act)
- False positive rate: How often do we cry wolf? (maintain operator trust)
Start here: Use XGBoost with a 24-hour prediction horizon. Engineer features from rolling windows of device metrics (mean, max, trend over last 1h, 6h, 24h). This simple approach often achieves 80%+ recall with acceptable false positive rates.