Lilly Tech Systems
Build a Real-Time Fraud Detector
Go from raw credit card transaction data to a production-grade, real-time fraud detection system. Train XGBoost and LightGBM models, serve predictions via FastAPI under 50ms, stream events through Kafka, explain decisions with SHAP, and monitor for drift — all in one end-to-end project.
Project Build Steps
Follow these lessons in order to build the complete fraud detection system from scratch, with working code at every step.
1. Project Setup
System architecture overview, credit card fraud dataset introduction, tech stack walkthrough (Python, XGBoost, FastAPI, Kafka), and environment setup with all dependencies.
2. Data Exploration & Feature Engineering
Exploratory data analysis on imbalanced transaction data, SMOTE oversampling, feature creation for amount patterns, velocity checks, and time-based aggregations.
3. Model Training
Train XGBoost and LightGBM classifiers, tune decision thresholds for fraud recall, implement stratified cross-validation, and compare model performance.
4. Evaluation & Explainability
Precision-recall tradeoff analysis, SHAP feature explanations, false positive deep-dive, confusion matrix visualization, and business-metric alignment.
5. Real-Time Inference API
Build a FastAPI prediction endpoint, compute features on the fly, achieve sub-50ms latency, add input validation with Pydantic, and load-test with Locust.
6. Kafka Streaming Pipeline
Ingest transaction events via Kafka, score them in real time, route alerts to downstream consumers, and handle backpressure and exactly-once semantics.
7. Monitoring & Retraining
Detect data drift with Evidently, track model performance over time, set up automated retraining triggers, and build a Grafana monitoring dashboard.
8. Enhancements & Next Steps
Add human-in-the-loop review, integrate feedback loops for continuous learning, graph-based fraud detection, and frequently asked questions.
What You Will Build
By the end of this project, you will have a complete, deployable fraud detection system with these capabilities:
ML Models That Catch Fraud
XGBoost and LightGBM classifiers trained on real-world credit card data, tuned for high recall with controlled false positive rates.
Sub-50ms Predictions
A FastAPI inference service that computes features and returns fraud scores in under 50 milliseconds, ready for production traffic.
Real-Time Streaming
Kafka-based event pipeline that ingests transactions, scores them instantly, and routes alerts to investigation queues.
Monitoring & Auto-Retrain
Drift detection, performance dashboards, and automated retraining triggers that keep the system accurate as fraud patterns evolve.