Back to all work 02 — Case study

Customer Retention Intelligence & Churn Prediction Platform

By the time a telecom customer cancels, the decision was made weeks earlier. I built a supervised ML platform over 250K+ customer records that surfaces the signals ahead of the cancellation — turning retention from a reaction into a plan.

Company

Colt Technology Services

Role

Junior Machine Learning Engineer

Timeline

May 2021 — Jun 2023

Domain

Customer analytics

The problem

Retention was reactive, and the data was scattered.

The signals that predict churn — declining usage, billing friction, a rising count of support contacts — were all being captured. They just lived in separate enterprise systems that never spoke to each other.

Without a unified view, retention teams learned a customer was leaving when the cancellation arrived. There was no ranked list of who was at risk and no evidence base for deciding where to spend retention effort.

The task was to join those systems into a single modeling substrate and produce something an account team could act on: a reliable, ranked view of at-risk customers.

The approach

Fix the data before touching the model.

Most of the accuracy gain came from the pipeline, not the algorithm. Cleaning and unification did more work than any model swap.

Pipeline diagram: usage, billing and support data flow through ETL unification into cleaning and feature engineering, then model training, evaluation by precision, recall and F1, and real-time scoring served via FastAPI. — Three source systems collapse into one ML-ready view before any model sees the data.

Collection & cleaning

250K+ customer records collected, cleaned, and transformed with Pandas and NumPy — raising data quality and cutting preprocessing time 30%.

ETL unification

Improved ETL pipelines to integrate usage, billing, and support data from multiple enterprise systems into a single ML-ready view.

Feature engineering

Behavioural and billing features derived from the joined data, capturing trajectory over time rather than point-in-time snapshots.

Model training

Supervised classification models built with Scikit-learn and TensorFlow, improving churn prediction accuracy 15% over the prior baseline.

Evaluation

Cross-validation, confusion matrices, and precision / recall / F1 analysis — because on imbalanced churn data, raw accuracy hides the failure mode that matters.

Serving

Trained models deployed as FastAPI REST APIs on AWS, cutting inference response time 20% and enabling real-time churn scoring.

Evaluation

Accuracy is the wrong metric for churn.

Churn datasets are imbalanced — most customers stay. A model that predicts "no churn" for everyone scores well on accuracy and is completely useless.

Model selection was driven by precision, recall, and F1 read together, with confusion matrices making the trade-off explicit: every gain in recall catches more real churners but spends retention budget on customers who were never going to leave. Cross-validation kept those estimates honest across data splits rather than fitting a favourable one.

Stack

Modeling

Scikit-learn
TensorFlow
Feature engineering
Cross-validation

Data

Pandas
NumPy
ETL pipelines
SQL

Serving

FastAPI
REST APIs
AWS
Linux

Process

Agile / Scrum
Model monitoring
Git

Outcome

A ranked list retention teams could actually work.

Prediction accuracy improved 15%, preprocessing time dropped 30%, and real-time scoring through the FastAPI service returned results 20% faster — fast enough to surface risk inside existing workflows instead of a monthly batch report.

I stayed with the system after launch, working with data engineering and business teams to monitor production performance, maintain the Linux-based ML environments, and keep improving the feature pipelines across Agile cycles.