Your First End-to-End Machine Learning Project with scikit-learn

Knowing individual algorithms is like knowing individual cricket shots. A project is playing a full innings. This lesson walks through a complete project in the order professionals follow — and in the order you should explain your own projects in interviews.

The problem

A telecom company wants to predict which customers are likely to leave (churn) in the next month, so the retention team can offer them a better plan. Our data has one row per customer: tenure in months, monthly charges, contract type, internet service, payment method, number of support calls — and whether they churned.

Step 1 — Frame it

• Task: binary classification (churn yes/no).
• Decision it supports: who gets a retention call this month — the team can call about 500 people.
• Metric: churners are a minority, and a missed churner costs more than an extra call, so we focus on recall and precision among the top-ranked customers, with PR-AUC to compare models.
• Baseline: "call the customers with the shortest tenure" — our model must beat that rule.

Step 2 — Explore before modelling

import pandas as pd

df = pd.read_csv("telecom_churn.csv")
print(df.shape, df["churn"].mean().round(3))            # how many rows, what share churned
print(df.isna().sum())
print(df.groupby("contract")["churn"].mean())            # month-to-month churns far more?
print(df.groupby(pd.cut(df["tenure"], [0, 6, 12, 24, 72]))["churn"].mean())

Exploration gives you hypotheses ("short tenure and month-to-month contracts churn most") and catches problems (missing values, impossible numbers) before they poison the model.

Step 3 — A pipeline and a fair comparison

from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier

numeric     = ["tenure", "monthly_charges", "support_calls"]
categorical = ["contract", "internet_service", "payment_method"]
X, y = df[numeric + categorical], df["churn"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

prepare = ColumnTransformer([
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]), numeric),
    ("cat", Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
                      ("onehot", OneHotEncoder(handle_unknown="ignore"))]), categorical),
])
candidates = {
    "logistic": LogisticRegression(max_iter=1000, class_weight="balanced"),
    "forest":   RandomForestClassifier(n_estimators=300, class_weight="balanced", random_state=42),
}
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
for name, clf in candidates.items():
    pipe = Pipeline([("prepare", prepare), ("model", clf)])
    scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="average_precision")
    print(f"{name:9s} PR-AUC {scores.mean():.3f} ± {scores.std():.3f}")

Step 4 — Tune the better model

from sklearn.model_selection import GridSearchCV

pipe = Pipeline([("prepare", prepare),
                 ("model", RandomForestClassifier(class_weight="balanced", random_state=42))])
grid = GridSearchCV(pipe,
                    param_grid={"model__n_estimators": [200, 400],
                                "model__max_depth": [None, 8, 12],
                                "model__min_samples_leaf": [1, 5]},
                    scoring="average_precision", cv=cv, n_jobs=-1)
grid.fit(X_train, y_train)
print(grid.best_params_, round(grid.best_score_, 3))
best = grid.best_estimator_

The model__ prefix tells the grid search which pipeline step a parameter belongs to. Because preparation is inside the pipeline, every fold is prepared using only its own training part — no leakage.

Step 5 — Evaluate once on the test set

from sklearn.metrics import average_precision_score, classification_report

proba = best.predict_proba(X_test)[:, 1]
print("Test PR-AUC:", round(average_precision_score(y_test, proba), 3))

top = pd.DataFrame({"churn": y_test.values, "score": proba}).sort_values("score", ascending=False)
k = max(1, int(len(top) * 0.10))                         # the 10% of customers the team can call
print(f"Churners caught in top 10%: {top['churn'].head(k).sum()} of {top['churn'].sum()}")

That last line speaks the business language: "If the team calls the top 10% our model ranks, they reach X% of all churners — compared with Y% using the tenure rule."

Step 6 — Save and use the model

import joblib

joblib.dump(best, "churn_model.joblib")                  # the whole pipeline: preparation + model

model = joblib.load("churn_model.joblib")
new_customer = pd.DataFrame([{"tenure": 3, "monthly_charges": 799, "support_calls": 4,
                              "contract": "Month-to-month", "internet_service": "Fiber",
                              "payment_method": "UPI"}])
print("Churn risk:", round(model.predict_proba(new_customer)[0, 1], 2))

Saving the whole pipeline means new raw data goes in exactly as the original did. To serve predictions to an app, wrap this in a small API (FastAPI or Flask) — a natural next step.

Step 7 — After launch

• Monitor input data and prediction rates; customer behaviour changes (data drift), and models quietly get worse.
• Measure the outcome: did retention calls to high-risk customers actually reduce churn?
• Retrain on fresh data on a schedule.