End-to-End ML Project Walkthrough
This walkthrough shows how the lessons in this track fit together using a customer churn prediction example.
1. Frame the problem
| Item | Decision |
|---|---|
| Business goal | Reduce customer churn with targeted offers |
| ML task | Binary classification: will the customer leave in 30 days? |
| Metric | Recall at a fixed precision (find churners without wasting offers); also PR-AUC |
| Constraint | Predictions needed weekly; explanations for the retention team |
2. Data and splitting
import pandas as pd
from sklearn.model_selection import train_test_split
df = pd.read_csv("customers.csv", parse_dates=["signup_date"])
X = df.drop(columns=["churned"])
y = df["churned"]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42
)
Split before exploring so the test set stays untouched. For time-dependent data split by time instead.
3. Baseline first
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score
base = DummyClassifier(strategy="prior").fit(X_train[["tenure"]], y_train)
print("dummy PR-AUC:", average_precision_score(y_test, base.predict_proba(X_test[["tenure"]])[:, 1]))
A model that does not beat a trivial baseline is not useful.
4. Preprocessing pipeline
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
num = ["tenure", "monthly_charges", "support_tickets"]
cat = ["plan", "payment_method"]
pre = ColumnTransformer([
("num", Pipeline([("imp", SimpleImputer(strategy="median")), ("sc", StandardScaler())]), num),
("cat", Pipeline([("imp", SimpleImputer(strategy="most_frequent")), ("oh", OneHotEncoder(handle_unknown="ignore"))]), cat),
])
5. Models and tuning
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import RandomizedSearchCV
model = Pipeline([("pre", pre), ("clf", HistGradientBoostingClassifier(random_state=0))])
search = RandomizedSearchCV(
model,
{"clf__learning_rate": [0.03, 0.1, 0.2], "clf__max_depth": [3, 6, None]},
n_iter=6, scoring="average_precision", cv=5, random_state=0,
)
search.fit(X_train, y_train)
print(search.best_params_, search.best_score_)
6. Final evaluation and errors
- Evaluate once on the test set; report PR-AUC, recall at the chosen threshold and the confusion matrix.
- Slice results by segment (plan type, region) to find weak spots.
- Inspect false negatives to find missing features.
7. Explain and deliver
| Deliverable | Content |
|---|---|
| Notebook / report | Problem, data, method, results, limitations |
| Model artefact | Saved pipeline with version |
| API or batch job | See ML in production |
| README | How to reproduce; data sources; how to run |
TIP
For your portfolio, publish the repository with a clear README, a results table, and a short write-up of what you tried that did not work. It shows real engineering judgement.
Loading Comments...