Lesson 60: sklearn Pipeline, ColumnTransformer & GridSearchCV

Discussion prompt

Before we open Lesson 60: sklearn Pipeline, ColumnTransformer & GridSearchCV: without looking back, what was the main idea of Hyperparameter Optimization, and what could you do by the end of it that you could not do before?

Hint: One sentence for the idea, one for the skill. If the second one is blank, that is the part to revisit.

Discussion prompt

Pipeline([('name', step), ...]) wraps a sequence of transformers ending in an estimator. Calling fit(X_train, y) on the pipeline fits every step on training data only.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Discussion prompt

Explain The assembly-line mental model by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Predict first

600 samples, 3 numeric features with 30 injected NaNs. Build an impute→scale pipeline and verify the scaler was fitted on training data only (Lesson 40 pattern).

Commit before you compute: what does Impute → Scale pipeline (numeric data) come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: imputer fill = 0.5336, scaler mean = 0.5336 — both computed from X_train only

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. The imputed column's mean IS the scaler's input mean.

Comparison matrix

From Impute → Scale pipeline (numeric data): refill the after impute column from what you know. The rest of the table is as it appeared.

Predict first

A student writes this, and it looks reasonable:

Scale everything, THEN split — feels fine because the scaler is 'just normalization'.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: The scaler's mean and std now encode information from the test rows.

Split first, then put the scaler INSIDE the pipeline.

Why: The scaler's mean and std now encode information from the test rows. The test set is no longer held-out — metrics are optimistically biased.

Discussion prompt

The rule this trap just fixed:

The pipeline calls scaler.fit only on X_train. X_test passes through transform (apply learned stats) — no leakage, honest metrics.

Now break it on purpose. Build a case that violates it and follow the consequences until something visibly fails. Where does the failure first show up — and would you have noticed it if you had not been looking?

Hint: The dangerous rules are the ones whose violation still produces an answer. If yours fails loudly, try to find one that fails quietly.

Comparison matrix

From ColumnTransformer: every row here is a choice with a cost. Fill the output cols column, then say which row you would actually pick and what you give up for it.

Step through it

Step through FunctionTransformer one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: input is 1.2000
2. Step 2: input is 0.5000
3. Step 3: input is 2.7000
4. Step 4: input is 0.1000
5. Step 5: input is 3.8000

Discussion prompt

Mixed dataset: 3 numeric columns (with NaNs) + 1 categorical column ('low'/'mid'/'high'). Chain ColumnTransformer into LogisticRegression.

What do you need to know — or decide — before the first line can be written? List everything the problem has to hand you.

Hint: Anything you would have to invent to get started is a thing the problem must supply.

Sort into buckets

Sort these by output shape, from memory, without looking back at Full Pipeline: impute→encode→scale→LogReg. Telling them apart on the spot is the skill; the table is only where the answer happens to be written down.

(480, 3)

numeric branch; cat branch

(480, 6)

hstack

predictions

LogReg

g1

output shape is "(480, 3)" for numeric branch, cat branch — that is what the table on "Full Pipeline: impute→encode→scale→LogRe…" records, and it is the single property separating this group from the rest.

g2

output shape is "(480, 6)" for hstack — that is what the table on "Full Pipeline: impute→encode→scale→LogRe…" records, and it is the single property separating this group from the rest.

g3

output shape is "predictions" for LogReg — that is what the table on "Full Pipeline: impute→encode→scale→LogRe…" records, and it is the single property separating this group from the rest.

Discussion prompt

Explain Tuning inside the pipeline to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Predict first

Tune clf__C (LogReg regularization) AND prep__num__imputer__strategy jointly over 5-fold CV.

Commit before you compute: what does GridSearchCV over Pipeline come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: best params: C=0.1, strategy='mean'; best CV=0.7083; test accuracy=0.7333

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. The grid explored 3×2=6 parameter combinations, each evaluated over 5 folds.

Step through it

Step through GridSearchCV over Pipeline one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: rank is 1
2. Step 2: rank is 1
3. Step 3: rank is 3

Predict first

A student writes this, and it looks reasonable:

Fit the scaler once on X_train, then pass the pre-scaled X_train to GridSearchCV — 'that's efficient: the scaler runs only once'.

It is wrong. Say what breaks — and say it before you turn the page.

Correct: Each CV fold's validation split was already transformed using all 480 training rows (including that fold's validation portion).

Pass the entire Pipeline to GridSearchCV so each fold re-fits preprocessing.

Why: Each CV fold's validation split was already transformed using all 480 training rows (including that fold's validation portion). The scaler statistics leak validation data into every fold — CV scores are optimistically biased.

Sort into buckets

Some of these hold up and some are the exact mistakes this lesson is built to prevent. Sort them.

Holds up

Pipeline([('name', step), ...]) wraps a sequence of transformers ending in an estimator. Calling fit(X_train, y) on the pipeline fits every step on training data only.; ColumnTransformer([(name, transformer, columns), ...]) applies a different sub-pipeline to each subset of columns and concatenates the results.; FunctionTransformer(func) wraps any numpy-compatible function as a sklearn transformer, so it plugs into a Pipeline or ColumnTransformer.

Breaks

Scale everything, THEN split — feels fine because the scaler is 'just normalization'.; Fit the scaler once on X_train, then pass the pre-scaled X_train to GridSearchCV — 'that's efficient: the scaler runs only once'.

sound

These are stated as this lesson states them — each one survives the edge cases Lesson 60: sklearn Pipeline, ColumnTransformer & GridSearchCV puts it through.

flawed

Each of these is lifted from a trap in this deck: reasonable-sounding, and wrong in a way that only shows up once you rely on it.

Discussion prompt

Explain Serializing a fitted pipeline to a student a year behind you. No notation, no jargon they have not met — and it still has to be true.

Hint: If your explanation needs a symbol they have never seen, you are describing the notation rather than the idea.

Discussion prompt

Explain Model versioning by analogy to something with no Machine Learning in it at all — a queue, a recipe, a map, a bank balance, whatever fits. Then say where your analogy breaks.

Hint: An analogy that never breaks is not an analogy, it is the same idea wearing a hat. Find the seam — that is the part that is actually new.

Predict first

Serialize clf_pipe, reload it, and confirm the reloaded version produces identical predictions.

Commit before you compute: what does dump, load, verify come out to? A rough magnitude and the right form is enough — the point is to have something concrete to be wrong about.

Correct: size=3354 bytes; original=0.7333; reloaded=0.7333; match=True

Why: A prediction you can defend turns the computation into a check rather than a leap of faith — and an answer that contradicts it is caught on the spot. joblib stores the entire pipeline object graph — all fitted parameters are preserved exactly.

Comparison matrix

From dump, load, verify: refill the reloaded column from what you know. The rest of the table is as it appeared.

Put in order

These are the steps of The production pipeline recipe, scrambled. Put them back in order before the next slide shows you.

1. Split first: train_test_split before any fitting
2. ColumnTransformer routes each column type to its own sub-pipeline
3. Pipeline chains ColumnTransformer → model; call fit(X_train) once
4. GridSearchCV(Pipeline, param_grid) tunes preprocessing + model in one search — preprocessing re-fits per fold
5. joblib.dump saves the fitted pipeline; include version + sklearn version in filename or metadata

Why: This is the order the recipe itself gives. Recalling the sequence without the slide in front of you is the difference between recognising the method and being able to run it — most of what goes wrong in practice is a step done out of turn.

Discussion prompt

The production pipeline recipe works on the cases you have just seen. Push it to the edge: what is the most degenerate input it still handles — empty, zero, one item, everything equal — and what is the first case where it stops being true? Name the case, not just "it breaks".

Hint: Try the smallest legal input, then the largest, then the one where two things collide. Methods are specified at their edges; the middle takes care of itself.

Eliminate the wrong options

You call scaler.fit_transform(X_all) on the full 600-row dataset, then do train_test_split. Why is the test accuracy reported by model.score(X_test_scaled, y_test) optimistically biased?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. The scaler's mean and std were computed from test rows, so X_test was already 'seen' during preprocessing
• B. StandardScaler changes the label distribution, inflating accuracy
• C. fit_transform returns a different dtype than transform, confusing the model
• D. The bias comes from using too many features, not the split order

Survives elimination: A

Why: The scaler fitted on X_all used 120 test-set rows to compute its mean/std. The 'test' rows were implicitly used in preprocessing, so their statistics leak into what looks like a held-out evaluation — the metric is an overly optimistic estimate of true generalization.

Check your understanding

You call scaler.fit_transform(X_all) on the full 600-row dataset, then do train_test_split. Why is the test accuracy reported by model.score(X_test_scaled, y_test) optimistically biased?

• A. The scaler's mean and std were computed from test rows, so X_test was already 'seen' during preprocessing (correct)
• B. StandardScaler changes the label distribution, inflating accuracy
• C. fit_transform returns a different dtype than transform, confusing the model
• D. The bias comes from using too many features, not the split order

Answer: A

Why: The scaler fitted on X_all used 120 test-set rows to compute its mean/std. The 'test' rows were implicitly used in preprocessing, so their statistics leak into what looks like a held-out evaluation — the metric is an overly optimistic estimate of true generalization.

Why B tempts people

StandardScaler is a feature transform; it never touches labels. Label distribution is unaffected.

Why C tempts people

Both fit_transform and transform return the same numpy float64 array — dtype is not the issue.

Why D tempts people

Leakage is about data flow across the train/test boundary, not feature count. Even a single-feature model leaks if the scaler was fitted on all data.

Predict first

In GridSearchCV(Pipeline([('prep', scaler), ('clf', model)]), param_grid, cv=5), what happens inside each of the 5 CV folds?

Answer it in your own words, now, with nothing to choose from. The options are on the next slide — and picking the right one off a list is an easier skill than producing it.

Correct: The scaler is re-fitted on the fold's train split; the fold's val split is only transformed

Why: GridSearchCV calls pipeline.fit(X_fold_train) for each fold, which triggers scaler.fit on that fold's training portion. The held-out val rows only pass through scaler.transform — exactly the leakage-free behavior you want.

Check your understanding

In GridSearchCV(Pipeline([('prep', scaler), ('clf', model)]), param_grid, cv=5), what happens inside each of the 5 CV folds?

• A. The scaler is re-fitted on the fold's train split; the fold's val split is only transformed (correct)
• B. The scaler is fitted once on all of X_train and then reused across all folds
• C. The scaler is not called inside GridSearchCV — it runs before the loop
• D. GridSearchCV removes preprocessing steps and only tunes the final estimator

Answer: A

Why: GridSearchCV calls pipeline.fit(X_fold_train) for each fold, which triggers scaler.fit on that fold's training portion. The held-out val rows only pass through scaler.transform — exactly the leakage-free behavior you want.

Why B tempts people

This is the leakage bug GridSearchCV+Pipeline avoids. Each fold gets its own scaler fit — not a shared one.

Why C tempts people

If preprocessing ran before the loop it would leak validation data into every fold's scaler statistics.

Why D tempts people

GridSearchCV tunes ALL named parameters in the pipeline, including preprocessing hyperparameters like strategy or n_components.

Eliminate the wrong options

After joblib.dump(clf_pipe, 'v1.joblib'), you load it on a different machine. Which statement is true?

3 of these 4 are wrong. Strike them one at a time, and say what rules each one out before you strike the next. The survivor is the answer.

• A. The reloaded pipeline contains all fitted parameters; calling predict() needs no X_train
• B. You must call fit() again after loading because joblib only stores the pipeline structure
• C. joblib stores only the model weights, not the scaler statistics — you must re-fit the scaler
• D. The loaded pipeline's accuracy will be higher because joblib optimizes the coefficients during serialization

Survives elimination: A

Why: joblib serializes the entire Python object graph — imputer statistics (fill values), scaler mean/std, and model coefficients. The reloaded pipeline calls predict() directly on new data with no re-fit. Our test confirmed accuracy 0.7333 before and after reload.

Check your understanding

After joblib.dump(clf_pipe, 'v1.joblib'), you load it on a different machine. Which statement is true?

• A. The reloaded pipeline contains all fitted parameters; calling predict() needs no X_train (correct)
• B. You must call fit() again after loading because joblib only stores the pipeline structure
• C. joblib stores only the model weights, not the scaler statistics — you must re-fit the scaler
• D. The loaded pipeline's accuracy will be higher because joblib optimizes the coefficients during serialization

Answer: A

Why: joblib serializes the entire Python object graph — imputer statistics (fill values), scaler mean/std, and model coefficients. The reloaded pipeline calls predict() directly on new data with no re-fit. Our test confirmed accuracy 0.7333 before and after reload.

Why B tempts people

This confuses joblib with saving only a model architecture (as in some deep learning frameworks that save state_dict separately from layer definitions). joblib always saves the full fitted object.

Why C tempts people

joblib serializes ALL attributes, including preprocessing statistics. The scaler's mean_ and scale_ arrays are part of the dumped object.

Why D tempts people

joblib is a serialization library, not an optimizer. It faithfully stores and restores the object state — no transformation of coefficients occurs.

Discussion prompt

Build a complete sklearn Pipeline for a mixed-type dataset, tune it with GridSearchCV, evaluate it on a held-out test set, and serialize the winner to disk.

That is stated as though it always holds. Do one of two things: produce a case where it fails, or say precisely what rules such a case out. "It just does" is not on the menu.

Hint: Hunt at the extremes first — zero, one, negative, empty, equal. If every extreme survives, the reason they survive is the proof.

Step through it

Step through Milestone 1 — ColumnTransformer pipeline one row at a time. What is driving the change, and what would the row after the last one be?

1. Step 1: branch is numeric
2. Step 2: branch is categorical
3. Step 3: branch is combined

Comparison matrix

From Milestone 3 — serialize and reload: every row here is a choice with a cost. Fill the accuracy column, then say which row you would actually pick and what you give up for it.

Comparison matrix

From The full program: refill the value column from what you know. The rest of the table is as it appeared.

Draw it

One page, no notation unless you need it: draw how these connect — Pipeline: chain without leaking · ColumnTransformer: mix-typed features · GridSearchCV over the full pipeline · joblib: save, load, version · Your turn: production pipeline. Put an arrow wherever one of them is what makes another possible, and label the arrow with why.