A reliable machine-learning workflow must apply preprocessing in exactly the same way during training, validation, testing, and prediction. Scikit-learn's Pipeline and ColumnTransformer make that workflow repeatable while protecting cross-validation from data leakage.
Short answer: put every learned preprocessing step—imputation, scaling, encoding, feature selection—inside the pipeline. Then pass the entire pipeline to cross-validation or hyperparameter search. Do not fit preprocessors on the full dataset before splitting or cross-validation.
Why use a scikit-learn Pipeline?
A pipeline connects transformations and a final estimator into one object. Calling fit learns each transformation from the training data and passes the transformed result to the next step. Calling predict reuses those learned transformations before generating predictions.
This design provides four practical benefits:
- Leakage prevention: preprocessing is learned only from the training portion of each fold.
- Repeatability: the same transformation sequence is used everywhere.
- Joint tuning: preprocessing and model parameters can be searched together.
- Simpler deployment: one fitted object contains the entire prediction workflow.
The leakage problem
Suppose missing values are imputed using medians calculated from the complete dataset, and only then cross-validation is run. Information from validation folds has influenced the medians. The effect may look small, but the validation estimate is no longer clean.
The same problem occurs with standardization, category encoding based on observed data, feature selection, PCA, and target-informed transformations. A pipeline fits these steps separately inside every training fold.
Pipeline, ColumnTransformer, and passthrough
| Tool | Purpose | Typical use |
|---|---|---|
Pipeline | Runs ordered transformations followed by a final estimator. | Imputer → scaler → classifier |
ColumnTransformer | Applies different transformations to different columns. | Scale numeric columns and one-hot encode categorical columns |
passthrough | Keeps a step or remaining columns unchanged. | Compare scaled and original features during tuning |
drop | Removes selected or remaining columns. | Exclude IDs or leakage-prone variables |
Complete mixed-data example
This example loads the Titanic dataset from OpenML, selects numeric and categorical features, creates separate preprocessing pipelines, and evaluates logistic regression. The final test set is kept untouched until the end.
After generating final predictions, inspect individual error types with the Confusion Matrix in Python guide, then compare the broader trade-offs in Accuracy vs Precision vs Recall vs F1 vs ROC AUC.
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.datasets import fetch_openml
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, roc_auc_score
from sklearn.model_selection import (
StratifiedKFold,
cross_validate,
train_test_split,
)
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
# Load data as a pandas DataFrame
titanic = fetch_openml("titanic", version=1, as_frame=True)
features = ["age", "fare", "embarked", "sex", "pclass"]
X = titanic.data[features].copy()
y = titanic.target.astype(int)
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.20,
stratify=y,
random_state=42,
)
numeric_features = ["age", "fare"]
categorical_features = ["embarked", "sex", "pclass"]
numeric_pipeline = Pipeline(
steps=[
("imputer", SimpleImputer(strategy="median")),
("scaler", StandardScaler()),
]
)
categorical_pipeline = Pipeline(
steps=[
("imputer", SimpleImputer(strategy="most_frequent")),
(
"encoder",
OneHotEncoder(handle_unknown="ignore"),
),
]
)
preprocessor = ColumnTransformer(
transformers=[
("numeric", numeric_pipeline, numeric_features),
("categorical", categorical_pipeline, categorical_features),
],
remainder="drop",
)
model = Pipeline(
steps=[
("preprocessor", preprocessor),
(
"classifier",
LogisticRegression(max_iter=2000, random_state=42),
),
]
)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_validate(
model,
X_train,
y_train,
cv=cv,
scoring={
"roc_auc": "roc_auc",
"accuracy": "accuracy",
"f1": "f1",
},
n_jobs=-1,
)
summary = pd.DataFrame(scores).agg(["mean", "std"]).T
print(summary.round(4))
model.fit(X_train, y_train)
test_probability = model.predict_proba(X_test)[:, 1]
test_prediction = model.predict(X_test)
print("Test ROC AUC:", roc_auc_score(y_test, test_probability))
print(classification_report(y_test, test_prediction))How the code works
1. Split before fitting
The held-out test set is created before any imputer, scaler, encoder, or model is fitted. Stratification preserves the class ratio in both subsets.
2. Separate numeric and categorical processing
Numeric values receive median imputation and standardization. Categorical values receive most-frequent imputation and one-hot encoding. handle_unknown="ignore" prevents prediction from failing when a new category appears outside the training data.
3. Evaluate the whole workflow
cross_validate receives the complete pipeline, not a preprocessed matrix. Every fold learns its own imputation values, scaling statistics, category vocabulary, and classifier coefficients.
4. Fit once for final testing
After model selection is complete, the pipeline is fitted on all training data and evaluated once on the untouched test set.
Tune preprocessing and the model together
Pipeline parameters use the syntax step__parameter. Nested parameters extend the pattern. For example, preprocessor__numeric__imputer__strategy reaches the imputer inside the numeric pipeline.
from sklearn.model_selection import GridSearchCV
parameter_grid = {
"preprocessor__numeric__imputer__strategy": [
"mean",
"median",
],
"preprocessor__numeric__scaler": [
StandardScaler(),
"passthrough",
],
"classifier__C": [0.01, 0.1, 1.0, 10.0],
"classifier__class_weight": [None, "balanced"],
}
search = GridSearchCV(
estimator=model,
param_grid=parameter_grid,
scoring="roc_auc",
cv=cv,
n_jobs=-1,
refit=True,
)
search.fit(X_train, y_train)
print("Best CV ROC AUC:", search.best_score_)
print("Best parameters:", search.best_params_)
best_model = search.best_estimator_
test_probability = best_model.predict_proba(X_test)[:, 1]
print("Final test ROC AUC:", roc_auc_score(y_test, test_probability))Using "passthrough" lets the search compare standardized and original numeric features. The same mechanism can temporarily remove optional steps or replace one estimator with another.
Regression pipeline example
The architecture is identical for regression. Change the estimator, cross-validation splitter when necessary, and scoring metrics.
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import KFold, cross_validate
from sklearn.pipeline import Pipeline
regression_model = Pipeline(
steps=[
("preprocessor", preprocessor),
(
"regressor",
RandomForestRegressor(
n_estimators=300,
min_samples_leaf=2,
n_jobs=-1,
random_state=42,
),
),
]
)
regression_cv = KFold(n_splits=5, shuffle=True, random_state=42)
regression_scores = cross_validate(
regression_model,
X_train,
y_train,
cv=regression_cv,
scoring={
"r2": "r2",
"mae": "neg_mean_absolute_error",
},
n_jobs=-1,
)Common Pipeline mistakes
Fitting the transformer before cross-validation
Avoid scaler.fit_transform(X) before calling cross-validation. Put the scaler inside the pipeline.
Encoding categories separately in train and test data
Independent encoding can produce different columns. Fit the encoder within the pipeline and use it to transform later data.
Forgetting unknown categories
Production data can contain categories unseen during training. For one-hot encoding, set handle_unknown="ignore" unless the application requires a stricter policy.
Applying scaling to tree models automatically
Tree-based models usually do not need standardization. Keep preprocessing choices model-specific, or compare a scaler with "passthrough".
Using an ordinary random split for grouped or temporal data
Pipelines prevent preprocessing leakage, but they cannot repair an inappropriate validation strategy. Use group-aware splits for repeated subjects and time-aware splits for forecasting.
Practical checklist
- Define the prediction target and final evaluation metric.
- Create the held-out test set before fitting transformations.
- Identify numeric, categorical, text, and date features.
- Put learned transformations inside pipelines.
- Use
ColumnTransformerfor heterogeneous columns. - Cross-validate the entire workflow.
- Tune preprocessing and model parameters together.
- Fit the selected pipeline on all training data.
- Evaluate once on the held-out test set.
- Save and deploy the complete fitted pipeline.
For tree-model tuning, continue with Decision Tree Hyperparameter Tuning with GridSearchCV. For regularized linear models, see Ridge vs Lasso vs Elastic Net.
Frequently asked questions
Does Pipeline automatically prevent every kind of leakage?
No. It prevents leakage from transformations placed inside it. Leakage can still enter through bad feature design, target-derived variables, duplicate records, future information, or an unsuitable cross-validation split.
Can a pipeline contain no preprocessing?
Yes. A step can be set to "passthrough" or None, and the pipeline can contain only an estimator. This is useful for testing original data against transformed alternatives.
Can GridSearchCV tune Pipeline parameters?
Yes. Prefix the parameter with its step name and two underscores. Nested pipelines continue the same naming pattern.
Should I save the preprocessor and model separately?
Usually no. Saving the fitted pipeline as one artifact reduces the risk of applying different preprocessing during inference.
Official references: Pipelines and composite estimators, ColumnTransformer, and cross-validation guide.
No comments:
Post a Comment