The test set can leak before the model sees its labels
A small preprocessing example: you want to standardize a numeric feature before fitting a model. If you calculate its mean and scale from the entire dataset and only then split, the held-out examples have already influenced the transformation.
A cleaner sequence is:
- Choose the train/test split appropriate to the task.
- Fit the preprocessing on the training subset.
- Apply that learned transformation to both subsets.
- Fit the model using the training subset and evaluate on the held-out subset.
Within cross-validation, fit the preprocessing inside each training fold. A pipeline helps keep the learned transformation with the estimator. It does not repair a poorly chosen split by itself.
For a review exercise, draw an arrow from every learned quantity to the rows that influenced it. If a held-out row feeds a training-time choice, investigate that path.
Source: scikit-learn, Common pitfalls—Data leakage:
https://scikit-learn.org/stable/common_pitfalls.html#data-leakage
Which preprocessing step would you inspect first in a pipeline you inherited?