If we split the data into train/test splits, is it still possible to over-fit our model?

Data Management.

Yes, it’s definitely possible. One common beginner mistake is re-tuning a model or training new models with different parameters after seeing its performance on the test set.

In this case, its the model selection process that causes the overfitting. The test set should not be tainted until you’re ready to make your final selection.

Supervised Learning