Overfitting and Underfitting: Why a Model Can Shine on Old Data and Fail on New Data

A machine learning model that reaches near-perfect accuracy on the data used to train it, yet gives disappointing results on new data, is a very common situation. The most frequent cause is overfitting. At the opposite end, a model that is too simple can perform poorly everywhere, which is called underfitting. These two concepts are the foundation for understanding why model evaluation must be done carefully.

The real goal: generalisation

We do not build a model so that it memorises old data, but so that it performs well on data it has never seen. This ability is called generalisation. Think of a student: memorising the answers of past exams earns a high mark if the same exam reappears, but is no help when the questions change. A student who understands the underlying ideas can handle new questions.

Overfitting: learning the noise as well

Overfitting occurs when a model is too complex for the amount of data available, so it learns not only the true pattern but also the noise and random quirks of the training set. The typical sign is very low error on the training set but clearly higher error on test data. A decision tree allowed to grow without limit, or a very large neural network trained on a small dataset, are easy examples.

Underfitting: not learning enough

Conversely, underfitting occurs when a model is too simple to capture the pattern in the data. For instance, using a straight line to describe a relationship that is actually curved. The sign is high error on both the training set and the test set. In that case, adding more data usually helps little; what needs to change is the model itself or its input features.

How to detect them: split the data and compare

The most basic tool is to divide the data into separate parts:

• Training set: used for the model to learn.
• Validation set: used to compare configurations and tune hyperparameters.
• Test set: kept until the end to estimate real-world quality, and never used to choose the model.

Then compare the results across sets. A large gap between training and test results suggests overfitting; poor results on both suggest underfitting. Cross-validation gives a more stable assessment when data is limited.

Common remedies

For overfitting, you can:

• Collect more and more varied data.
• Simplify the model, for example by limiting tree depth or reducing the number of network layers.
• Apply regularisation to penalise overly complex models.
• Stop training early once results on the validation set stop improving.

For underfitting, you can use a model with greater representational capacity, add meaningful features, or train for longer. The aim is to find a balance between complexity and generalisation, often discussed as the bias-variance trade-off.

A related pitfall: data leakage

Sometimes test results are suspiciously good because information from the test set has slipped into training, for example when data is normalised across the whole dataset before splitting. This is called data leakage, and it makes us judge the model too optimistically. Always carry out preprocessing steps after the data has been split.

Conclusion

Overfitting is learning too closely details that are not worth learning; underfitting is not learning enough of what needs to be learned. The most effective safeguard is to always evaluate a model on data it has never seen, watch the gap between training and test results, and adjust complexity to suit the problem.