Underfitting

Underfitting happens when a model is too simple to represent the pattern hiding in the data. Picture trying to fit a straight line to points that clearly bend in a curve — no matter how you place the line, it misses. That gap between what the model can express and what the data actually does is called bias, and underfitting is the high-bias failure. The tell is distinctive. An underfit model does badly on the very data it was trained on, and it does badly on new data too, because it never learned the shape of the problem in the first place. It is not confused by noise; it is blind to signal. A model that cannot even fit its own training set is almost always underfitting rather than struggling with anything subtler.

Underfitting comes from a model that lacks the capacity, flexibility, or information to do the job. A linear model asked to learn a curved relationship will underfit; so will a decision tree stunted to two levels, or a model trained for too few steps to converge. Heavy-handed regularization — the penalties added to discourage complexity — can also force underfitting by squeezing the model until it is too rigid. Missing or weak features starve the model of the inputs it would need to see the pattern. In every case the diagnosis is the same. The model is not expressive enough for the truth in the data, and its errors are systematic rather than random. Adding capacity, richer features, more training, or lighter regularization is the general cure.

Underfitting and overfitting are opposite failures, and a model sits somewhere on the line between them. Underfitting is too little learning — the model is too simple, has high bias, and misses the pattern even on training data. Overfitting is too much — the model is so flexible that it memorizes the training data, including its noise and quirks, and then fails on new data. The clearest diagnostic is the gap between training and test performance. An underfit model is bad on both, and the two scores sit close together, both poor. An overfit model is excellent on training data but much worse on test data, and the gap between them is wide. Reading those two numbers together tells you which way the model is failing.

The goal is the sweet spot between the two, often framed as the bias-variance trade-off. Underfitting is high bias and low variance, the model consistently wrong in the same way. Overfitting is low bias and high variance, the model erratic, swinging with the particular training sample. You move away from underfitting by adding capacity — a richer model, better features, longer training, weaker regularization — and away from overfitting by doing the reverse. Because the two are opposites, an over-aggressive fix for one can cause the other. Tighten a model to stop overfitting and you can tip it into underfitting. The craft is finding the balance where the model is complex enough to capture the signal but not so complex it starts chasing the noise.

To catch underfitting, look first at training performance. If a model cannot do well on the data it was trained on, it is almost certainly underfitting, and no amount of extra data will help, because the problem is the model, not the sample. From there the fixes are direct. Give the model more capacity — a more flexible algorithm, more layers, more parameters. Engineer better features so the pattern is visible in the inputs. Train longer if it stopped before converging. Loosen regularization if the penalties are strangling it. Each of these adds the expressiveness the model was missing. The point is to raise the model's ability to represent the pattern until it can fit the training data well, then check that it still generalizes to new data.

The traps are mistaking underfitting for something else and reaching for the wrong lever. Throwing more data at an underfit model wastes effort, since the model still cannot use it. Fighting imagined overfitting by adding regularization makes underfitting worse. And declaring a model finished because its error is low can hide underfitting if the baseline itself is weak. The honest check is always the training score. If it is poor, add capacity; if training is strong but test is weak, you have swung to overfitting instead. Underfitting is the easier failure to fix once you have named it correctly, because the remedy — a more capable model with better inputs — is well understood. The mistake is treating a too-simple model as if its problem were too much complexity.

Underfitting joins the prefix under- (too little) with fit, the statistical sense of how closely a model matches data, naming a model that fits the pattern too loosely.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where underfitting is a core concern: