Regression (Machine Learning)

Regression, in machine learning, is a type of supervised learning that predicts a continuous numeric value from input features. Give the model examples where the answer is known — houses with their features and their sale prices, days with their conditions and the demand that followed — and it learns a function that maps inputs to a number, so it can estimate that number for new, unseen cases. The output is a quantity on a continuous scale: a price, a temperature, a sales figure, an expected lifetime value, a probability. That continuous output is what defines regression and separates it from classification, which predicts a discrete category. Regression answers how much or how many, rather than which class a case belongs to.

Regression ranges from simple to sophisticated. Linear regression, the classic form, fits a straight-line relationship between inputs and the target and is prized for being interpretable. More flexible methods — decision-tree ensembles, gradient boosting, neural networks — can capture complicated, nonlinear patterns at the cost of some transparency. Whatever the method, the model is trained by minimizing the gap between its predictions and the true values in the training data, and its quality is judged by error metrics that measure how far off its predictions are, such as mean absolute error or root mean squared error. The aim is a model that predicts the continuous target accurately on new data, not merely one that fits the examples it has already seen.

The defining contrast is with classification, the other main branch of supervised learning. Classification predicts a category — spam or not spam, which of several products a customer will pick, whether a transaction is fraud. Regression predicts a number on a continuous scale — the dollar value of the transaction, the expected revenue, the days until churn. The nature of the target decides which you need: if the thing you are predicting is a quantity, it is regression; if it is a label or class, it is classification. This choice shapes everything downstream — the algorithms, the loss functions used to train them, and the metrics used to judge them — so naming the problem correctly is the first step in any project.

The line between them is real but not a wall. Some methods handle both with different settings, and a problem can be framed either way depending on the question. Predicting the exact probability of churn is regression (a continuous value); predicting simply whether a customer will churn is classification (a yes or no label). You can even turn a regression output into a classification by thresholding it, or handle ordered categories with methods that borrow from both. The point is to be deliberate: decide whether you truly need a number or a category, because using classification when you need a quantity throws away information, and forcing regression onto a categorical target predicts a number where none exists. Match the method to the shape of the answer you need.

Using regression well means being clear that you are predicting a continuous value, choosing a method that fits the complexity of the relationship, and judging the model by how well it predicts on data it has not seen. That means holding out test data, watching error metrics that reflect what matters for the decision, and guarding against overfitting — a model that memorizes the training examples but fails on new ones. It means understanding the features that drive the prediction, checking that the model behaves sensibly across the range of inputs, and preferring an interpretable model when understanding the relationship matters as much as the prediction itself. A regression model earns trust by predicting accurately in production, not by fitting history perfectly.

The failures are confusing regression with classification (predicting a number when you needed a category, or the reverse), overfitting so the model looks great on training data and poor in the real world, judging the model on the wrong error metric, and extrapolating confidently far beyond the range of the training data where the learned relationship may not hold. Regression models also inherit the general fragility of machine learning — they drift as the underlying relationship changes and must be monitored and retrained. The discipline is to frame the problem correctly as continuous prediction, validate honestly on unseen data, pick error metrics that match the decision, respect the limits of the training range, and maintain the model over time as the world it predicts keeps moving.

Regression in machine learning — supervised prediction of a continuous numeric value from input features — contrasts with classification, which predicts discrete categories; the term traces to statistical regression analysis.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where regression (machine learning) is a core concern: