regression

Regression is the supervised learning task of predicting a continuous number — a house price, a wafer's projected yield percentage, tomorrow's temperature — as opposed to classification, which predicts a discrete category. The distinction isn't cosmetic: it decides the loss function, the output layer's shape, and how the model's error even gets measured, since "how far off was the prediction" is a meaningful question for a number in a way it isn't for a category label. At a high level, a regression model's training loop looks like this:

Linear regression is the simplest form, fitting a straight line through the data, and its coefficients are directly interpretable in a way almost nothing else in machine learning is. Each coefficient tells you exactly how much the predicted output changes per unit change in that input, holding everything else constant — a coefficient of 0.3 on square footage means every additional square foot adds 0.3 units to the predicted price, full stop, with zero translation layer needed. Polynomial regression fits a curve instead of a line, capturing more complex relationships, but at real risk: a high-degree polynomial can bend itself to pass through every training point exactly while describing nothing real about the underlying trend, the textbook example of overfitting.

Ridge and lasso regression both fight overfitting by adding a penalty for large coefficients, but the specific penalty shape decides whether a feature actually gets removed from the model. Ridge (L2) penalizes the sum of squared coefficients, shrinking them all toward zero but never exactly to zero — it stabilizes a model without ever truly eliminating a feature's influence. Lasso (L1) penalizes the sum of absolute coefficient values, which has the geometric property of often shrinking some coefficients to exactly zero, effectively performing automatic feature selection as a side effect of regularization rather than requiring a separate feature-selection step.

Which error metric a regression model gets evaluated on is a real modeling decision, not a reporting afterthought, because different metrics penalize mistakes differently. Mean absolute error treats every unit of error the same regardless of size, which is more robust when a dataset has occasional extreme outliers that shouldn't dominate the evaluation; mean squared error squares the error before averaging, which penalizes large mistakes disproportionately harder than small ones — appropriate when a big miss is genuinely much worse than several small ones, like a yield-prediction model that badly underestimates defect density on a critical lot. Choosing the metric that matches the real cost of being wrong, not defaulting to whichever is more common, is what makes a regression model's reported accuracy actually meaningful for the decision it's feeding into.

Read regression as the task where a model's error has a magnitude, not just a right-or-wrong verdict, which is exactly what makes metric choice and regularization strategy such consequential decisions: a classification model is either correct or it isn't, but a regression model's usefulness depends entirely on how wrong "wrong" is allowed to be, and on whether the chosen error metric actually reflects that.