Linear regression — fit a straight weighted sum
Ordinary least squares finds the coefficients that minimize the sum of squared residuals. The prediction is an intercept plus one weight per column. With more rows than columns and columns that are not linear copies of each other, the solution is unique and closed-form. R² then says what fraction of the target's variance those residuals no longer contain.
A coefficient is the change in the prediction for a one-unit change in that column, with the others held fixed. That unit is the column's unit. Comparing a coefficient on income to a coefficient on a 0/1 flag is meaningless until the columns are standardized, and even then a large coefficient on a column that is collinear with another is not a causal effect.
Squared error treats a residual of 10 as a hundred times worse than a residual of 1. One target value that is a data-entry error will tilt the whole line. Plot residuals against the prediction before you quote R².
Settings that change this fit
- fit intercept
- Leave it on unless the target is already centered and a zero-input row must predict zero.
- columns
- The only real hyperparameter. Extra noise columns still receive a coefficient and will move the prediction.