Ridge regression — shrink coefficients, drop none
Ridge minimizes squared error plus alpha times the sum of squared coefficients. Every coefficient gets smaller. None is forced to zero. That is the difference from lasso, and it is why ridge is the stable choice when two columns carry the same signal: the weight is shared instead of being assigned, almost at random, to one of them.
Alpha is the shrinkage. At alpha 0 you have ordinary least squares. As alpha grows, the coefficients and the predictions both move toward zero (toward the mean, if the intercept is unpenalized). The right alpha is the one that minimizes error on rows the fit did not see, typically chosen on a log grid.
Standardize columns before comparing alphas across datasets. Alpha is calibrated to the scale of the coefficients, which inherit the scale of the columns.
Settings that change this fit
- alpha
- Penalty on the squared length of the coefficient vector. Larger alpha means a flatter prediction and a smaller training R².
- solver
- Changes speed, not the objective. On a small CSV every solver should land on the same coefficients up to numerical noise.