Lasso regression — zero the coefficients you do not need
Lasso minimizes squared error plus alpha times the sum of absolute coefficients. The geometry of the absolute-value penalty puts the optimum on the axes, so some coefficients land on exactly zero. Those columns are out of the model. That is feature selection done inside the fit, not as a separate ranking step.
The path over alpha is the useful picture. At a tiny alpha almost every column is in. As alpha rises, coefficients hit zero one by one and stay there. Correlated columns make this path unstable: a small change in the rows can swap which of two twins survives. If that swap would embarrass the write-up, use elastic net instead.
Do not interpret a zero as proof the column is unrelated to the target. It means that, given the other columns and this alpha, the column did not earn its coefficient.
Settings that change this fit
- alpha
- How hard zeros are enforced. Pick it by cross-validated error, then look at which columns survived, not only at the error.
- max iter
- The solver is iterative. If it warns about convergence, the zeros you see are not the zeros of the objective yet.