3.4 Elastic net (0 < α < 1)🔗ℹ

The elastic net blends the two penalties: with 0 < α < 1 the objective mixes the lasso’s L1 term and the ridge’s L2 term. It keeps the lasso’s ability to select variables while borrowing the ridge’s stability — in particular it tends to keep or drop groups of correlated predictors together, where the pure lasso would arbitrarily pick one. Setting α = 0 recovers Ridge regression (L2, α = 0) exactly and α = 1 recovers Lasso (L1, α = 1); the interesting models live in between.

On the running fixture, an α = 0.5 fit at a moderate λ both shrinks its coefficients (ridge-like) and drives at least one exactly to zero (lasso-like). At a shared λ, the number of coefficients it zeros sits between ridge (which zeros none) and lasso (which zeros the most) — the characteristic in-between behaviour of the blend.

elastic-net takes an explicit #:alpha. Compare its fit to the ridge and lasso fits at the same λ to see it land between them.