1.2.2 The penalty: α and λ🔗ℹ

For the Gaussian family, glmnet solves

min_{β₀,β} 1/(2n) ‖y − β₀ − Xβ‖² + λ [ α‖β‖₁ + (1−α)/2 ‖β‖₂² ]

over the intercept β₀ and the coefficients β. The other families replace the squared-error term with their negative log-likelihood; the penalty is the same for all of them. It has two knobs:

• #:alpha, α ∈ [0, 1], mixes the penalty. α = 0 is the ridge (L2) penalty, which shrinks every coefficient but zeroes none; α = 1 is the lasso (L1) penalty, which sets coefficients exactly to zero; values in between are the elastic net. Every fit procedure defaults to 1.0.

• #:lambda, λ ≥ 0, sets the penalty’s strength. A single fit requires it; to fit a whole sequence of values at once, use a regularization path (Regularization paths). λ = 0 is the unpenalized fit.

The four Gaussian models are one routine, elnet-fit, with different arguments; the named procedures only fix #:alpha:

> (equal? (elnet-result-coefficients (lasso X y #:lambda 0.1))           (elnet-result-coefficients (elnet-fit X y #:alpha 1.0 #:lambda 0.1)))

#t

> (lasso X y)

lasso: arity mismatch;

the expected number of arguments does not match the given

number

expected: 2 plus an argument with keyword #:lambda plus

optional arguments with keywords #:intercept?, #:max-iters,

#:standardize?, and #:thresh

given: 1

arguments...:

'(((1.0 2.0) (2.0 1.0) (3.0 4.0) (4.0 3.0) (5.0 6.0)) (1

4 3 6 5))