1.2.6 Predictions and coefficients
Every result, whether a single fit or a regularization path, is a glmnet-model?. Three procedures work on all of them:
predict evaluates the model on new rows, given as a design matrix with one column per predictor, as R’s predict does, or, for a model fitted from a formula, as a table;
coef returns the intercept, then one coefficient per predictor, as R’s coef does, keyed by name for a model fitted from a formula;
deviance-ratio returns the fraction of null deviance explained, which R keeps in a fit’s dev.ratio field.
> (define fit (lasso X y #:lambda 0.1)) > (predict fit '((6.0 5.0) (7.0 8.0))) '(7.3953337984542635 6.978083815718123)
> (coef fit) '#(1.1471547078412585 1.602368245039978 -0.6732060759253727)
> (deviance-ratio fit) 0.9619734815443559
1.2.6.1 What is predicted
#:type chooses what predict returns, with R’s names: 'link, the default, is the linear predictor; 'response is on the scale of the response; and 'class is the predicted class, for the two families that have classes:
Family |
| 'link |
| 'response |
| 'class |
Gaussian |
| η = β₀ + xβ |
| η |
| --- |
Binomial |
| log-odds η |
| P(y = 1) |
| 1 if η > 0, else 0 |
Multinomial |
| ηk, one per class |
| softmax of the ηk |
| the class with the largest ηk |
Cox |
| xβ |
| relative risk exp(xβ) |
| --- |
Poisson |
| log μ = η |
| mean μ = exp(η) |
| --- |
Multi-response |
| ηr, one per response |
| ηr |
| --- |
> (define labels '(0 1 0 1 1)) > (define clf (logistic-fit X labels #:lambda 0.05)) > (predict clf '((1.0 1.0) (5.0 5.0))) '(-1.043101685663038 2.4705027624505362)
> (predict clf '((1.0 1.0) (5.0 5.0)) #:type 'response) '(0.26055196509639184 0.9220479088732567)
> (predict clf '((1.0 1.0) (5.0 5.0)) #:type 'class) '(0 1)
> (predict fit X #:type 'class) predict: type 'class is only for the binomial and
multinomial families
family: 'gaussian
The prediction helpers of each family are predict at a fixed type: logistic-predict-proba is #:type 'response, for example, and elnet-predict is the Gaussian default.
1.2.6.2 Predicting along a path
For a path, #:lambda chooses the λ at which to evaluate, as R’s s does. It is a single λ or a list; for a list, the result has one entry per element, in the order given. Without #:lambda, a path gives one entry per fitted λ:
> (glmnet-path-lambda user-path) '#(0.5 0.1 0.01)
> (coef user-path #:lambda 0.1) '#(1.1471547078412585 1.602368245039978 -0.6732060759253727)
> (predict user-path '((6.0 5.0))) '((5.739339828220178) (7.3953337984542635) (7.938934279162218))
> (predict user-path '((6.0 5.0)) #:lambda '(0.01 0.5)) '((7.938934279162218) (5.739339828220178))
A λ that was not fitted is handled as R handles it by default: the coefficients are interpolated linearly in λ between the two fitted values on either side. For λl > s > λr,
β(s) = w β(λl) + (1 − w) β(λr), w = (s − λr) / (λl − λr),
and the same holds for the intercept. At s = 0.2, between 0.5 and 0.1, the weight is w = 0.25:
> (coef user-path #:lambda 0.2) '#(1.3255310738258994 1.3633878361316651 -0.5049045569440296)
> (for/vector ([hi (in-vector (coef user-path #:lambda 0.5))] [lo (in-vector (coef user-path #:lambda 0.1))]) (+ (* 0.25 hi) (* 0.75 lo))) '#(1.3255310738258994 1.3633878361316651 -0.5049045569440296)
A λ above the largest fitted value, or below the smallest, is clamped to that end of the path:
> (equal? (coef user-path #:lambda 5.0) (coef user-path #:lambda 0.5)) #t
> (equal? (coef user-path #:lambda 0.0) (coef user-path #:lambda 0.01)) #t
Interpolated coefficients approximate the fit at s; they are not that fit. The closer together the fitted values, the better the approximation. To get the fit itself, fit at s: R’s exact = TRUE refits, and here a single fit or a path through s does the same:
> (coef (lasso X y #:lambda 0.2)) '#(1.2940231128148407 1.2050782064058927 -0.3466430412601621)
A single fit is a path with one λ, so, as in R, it gives the same answer whatever #:lambda says:
> (equal? (coef fit #:lambda 0.5) (coef fit)) #t