1.4.5.1 Squares, and R’s I()🔗ℹ

(I expr) is R’s I(): the transform whose values are those of expr, any Racket arithmetic. Inside a transform the operators are Racket’s, as they are inside R’s I() and any other function call, so (I (* age dose)) is the product of two columns and (log (+ dose 1)) adds 1.

This is how a formula squares a column. (^ age 2) is crossing, and the same as age (see Powers, and R’s x^2); R’s y ~ x + x^2 is y ~ x. The square is (I (expt age 2)), as R writes I(x^2), or (sqr age), with sqr from racket/math:

> (require racket/math)

> (formula-predictor-names (bp . ~ . age + age ^ 2) patients)

'("age")

> (formula-predictor-names (bp . ~ . age + (sqr age)) patients)

'("age" "(sqr age)")

> (coef (formula-fit (bp . ~ . age + (sqr age) + dose) patients #:lambda 0.1))

'(("(Intercept)" . 92.61009414594717)   ("age" . 0.5425216180401342)   ("(sqr age)" . 0.0)   ("dose" . -1.9132285961605522))

At λ = 0.1 the lasso leaves the square out, since blood pressure in the simulated data is linear in age. ^ is not a Racket function, and a transform that uses it is a syntax error that says so:

> (~ bp age (I (^ age 2)))

eval:68:0: ~: ^ is not a Racket function; inside a

transform, write a power as (expt x 2), or a square as (sqr

x)

at: ^

in: (I (^ age 2))

1.4.5.2 Names in a transform🔗ℹ

A transform reads names as R does, from the data first and then from the program. An identifier in argument position, that is, anywhere but first in a group, is the table’s column of that name when the table has one, and otherwise the Racket binding of that name where the formula is written. So a constant of the program can scale a column:

> (define reference-age 50)

> (define scaled (formula-design-matrix (~ bp (I (/ age reference-age))) patients))

> (design-matrix->rows (design-matrix-select-rows scaled '(0 1)))

'((1.1) (1.46))

In this chapter age is also a Racket variable, the list of ages that patients was built from, but inside the formula it names the column. The identifier first in a group, the function, is always the Racket binding, even when the table has a column of that name, as R looks up a function by name and skips a column: with a column max, (max max x) is the larger of that column and x, R’s pmax(max, x). Names that the transform binds itself, with let, lambda or for/sum, are its own, and so are not read from the table, and neither are the names in quoted data, nor the names that Racket’s forms match as literals, such as cond’s => and else and quasiquote’s unquote. A column named like one of these literals, such as else or _, cannot be read inside a transform, where (log else) is Racket’s syntax error else: not allowed as an expression; write it as an ordinary term, or rename it in the table. predict needs only the columns that a transform reads. A name that is neither a column nor bound is an error when the formula is fitted, which names it:

> (formula-fit (~ bp (I (* age scale)) dose) patients #:lambda 0.1)

formula-fit: a name in a transform is neither a column of

the table nor a defined variable

name: "scale"

transform: "(I (* age scale))"

At the top level, as in the REPL and in this chapter, a transform’s function must be defined before the formula, since a later definition cannot be seen there; in a module it can be defined anywhere in the module. A name in argument position can be defined later in both, since the transform reads it when it runs. A bug in Racket’s contracts at the top level makes a formula procedure whose first call in a session had a syntax error in its formula fail from then on with lifted/1.1: undefined; restart the REPL when that happens, or define the formula on its own first, as (define f (~ ....)), where a syntax error does no harm, and pass f to the procedure.

A transform is elementwise: it sees one row at a time. R’s scale(x) and x - mean(x) read the whole column; compute such a column in the table instead.

1.4.5.3 Values and new data🔗ℹ

Each value of a transform must be a real number and finite, and an error names the transform and the row where it is not, unless its values are strings, symbols or booleans, which make it a factor (see Transforms that make factors). predict evaluates the transforms again on the new table’s rows, so it needs the columns they read, and it reads their Racket bindings again, as R’s predict does:

> (define curved     (formula-fit (bp . ~ . age + (sqr age) + (log dose)) patients #:lambda 0.1))

> (predict curved new-patients)

'(107.9521299561662 117.50849519274944)

> (predict curved (list (cons "age" '(40)) (cons "dose" '(0.0))))

predict: a transform's value is not finite

transform: "(log dose)"

row: 0

value: -inf.0

1.4.5.4 Transforms as data🔗ℹ

A formula prints each transform as its source, and two formulas that write a transform the same way are equal?. make-formula takes a transform as a transform-term: a name, the columns it reads and a procedure of their values, which has no source to name it by:

> (define log-age (transform-term "(log age)" '("age") log))

> (make-formula 'bp log-age 'dose)

(~ bp (log age) dose)

> (equal? (make-formula 'bp log-age 'dose) (~ bp (log age) dose))

#t

1.4.5.5 From R’s transforms🔗ℹ

Transforms are named by their Racket source, so R’s log(hp) is the column "(log hp)" here:

R’s vectorized pmin is Racket’s min here, since a transform sees one row at a time.

A name is the source as the reader writes it, so quoted data keeps its abbreviation, 'x and not (quote x). A transform reads a column’s values as flonums, so the data it compares them with are flonums too, '(40.0 50.0) and not '(40 50), which memv would never find:

> (formula-predictor-names (~ bp (I (if (memv age '(40.0 50.0)) 1 0))) patients)

'("(I (if (memv age '(40.0 50.0)) 1 0))")