1.3.6 Polars dataframes🔗ℹ

glmnet/data/polars converts between the dataframes of polars (rkt-polars) and glmnet’s data: design matrices, responses and tables, both ways. (require glmnet) does not load it, and so does not load Polars.

See Polars dataframes in the Reference for the five procedures.

Reading a file with Polars’ read-csv and converting it is the fast way to fit real data: the predictors leave Polars in one bulk copy, already in the column-major layout, and become a design matrix without another. Here Polars reads R’s iris from the copy that glmnet/datasets ships:

> (require glmnet/data/polars            (only-in polars read-csv column-names dataframe series cast))

> (define flowers (read-csv (collection-file-path "iris.csv" "glmnet" "datasets")))

> (column-names flowers)

'("Sepal.Length" "Sepal.Width" "Petal.Length" "Petal.Width" "Species")

> (define measures     (polars->design-matrix flowers '("Sepal.Width" "Petal.Length" "Petal.Width")))

> (design-matrix-column-names measures)

'("Sepal.Width" "Petal.Length" "Petal.Width")

> (define sepal-length (polars->response flowers "Sepal.Length"))

> (elnet-result-coefficients (lasso measures sepal-length #:lambda 0.01))

'#(0.5717696868678742 0.534050705108672 -0.16945221616247616)

A formula fits from a dataframe once it is a table, which polars->table makes of it. Its string column, "Species", is a factor:

> (define flower-model     (formula-fit (Sepal.Length . ~ . Petal.Width + Species) (polars->table flowers)                  #:lambda 0.01))

> (coef flower-model)

'(("(Intercept)" . 4.799239099455094)   ("Petal.Width" . 0.8636539037617951)   ("Speciesversicolor" . 0.0)   ("Speciesvirginica" . 0.024855955899776596))

A categorical or enum column becomes a column of symbols, a factor too. Its levels are sorted as strings, as a column of strings’ are: the order of an enum’s categories is not kept. R’s factor keeps the order of its levels, so R takes low as the baseline of factor(dose, levels = c("low", "mid", "high")) and fits dosemid and dosehigh, where here "high" is the baseline:

> (define doses     (dataframe      (list (series '(1.0 3.2 2.1 0.9 2.0 3.1) #:name "y")            (cast (series '("low" "high" "mid" "low" "mid" "high") #:name "dose")                  '(enum low mid high)))))

> (polars->table doses '("dose"))

'(("dose" . #(low high mid low mid high)))

> (coef (formula-fit (~ y dose) (polars->table doses) #:lambda 0))

'(("(Intercept)" . 3.1499832153320315)   ("doselow" . -2.199966430664063)   ("dosemid" . -1.0999832153320321))

A missing value (a Polars null), a column that is not numeric where numbers are needed, and a value that is not finite are errors that name the column, and the row or the dtype. design-matrix->polars and table->polars convert back. The columns of a design matrix with no names are named "V1", "V2" and so on, as R names them:

> (design-matrix->polars (design-matrix-select-rows measures '(0 50 100)))

shape: (3, 3) ┌─────────────┬──────────────┬─────────────┐ │ Sepal.Width ┆ Petal.Length ┆ Petal.Width │ │ ---         ┆ ---          ┆ ---         │ │ f64         ┆ f64          ┆ f64         │ ╞═════════════╪══════════════╪═════════════╡ │ 3.5         ┆ 1.4          ┆ 0.2         │ │ 3.2         ┆ 4.7          ┆ 1.4         │ │ 3.3         ┆ 6.0          ┆ 2.5         │ └─────────────┴──────────────┴─────────────┘

> (design-matrix->polars (rows->design-matrix '((1 2) (3 4))))

shape: (2, 2) ┌─────┬─────┐ │ V1  ┆ V2  │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞═════╪═════╡ │ 1.0 ┆ 2.0 │ │ 3.0 ┆ 4.0 │ └─────┴─────┘

> (table->polars (list (cons "y" '(1.5 2.5)) (cons "group" '("a" "b"))))

shape: (2, 2) ┌─────┬───────┐ │ y   ┆ group │ │ --- ┆ ---   │ │ f64 ┆ str   │ ╞═════╪═══════╡ │ 1.5 ┆ a     │ │ 2.5 ┆ b     │ └─────┴───────┘

> (polars->design-matrix flowers '("Petal.Width" "Species"))

polars->design-matrix: contract violation

expected: a non-empty list of distinct names of numeric

columns of the dataframe

given: column "Species", of dtype string

in: the columns argument of

(->i

((df

(and/c dataframe? dataframe-with-rows/c))

(columns

(df)

(frame-columns/c

df

numeric-dtype?

"numeric")))

(result design-matrix?))

contract from: <pkgs>/glmnet/data/polars.rkt

blaming: top-level

(assuming the contract is correct)

at: <pkgs>/glmnet/data/polars.rkt:22:3

design-matrix->polars is the slower direction. rkt-polars makes a series only from a list or a vector, which it reads element by element, so each column goes through a vector of flonums. A series made from an f64vector in one copy, which rkt-polars does not have yet (rkt-polars#145), would make it as fast as the way in.