1.3.2.1.1 Creating an Enum🔗ℹ

> (define-enum bears-enum Polar Panda Brown)

> (define bears (series '(Polar Panda Brown Brown Polar) #:dtype bears-enum))

> bears

shape: (5,) Series: '' [enum] [ "Polar" "Panda" "Brown" "Brown" "Polar" ]

1.3.2.1.2 Invalid values🔗ℹ

> (series '(Polar Panda Brown Polar Shark) #:dtype bears-enum)

series: cannot convert to '(enum Polar Panda Brown):

conversion from `str` to `enum` failed in column '' for 1

out of 5 values: ["Shark"]

Ensure that all values in the input column are present in

the categories of the enum datatype.

1.3.2.1.3 Category ordering and comparison🔗ℹ

An Enum sorts and compares in the order of its categories.

> (define-enum log-levels debug info warning error)

> (define logs     (dataframe      (list (series '(debug info debug error) #:name "level" #:dtype log-levels)            (series '("process id: 525" "Service started correctly"                      "startup time: 67ms" "Cannot connect to DB!")                    #:name "message"))))

> (define non-debug-logs (filter logs (> (col "level") 'debug)))

> non-debug-logs

shape: (2, 2) ┌───────┬───────────────────────────┐ │ level ┆ message                   │ │ ---   ┆ ---                       │ │ enum  ┆ str                       │ ╞═══════╪═══════════════════════════╡ │ info  ┆ Service started correctly │ │ error ┆ Cannot connect to DB!     │ └───────┴───────────────────────────┘

A value outside the categories is an error:

> (select logs (> (col "level") "Pretty bad"))

lazyframe-collect: failed to collect the query: conversion

from `str` to `enum` failed for value "Pretty bad"

An Enum compares with an Enum, or with strings that are its categories:

> (define str-series (series '("info" "debug" "debug" "error")))

> (= (ref logs "level") str-series)

shape: (4,) Series: 'level' [bool] [ #f #f #t #t ]

1.3.2.2.1 Creating a Categorical series🔗ℹ

> (define bears-cat (series '(Polar Panda Brown Brown Polar) #:dtype 'categorical))

> bears-cat

shape: (5,) Series: '' [cat] [ "Polar" "Panda" "Brown" "Brown" "Polar" ]

> (dtype (series '(Polar Panda)))

'categorical

1.3.2.2.2 Using Categories objects🔗ℹ

API gap: no pl.Categories. Every categorical column shares Polars’ one global mapping.

1.3.2.2.3 Lexical comparison with strings🔗ℹ

A categorical compares with strings by the strings, not by the codes:

> (~> (dataframe (list (rename bears-cat "categorical")))       (with-columns (alias (< (col "categorical") "Cat") "categorical < \"Cat\"")))

shape: (5, 2) ┌─────────────┬─────────────────────┐ │ categorical ┆ categorical < "Cat" │ │ ---         ┆ ---                 │ │ cat         ┆ bool                │ ╞═════════════╪═════════════════════╡ │ Polar       ┆ false               │ │ Panda       ┆ false               │ │ Brown       ┆ true                │ │ Brown       ┆ true                │ │ Polar       ┆ false               │ └─────────────┴─────────────────────┘

> (~> (dataframe (list (rename bears-cat "categorical")                        (series '("Panda" "Brown" "Brown" "Polar" "Polar") #:name "string")))       (with-columns (alias (= (col "categorical") (col "string")) "categorical == string")))

shape: (5, 3) ┌─────────────┬────────┬───────────────────────┐ │ categorical ┆ string ┆ categorical == string │ │ ---         ┆ ---    ┆ ---                   │ │ cat         ┆ str    ┆ bool                  │ ╞═════════════╪════════╪═══════════════════════╡ │ Polar       ┆ Panda  ┆ false                 │ │ Panda       ┆ Brown  ┆ false                 │ │ Brown       ┆ Brown  ┆ true                  │ │ Brown       ┆ Polar  ┆ false                 │ │ Polar       ┆ Polar  ┆ true                  │ └─────────────┴────────┴───────────────────────┘

1.3.2.2.4 Combining categorical columns🔗ℹ

Categorical columns share the global mapping, so they stack with no re-encoding:

> (define male-bears     (dataframe (list (series '(Polar Brown Panda) #:name "species")                      (series '(450 500 110) #:name "weight"))))

> (define female-bears     (dataframe (list (series '(Brown Polar Panda) #:name "species")                      (series '(340 200 90) #:name "weight"))))

> (vstack male-bears female-bears)

shape: (6, 2) ┌─────────┬────────┐ │ species ┆ weight │ │ ---     ┆ ---    │ │ cat     ┆ i64    │ ╞═════════╪════════╡ │ Polar   ┆ 450    │ │ Brown   ┆ 500    │ │ Panda   ┆ 110    │ │ Brown   ┆ 340    │ │ Polar   ┆ 200    │ │ Panda   ┆ 90     │ └─────────┴────────┘

1.3.2.3.1 Encodings🔗ℹ

The codes are small integers, so grouping, joining and comparing work on integers, not strings. They stay inside Polars: Racket sees symbols, which Racket interns, so a conversion builds one symbol per distinct string.

1.3.2.3.2 Enum encodings are fixed🔗ℹ

An Enum’s codes follow its declared categories, so two columns of one Enum share an encoding and need no re-encoding.

1.3.2.3.3 Categorical encodings🔗ℹ

A categorical’s codes come from the global mapping in order of first appearance, across every categorical column in the process; the mapping restarts once the last categorical column is gone. API gap: no Series.extend, so stack one-column frames:

> (define cat-bears (series '(Polar Panda Brown Brown Polar) #:dtype 'categorical))

> (define cat2-series (series '(Panda Brown Brown Polar Polar) #:dtype 'categorical))

> (ref (vstack (dataframe (list cat-bears)) (dataframe (list cat2-series))) 0)

shape: (10,) Series: '' [cat] [ "Polar" "Panda" "Brown" "Brown" "Polar" "Panda" "Brown" "Brown" "Polar" "Polar" ]