1 User guide

9.3

Racket

1 User guide🔗ℹ

Every snippet has a runnable counterpart under user-guide/<chapter>/ in the project repository, paired with the equivalent Python program:

racket user-guide/getting-started/expressions-and-contexts.rkt

python user-guide/getting-started/expressions_and_contexts.py

The exception is the closing case study, Case study: nycflights, which replays a blog post rather than an upstream page; examples/69-nycflights-replay.rkt runs its main steps as a script.

Snippets use the fluent, thread-first style: each verb takes the frame — or the expression — as its first argument, and ~> (re-provided by polars) chains them, within an expression as much as across frames. Where the bindings have no spelling for an upstream call, the chapter says so and shows the nearest workaround. For the definition of every binding mentioned, see the Reference.

1.1 Getting started🔗ℹ

See Operators and pipelines for the definition of every verb used here.

1.1.1 Installing

1.1.2 Reading & writing

1.1.3 Expressions and contexts

1.1.3.5 More complex queries

1.1.4 Combining dataframes

1.1.4.1 Joining

1.1.4.2 Concatenating

1.2 Concepts🔗ℹ

See Operators and pipelines for the operators and contexts used here.

1.2.1 Data types and structures

1.2.1.1 Series

1.2.1.2 Dataframe

1.2.1.2.1 Inspecting a dataframe

1.2.1.3 Schema

1.2.1.4 Data types

1.2.2 Expressions and contexts

1.2.2.1 Expressions

1.2.2.2 Contexts

1.2.2.2.1 select

1.2.2.2.2 with-columns

1.2.2.2.3 filter

1.2.2.2.4 group-by and aggregations

1.2.2.3 Expression expansion

1.2.3 Lazy API

1.3 Expressions🔗ℹ

See Operators and pipelines for col, all, exclude and over.

1.3.1 Expression expansion

1.3.1.1 Function col

1.3.1.1.1 Explicit expansion by column name

1.3.1.1.2 Expansion by data type

1.3.1.1.3 Expansion by pattern matching

1.3.1.1.4 Arguments cannot be of mixed types

1.3.1.2 Selecting all columns

1.3.1.3 Excluding columns

1.3.1.4 Column renaming

1.3.1.4.1 Renaming a single column with alias

1.3.1.4.2 Prefixing and suffixing column names

1.3.1.4.3 Dynamic name replacement

1.3.1.5 Programmatically generating expressions

1.3.1.6 More flexible column selections

1.3.1.6.1 Debugging selectors

1.3.2 Categorical data and enums

1.3.2.1 Data type Enum

1.3.2.1.1 Creating an Enum

1.3.2.1.2 Invalid values

1.3.2.1.3 Category ordering and comparison

1.3.2.2 Data type Categorical

1.3.2.2.1 Creating a Categorical series

1.3.2.2.2 Using Categories objects

1.3.2.2.3 Lexical comparison with strings

1.3.2.2.4 Combining categorical columns

1.3.2.3 Performance considerations

1.3.2.3.1 Encodings

1.3.2.3.2 Enum encodings are fixed

1.3.2.3.3 Categorical encodings

1.3.3 Window functions

1.3.3.1 Operations per group

1.3.3.2 Mapping results to dataframe rows

1.3.3.2.1 group_to_rows

1.3.3.2.2 explode

1.3.3.2.3 join

1.3.3.3 Windowed aggregation expressions

1.3.3.4 More examples

1.4 Time series🔗ℹ

See Dates, times, datetimes and durations for how gregor values map to the temporal dtypes.

1.4.1 Parsing

1.4.1.1 Parsing dates from a file

1.4.1.2 Casting strings to dates

1.4.1.3 Extracting date features

1.4.1.4 Mixed offsets

1.4.1.5 Dates from Racket values

1.4.2 Filtering

1.4.2.1 Filtering by single dates

1.4.2.2 Filtering by a date range

1.4.2.3 Filtering with negative dates

1.4.3 Time zones

1.4.3.1 Zoned values from Racket

The examples read upstream’s "apple_stock.csv", a hundred closing prices of Apple stock from 1981 to 2014.

API gaps: no group_by_dynamic, rolling or upsample, so upstream’s Grouping and Resampling pages have no counterpart.

1.5 IO🔗ℹ

See Operators and pipelines for read-csv, scan-csv and the other readers.

1.5.1 CSV

1.5.1.1 Read & write

1.5.1.2 Scan

1.5.1.3 Reading options

1.5.1.4 Writing options

1.5.1.5 Parsing dates

1.5.2 JSON files

1.5.2.1 Read

1.5.2.1.1 JSON

1.5.2.1.2 Newline Delimited JSON

1.5.2.2 Write

1.5.2.3 Scan

1.5.2.4 Reading options

1.5.3.5 Projection and predicate pushdown

1.5.3.6 Options

1.5.4 Multiple files

1.5.4.1 Reading into a single dataframe

1.5.4.2 Reading and processing in parallel

1.6 Interoperability🔗ℹ

See Converting to Racket values for every conversion used here.

Handing Polars data to Racket code that is not Polars: a loop, a numeric routine, a foreign solver, a plot. The chapter mirrors upstream’s Arrow producer/consumer and Visualization pages, with the Python export calls those pages lean on: Series.to_list, Series.to_numpy, DataFrame.to_dict and DataFrame.to_numpy.

1.6.1 Series to Racket values

1.6.2 Iterating

1.6.3 Columns as Racket data

1.6.4 Numeric buffers

1.6.5 Data for a plot

1.7 Case study: nycflights🔗ℹ

See Operators and pipelines for the verbs used here, and Series and DataFrames for the conversions, define-enum, describe and the accessors.

Data frames for Racket (aliquote.org, July 2023) loads the 29 MB nycflights.tsv, every flight that left New York in 2013, with three Racket libraries, then walks tabular-asa through a select, a distinct, a group-by and a density plot. Along the way it lists what the libraries cannot do: read another separator, keep categorical variables, parse dates, describe a string column usefully. This chapter replays each step in rkt-polars, the post’s code first, and answers each complaint.

1.7.1 The data

1.7.2 Requiring polars

1.7.3 Loading the file

1.7.4 Selecting columns

1.7.5 Distinct destinations

1.7.6 Counting and averaging by destination

1.7.7 Delays into Racket, and a density plot

1.7.8 Filtering

1.7.9 The largest delays

1.7.10 Other separators

1.7.11 Categorical variables

1.7.12 Describing strings and datetimes

1.7.13 Performance

1.1	Getting started
1.2	Concepts
1.3	Expressions
1.4	Time series
1.5	IO
1.6	Interoperability
1.7	Case study: nycflights