1.4.1.1 Read & write🔗ℹ

Writing a CSV file uses write-csv, and reading one should look familiar:

> (define df     (dataframe (list (series '(1 2 3) #:name "foo")                      (series (list polars-null "bak" "baz") #:name "bar"))))

> (write-csv df path)

> (read-csv path)

shape: (3, 2) ┌─────┬──────┐ │ foo ┆ bar  │ │ --- ┆ ---  │ │ i64 ┆ str  │ ╞═════╪══════╡ │ 1   ┆ null │ │ 2   ┆ bak  │ │ 3   ┆ baz  │ └─────┴──────┘

1.4.1.2 Scan🔗ℹ

Polars can also scan a CSV input. Scanning delays the actual parsing of the file and returns a lazyframe, a lazy computation holder; collect runs it. See Lazy API for why that is desirable.

> (scan-csv path)

#<lazyframe>

> (collect (scan-csv path))

shape: (3, 2) ┌─────┬──────┐ │ foo ┆ bar  │ │ --- ┆ ---  │ │ i64 ┆ str  │ ╞═════╪══════╡ │ 1   ┆ null │ │ 2   ┆ bak  │ │ 3   ┆ baz  │ └─────┴──────┘

1.4.1.3 Reading options🔗ℹ

Upstream documents read_csv’s options on its reference page; here they are with their Racket spellings. Each keyword keeps Python’s name and works the same on read-csv and scan-csv. The file, "flights.tsv", is 102 rows of nycflights13, tab-separated, with NA for a missing value.

Separator. Read with the default comma, a tab-separated file is one column whose name is the whole header line. Python’s read_csv returns that column; read-csv is stricter and raises, naming the separator to pass. Passing any #:separator, even #\,, turns the check off, and scan-csv does not check.

> (read-csv "flights.tsv")

dataframe-read-csv: flights.tsv reads as one column whose

header splits on #\tab into 19 fields; pass #:separator

#\tab, or #:separator #\, to keep one column

> (shape (read-csv "flights.tsv" #:separator #\,))

'(102 1)

Missing values. Types are inferred from the first 100 rows, and the first NA comes later. #:null-values takes one marker or a list of them:

> (read-csv "flights.tsv" #:separator #\tab)

dataframe-read-csv: failed to read csv from flights.tsv:

could not parse `NA` as dtype `i64` at column 'arr_delay'

(column number 9)

The current offset in the file is 648 bytes.

You might want to try:

- increasing #:infer-schema-length (e.g.

#:infer-schema-length 10000, or #f for every row),

- specifying correct dtype with #:schema-overrides

- setting #:ignore-errors to #t,

- adding `NA` to #:null-values.

Original error: ```invalid primitive value found during CSV

parsing```

> (define flights (read-csv "flights.tsv" #:separator #\tab #:null-values "NA"))

> (~> flights (select "dep_delay" "arr_delay") (tail 3))

shape: (3, 2) ┌───────────┬───────────┐ │ dep_delay ┆ arr_delay │ │ ---       ┆ ---       │ │ i64       ┆ i64       │ ╞═══════════╪═══════════╡ │ -7        ┆ -4        │ │ -5        ┆ null      │ │ null      ┆ null      │ └───────────┴───────────┘

> (~> (read-csv "flights.tsv" #:separator #\tab #:null-values '("NA" "-"))       (ref #:columns "air_time")       null-count)

2

Types. #:schema-overrides fixes a column’s type, in any spelling series accepts; #:infer-schema-length sets how many rows inference reads (#f for all of them, 0 for strings throughout); #:ignore-errors reads what does not parse as null; #:try-parse-dates reads ISO dates and datetimes as such.

> (~> (read-csv "flights.tsv" #:separator #\tab #:null-values "NA"                 #:schema-overrides '(("dep_delay" . f64) ("flight" . int32)))       (select "dep_delay" "flight")       (tail 2))

shape: (2, 2) ┌───────────┬────────┐ │ dep_delay ┆ flight │ │ ---       ┆ ---    │ │ f64       ┆ i32    │ ╞═══════════╪════════╡ │ -5.0      ┆ 4525   │ │ null      ┆ 4308   │ └───────────┴────────┘

> (~> (read-csv "flights.tsv" #:separator #\tab #:infer-schema-length #f)       (ref #:columns "dep_delay")       dtype)

'string

> (~> (read-csv "flights.tsv" #:separator #\tab #:ignore-errors #t)       (ref #:columns "dep_delay")       null-count)

1

> (~> (read-csv "flights.tsv" #:separator #\tab #:null-values "NA"                 #:try-parse-dates #t)       (select "time_hour")       (head 2))

shape: (2, 1) ┌─────────────────────┐ │ time_hour           │ │ ---                 │ │ datetime[μs]        │ ╞═════════════════════╡ │ 2013-01-01 05:00:00 │ │ 2013-01-01 05:00:00 │ └─────────────────────┘

Layout. "notes.csv" starts with a comment line, separates fields with ; and quotes a field that holds one with '; "latin1.csv" is not UTF-8. #:has-header, #:skip-rows and #:n-rows frame the rows to read.

> (read-csv "notes.csv" #:separator #\; #:comment-prefix "#" #:quote-char #\')

shape: (2, 2) ┌─────────┬───────────────┐ │ carrier ┆ note          │ │ ---     ┆ ---           │ │ str     ┆ str           │ ╞═════════╪═══════════════╡ │ UA      ┆ late; weather │ │ AA      ┆ on time       │ └─────────┴───────────────┘

> (read-csv "latin1.csv" #:encoding 'utf8-lossy)

shape: (2, 2) ┌─────────┬──────────┐ │ carrier ┆ name     │ │ ---     ┆ ---      │ │ str     ┆ str      │ ╞═════════╪══════════╡ │ B6      ┆ JetBlue  │ │ ZZ      ┆ Caf� Air │ └─────────┴──────────┘

> (read-csv "parts/part-1.csv" #:has-header #f #:skip-rows 1 #:n-rows 1)

shape: (1, 3) ┌──────────┬──────────┬──────────┐ │ column_1 ┆ column_2 ┆ column_3 │ │ ---      ┆ ---      ┆ ---      │ │ str      ┆ str      ┆ i64      │ ╞══════════╪══════════╪══════════╡ │ EWR      ┆ IAH      ┆ 2        │ └──────────┴──────────┴──────────┘

API gaps: null_values takes no per-column mapping (#101); no columns, new_columns, eol_char, row_index_name, truncate_ragged_lines or decimal_comma.