Rash: The Reckless Racket Shell

William Hatch <[email protected]>

Rash, adj 1. Hurrying into action or assertion without due caution and regardless of prudence, hasty, reckless, precipitate. “A rash programmer is likely to produce shell scripts.”

1 Stability

Rash is not stable.

But it’s getting closer, and it’s definitely usable as an interactive shell/repl.

2 Rash Guide

Rash is a shell language embedded in Racket. It has a concrete syntax that is amenable to quick and easy interactions without lots of punctuation overhead. It aims to allow shell-style interaction and programming to be freely mixed with more general-purpose Racket code. Like shells you can use it as a day-to-day interface for computing, run programs, wire processes together, etc. You can copy interactive code into a file and have a working script. Then you can generalize from there. However, you have access to all of Racket, its data structures, its libraries, its macro system, and its other DSLs as you make your scripts more general. Also it allows shell-style process wrangling to be mixed with Racket code, pipelines of functions that pass objects, and much more. You can gradually move between shell-style Rash code and more normal and general Racket code in different parts of a script, or throw verbatim interactions directly into existing programs as you explore the solution space.

Here follows a quick overview that assumes some knowledge of shell languages like Bash as well as Racket.

#lang rash

cd project-directory

echo How many Racket files do we have?

ls *.rkt | wc -l

You can use Racket functions in pipelines.

;; This returns the hostname as a string in all caps

cat /etc/hostname |> port->string |> string-upcase

Pipelines pass objects. When a process pipeline segment is next to a function pipeline segment, it passes a port to the function segment. When a function segment is followed by a process segment, the return value of the function segment is printed to the stdin of the process.

The |>> operator is like the |> operator except that if it receives a port it converts it to a string automatically. So the above example can be simplified as:

cat /etc/hostname |>> string-upcase

You can also make pipelines composed entirely of Racket functions.

|> directory-list |> map delete-file

Pipelines always start with an operator, and if none is specified the default-pipeline-starter is inserted. Pipeline operators are user-definable with define-pipeline-operator. Defining new operators can help make common patterns shorter, simpler, and flatter. For instance the =map= operator wraps the map function, allowing you to specify just the body of a lambda.

;; =map= is in the demo file still

(require rash/demo/setup)

;; These two are the same.

|> list 1 2 3 |> map (λ (x) + 2 x)

|> list 1 2 3 =map= + 2

Pipeline operators are macros, and therefore can play any of the tricks that macros generally can in Racket. The | operator can auto-quote symbols, turn asterisks into glob expansion code, etc. The |> operator can detect whether the current-pipeline-argument is used and insert it automatically.

If you put parentheses in the middle of a pipeline, you escape to normal Racket code.

;; This will either show hidden files or give a long listing

ls (if (even? (random 2)) '-l '-a)

;; in-dir is in the demo file still

(require rash/demo/setup)

in-dir $HOME/project {

make clean

}

For instance, in-dir executes code with current-directory parameterized based on its first argument. Note that logical rash lines don’t necessarily line-up with physical lines. Newlines can be escaped with a backslash, commented out with multiline comments, and if they are inside parentheses or braces they are handled by the recursive read.

echo This is \

all #|

|# one (string-append

"logical"

"line")

Braces trigger a recursive line-mode read. They are available in line-mode as well as in the embedded s-expression mode. So they can be used to create blocks in line-mode as with the above in-dir example, or be used to escape from the s-expression world to line-mode.

Note that subprocess pipelines are connected to stdout and stdin in the REPL and at the top level of #lang rash. The most common thing you want when embedding some Rash code is for subprocess output to be converted to a string. Using #{} switches to line-mode with defaults changed so that subprocess output is converted to a string and passed through string-trim.

;; #%hash-braces is provided by the demo library right now...

(require rash/demo/setup)

;; I do this a lot when I don't remember what a script does

cat #{which my-script.rkt}

TODO - how to more generally parameterize such settings.

Every line in Rash is actually a line-macro. If a line does not start with a line-macro name explicitly, default-line-macro is inserted. By default this is run-pipeline in Rash.

TODO - actually the default is run-pipeline/logic, which I haven’t documented yet, which adds && and ||.

You can also write normal parenthesized Racket code. If the first (non-whitespace) character on a line is an open parenthesis, the line is read as normal Racket code and no line-macro is inserted.

(define flag '-l)

ls $flag

Note that practically all Racket code starts with an open-paren, so Rash is almost a superset of normal Racket. The only thing lost is top-level non-parenthesized code, which is really only useful to see the value of a variable. Most programs in #lang racket/base could be switched to #lang rash and still function identically, and I never launch the Racket repl anymore because rash-repl is both a shell and a full Racket repl.

(define x 1234)

;; Now let's see the value of x.

;; We can't just write `x`, but we can do any of these:

(values x)

|> values x

echo $x

(require rash/demo/setup)

val x

Avoiding line-macros by starting with a paren causes an annoying inconsistency – you can’t have a line-macro auto-inserted if the first argument to the line macro is a parenthesized form.

;; We want to choose a compiler at runtime.

run-pipeline (if use-clang? 'clang 'gcc) -o prog prog.c

;; If we leave off the line-macro name, it will not be inserted

;; because the line starts with a parenthesis.

;; This will probably cause an error!

(if use-clang? 'clang 'gcc) -o prog prog.c

This problem can be fixed by prepending the line-macro name or by using [square] brackets instead of parentheses. The issue doesn’t come up much in practice, and it’s a small sacrifice for the convenience of having both normal Racket s-expressions and Rash lines.

Rash is primarily a combination of two libraries – Linea: line oriented reader, which will explain the details of the line-oriented concrete syntax, and the Pipeline Macro Library, which will explain the details of the run-pipeline macro and pipeline operators. You should read their documentation as well if you want a more thorough understanding or Rash.

What else belongs in a quick overview? Pipelines with failure codes don’t fail silently – they raise exceptions. More fine-grained behavior can be configured per-pipeline, or using aliases (eg. whether other exit codes besides 0 are successful, whether to check for success in subprocesses in the middle of a pipeline, etc). There are probably more things I should say. But probably the best way forward from here is to read the run-pipeline macro documentation.

You can define aliases with define-pipeline-alias or define-simple-pipeline-alias. Aliases are currently supported only by =unix-pipe=, though I may change that. Aliases basically bypass the pipe itself – basically so you can have =unix-pipe= be the default operator but have a set of key-words that bypass it for a different operator without having to write the operator when it’s in starting position. And to be able to define aliases that are a little more familiar to people.

You can access environment variables with getenv and putenv, or by accessing the current-environment-variables parameter. Individual pipelines should have some sugar to set environment variables more conveniently, but I haven’t added that yet. Also, the dollar escapes done by =unix-pipe= access environment variables instead of normal lexical variables if you use a variable name in ALL CAPS.

Also, for those who just want to pipeline subprocesses in a Racket program using a lispy syntax, you probably want the shell/pipeline library, a basic component of Rash that can be used on its own: Basic Unix-style Pipelines

While you can use Rash as a #lang, you can also just use the bindings it exports via (require rash). Note that requiring the rash module will not affect the reader, so the line-oriented syntax will not be available unless you take steps to use it too (IE use the Linea reader, or use the rash macro).

3 Media

I made a quick demo recording of an interactive repl here. It’s a little out of date. I should make a new and better one.

Also I gave a talk at RacketCon 2017 about it, which can be viewed here. There have been various changes since the talk was given, but the core ideas are the same. The biggest change since then is that embedding the line-syntax is encouraged with braces in the Linea syntax rather than string embedding.

4 Rash Reference

Note that all the pipeline things (run-pipeline, =unix-pipe=, =object-pipe=, define-pipeline-operator, etc) are documented in the Pipeline Macro Library module.

All the things about reading and line-macros (define-line-macro, #%linea-line, etc) are documented in the Linea: line oriented reader module.

TODO: document forms for configuring Rash besides the rash macro, document forms for creating customized versions of #lang rash (including reader modifications, default line-macro and pipeline operator, bindings available...), etc

Options:

#:out sets the default output port or transformer for unix pipelines (as run by run-pipeline). The default runs port->string on the output.

#:in sets the default input port for unix pipelines (as run by run-pipeline). The default is an empty port.

#:err sets the default error port for unix pipelines (as run by run-pipeline). The default is to turn the errors into a string that is put into the exception generated by an error in the pipeline.

#:starter sets the default starting pipeline operator for run-pipeline when one is not explicitly given in the pipeline. The default is =unix-pipe=.

#:line-macro sets the default line-macro for lines that don’t explicitly have one. The default is the run-pipeline line-macro.

Note that in, out, and err are evaluated once for each pipeline run in the body.

Options:

The options are the same as with-rash-parameters.

TODO - options for changing the reader, etc.

Note that the input/output/error-output have different defaults for the rash macro than for the #lang or repl.

This takes all the same options as rash, but doesn’t take a code string. It produces a transformer like rash, but with different default values for the available options.

(define-syntax my-rash (make-rash-transformer #:starter =basic-object-pipe=))

Note that the default #lang rash has its input/output/error-output as stdin/stdout/stderr, which is different than the rash macro.

If no argument is given, it changes to the user’s home directory.

Eventually this will be replaced by a better version, but it should be backwards compatible.

The default takes the result out of terminated pipelines to print rather than the pipeline object itself. I plan to change the default. But the idea of having this parameter is that you can set up your repl to print things in a more useful way, and then have it print the same way in a #lang rash script.

5 Interactive Use

You can run the repl by running racket -l rash/repl. An executable named rash-repl is installed in Racket’s bin directory, so if you have it on your path you can run rash-repl instead.

Various details of the repl will change over time.

Note that in the repl the default input/output/error-output are to the stdin/out/err connected to the terminal unless you change them. This is different than the rash macro, and allows you to do things like run curses programs that have access to terminal ports.

The repl can be customized with rc files. First, if $HOME/.config/rash/rashrc.rkt exists, it is required at the top level of the REPL. Then, if $HOME/.config/rash/rashrc (note the lack of .rkt) exists, it is evaluated at the top level more or less as if typed in (much like rc files for bash and friends). Note that rashrc.rkt files are modules, can be in any #lang, and get all the normal and good compilation guarantees that Racket modules enjoy. rashrc is NOT a module, and gets none of them. rashrc is mainly there to let you muck up the namespace that you use interactively. Prefer rashrc.rkt.

Note that “the top-level is hopeless”. This applies to the Rash REPL as well as rashrc files. But the hopelessness is mostly to do with defining macros, particularly complicated macros such as mutually recursive or macro-defining macros. So the hopelessness doesn’t affect the types of things most people are likely to do in a shell. But if you don’t remember that, you might put “hopeless” things in a rashrc file. Don’t. Put it in a module like rashrc.rkt (or any other module or library you make). (For more hopelessness, see this.)

A few nice things (like stderr highlighting) are in a demo-rc file you can require. To do so, add this to $HOME/.config/rash/rashrc:

(Rash actually follows the XDG basedir standard – you can have rashrc.rkt or rashrc files in any directory of $XDG_CONFIG_HOME or $XDG_CONFIG_DIRS, and the rash repl will load all of them)

The repl uses the readline module for line-editing and completion. The readline module by default uses libedit (or something like that) instead of the actual libreadline for licensing reasons. Libedit doesn’t seem to handle unicode properly. Installing the readline-gpl package fixes that (raco pkg install readline-gpl).

All the following repl functions are not stable.

The given function’s arity is tested to see if it receives various keywords, so that the protocol can be extended and the user only has to specify the keywords that are needed for a given prompt.

Keywords optionally given:

#:last-return-value - fairly self explanatory. If multiple values were returned, they will be given as a list. This will be (void) for the prompt before the first command. The default prompt function formats the return value with current-rash-top-level-print-formatter before printing it.

#:last-return-index - This increments once for every command run in the repl. It will be 0 for the prompt before the first command. This is the index that can be used for result-n and return-n. The default prompt function prints the number of the result before printing the result itself.

6 Demo stuff reference

I’ve written various pipeline operators and line macros that I use, but I haven’t decided what should be in the default language yet. So for now they are sitting in a demo directory. But I need some examples. So here I’m documenting a bit.

Use it with (require rash/demo/setup).

Eg.

The _ argument is appended to =map=’s argument list if it is not written explicitly.

Eg.

The _ argument is appended to =filter=’s argument list if it is not written explicitly.

Eg.

in-dir $HOME/projects/* {

make clean

make

}

7 Code and License

The code is available on github.

This library is licensed under the terms of the LGPL version 3, or (at your option) any later version published by the Free Software Foundation (IE LGPL3+).