Rash: The Reckless Racket Shell🔗ℹ

William Hatch <william@hatch.uno>

Rash, adj 1. Hurrying into action or assertion without due caution and regardless of prudence, hasty, reckless, precipitate. “A rash programmer is likely to produce shell scripts.”

1 Stability🔗ℹ

Some things in Rash are not entirely stable. However, things from the GPCE paper are all stable. If any of them don’t work at any point it’s a bug. Unstable things should all be labelled in the documentation.

Rash is definitely usable as an interactive shell/repl as well as for writing scripts, and I’m anxious to hear feedback from more users to understand what they like and what can be improved.

2 Rash Guide🔗ℹ

Rash is a shell language embedded in Racket. It has a concrete syntax that is amenable to quick and easy interactions without lots of punctuation overhead. It aims to allow shell-style interaction and programming to be freely mixed with more general-purpose Racket code. Like shells you can use it as a day-to-day interface for computing, run programs, wire processes together, etc. You can copy interactive code into a file and have a working script. Then you can generalize from there. However, you have access to all of Racket, its data structures, its libraries, its macro system, and its other DSLs as you make your scripts more general. Also it allows shell-style process wrangling to be mixed with Racket code, pipelines of functions that pass objects, and much more. You can gradually move between shell-style Rash code and more normal and general Racket code in different parts of a script, or throw verbatim interactions directly into existing programs as you explore the solution space.

Here follows a quick overview that assumes some knowledge of shell languages like Bash as well as Racket.

#!/usr/bin/env racket

#lang rash

cd project-directory

echo The number of Racket files in this directory:

ls *.rkt | wc -l

;; This returns the hostname as a string in all caps

cat /etc/hostname |> port->string |> string-upcase

Pipelines pass objects. When a process pipeline segment is next to a function pipeline segment, it passes a port to the function segment. When a function segment is followed by a process segment, the return value of the function segment is printed to the stdin of the process.

The |>> operator is like the |> operator except that if it receives a port it converts it to a string automatically. So the above example can be simplified as:

cat /etc/hostname |>> string-upcase

You can also make pipelines composed entirely of Racket functions.

|> directory-list |> map file->string

Pipelines always start with an operator, and if none is specified the default-pipeline-starter is inserted. Pipeline operators are user-definable with define-pipeline-operator. Defining new operators can help make common patterns shorter, simpler, and flatter. For instance the =map= operator wraps the map function, allowing you to specify just the body of a lambda.

;; =map= is in the demo file still,

;; in the rash-demos package

(require rash/demo/setup)

;; These two are the same.

|> list 1 2 3 |> map (λ (x) (+ 2 x))

|> list 1 2 3 =map= + 2

Pipeline operators are macros, and therefore can play any of the tricks that macros generally can in Racket. The | operator can auto-quote symbols, turn asterisks into glob expansion code, etc. The |> operator can detect whether the current-pipeline-argument is used and insert it automatically.

If you put parentheses in the middle of a pipeline, you escape to normal Racket code.

;; This will either show hidden files or give a long listing

ls (if (even? (random 2)) '-l '-a)

;; in-dir is in the demo file still

(require rash/demo/setup)

in-dir $HOME/project {

make clean

}

For instance, in-dir executes code with current-directory parameterized based on its first argument. Note that logical rash lines don’t necessarily line-up with physical lines. Newlines can be escaped with a backslash, commented out with multiline comments, and if they are inside parentheses or braces they are handled by the recursive read.

echo This is \

all #|

|# one (string-append

"logical"

"line")

Braces trigger a recursive line-mode read. They are available in line-mode as well as in the embedded s-expression mode. So they can be used to create blocks in line-mode as with the above in-dir example, or be used to escape from the s-expression world to line-mode.

Note that subprocess pipelines are connected to stdout and stdin in the REPL and at the top level of #lang rash. The most common thing you want when embedding some Rash code is for subprocess output to be converted to a string. Using #{} switches to line-mode with defaults changed so that subprocess output is converted to a string and passed through string-trim.

;; #%hash-braces is provided by the demo library right now...

(require rash/demo/setup)

;; I do this a lot when I don't remember what a script does

cat #{which my-script.rkt}

TODO - how to more generally parameterize such settings.

Every line in Rash is actually a line-macro. If a line does not start with a line-macro name explicitly, #%linea-default-line-macro is inserted. By default this is run-pipeline in Rash.

TODO - actually the default is run-pipeline/logic, which I haven’t documented yet, which adds && and ||.

You can also write normal parenthesized Racket code. If the first (non-whitespace) character on a line is an open parenthesis, the line is read as normal Racket code and no line-macro is inserted.

(define flag '-l)

ls $flag

Note that practically all Racket code starts with an open-paren, so Rash is almost a superset of normal Racket. The only thing lost is top-level non-parenthesized code, which is really only useful to see the value of a variable. Most programs in #lang racket/base could be switched to #lang rash and still function identically, and I never launch the Racket repl anymore because rash-repl is both a shell and a full Racket repl.

(define x 1234)

;; Now let's see the value of x.

;; We can't just write `x`, but we can do any of these:

(values x)

|> values x

echo $x

(require rash/demo/setup)

val x

Avoiding line-macros by starting with a paren causes an annoying inconsistency – you can’t have a line-macro auto-inserted if the first argument to the line macro is a parenthesized form.

;; We want to choose a compiler at runtime.

run-pipeline (if use-clang? 'clang 'gcc) -o prog prog.c

;; If we leave off the line-macro name, it will not be inserted

;; because the line starts with a parenthesis.

;; This will probably cause an error!

(if use-clang? 'clang 'gcc) -o prog prog.c

This problem can be fixed by prepending the line-macro name or by using [square] brackets instead of parentheses. The issue doesn’t come up much in practice, and it’s a small sacrifice for the convenience of having both normal Racket s-expressions and Rash lines.

Rash is primarily a combination of two libraries – Linea: line oriented reader, which will explain the details of the line-oriented concrete syntax, and the Pipeline Macro Library, which will explain the details of the run-pipeline macro and pipeline operators. You should read their documentation as well if you want a more thorough understanding of Rash.

What else belongs in a quick overview? Pipelines with failure codes don’t fail silently – they raise exceptions. More fine-grained behavior can be configured per-pipeline, or using aliases (eg. whether other exit codes besides 0 are successful, whether to check for success in subprocesses in the middle of a pipeline, etc). There are probably more things I should say. But probably the best way forward from here is to read the run-pipeline macro documentation.

You can define aliases with define-pipeline-alias or define-simple-pipeline-alias. Aliases are currently supported only by =unix-pipe=, though I may change that. Aliases basically bypass the pipe itself – basically so you can have =unix-pipe= be the default operator but have a set of key-words that bypass it for a different operator without having to write the operator when it’s in starting position. And to be able to define aliases that are a little more familiar to people.

You can access environment variables with getenv and putenv, or by accessing the current-environment-variables parameter. Individual pipelines should have some sugar to set environment variables more conveniently, but I haven’t added that yet. Also, the dollar escapes done by =unix-pipe= access environment variables instead of normal lexical variables if you use a variable name in ALL CAPS.

Also, for those who just want to pipeline subprocesses in a Racket program using a lispy syntax, you probably want the shell/pipeline library, a basic component of Rash that can be used on its own: Basic Unix-style Pipelines

While you can use Rash as a #lang, you can also just use the bindings it exports via (require rash). Note that requiring the rash module will not affect the reader, so the line-oriented syntax will not be available unless you take steps to use it too (IE use the Linea reader, or use the rash macro).

Here are some miscellaneous little sections I am adding off-the-cuff based on discussions until I rewrite the whole Rash Guide to be better.

2.1 quoting globs🔗ℹ

How do you stop globs and dollars from expanding without putting them in parenthesized forms? Quote them.

;; print "*.rkt" literally

echo '*.rkt

;; print "$HOME" literally

echo '"$HOME"

Globs desugar to a use of the glob function.

TODO - I need a better documentation section on globbing generally.

2.2 Rash’s names are terrible, how do I rename them?🔗ℹ

Line macros (like cd), pipeline operators (like \|>), and pipeline option flags (like &> and &bg) are all macros, so you can rename them with make-rename-transformer:

(define-syntax > (make-rename-transformer #'&>!))

2.3 How does the underscore work?🔗ℹ

The _ identifier is short for current-pipeline-argument. It is actually a syntax parameter that pipeline operators can make mean whatever they want. So there isn’t one single thing that it means. But there is a convention.

Probably the best way to explain _ is to start with the most basic pipeline operator that uses it, which is called =basic-object-pipe/expression=. The name is terrible, and it should probably have a short name like \| and \|> do. But I haven’t decided what yet. Suggestions appreciated.

=bop/e= takes an expression that (probably) contains _. And it uses it as the body of a function where _ is the argument. Unless it is in the first part of the pipeline, in which case _ is an error. Actually, it’s not a bad operator in start position – it is like cat for racket values.

=basic-object-pipe=, or \|>, basically unwraps a layer of parentheses and lets _ be implicit when it is at the end of the pipeline. So the following two pipelines are the same as the above.

=bop/e= 5 |> + 6 _

=bop/e= 5 |> + 6

But you can also explicitly place the underscore elsewhere.

=bop/e= 5 |> + _ 6

Then =object-pipe= (or \|>>) is the same, except that it automatically detects ports and turns them into strings.

Other operators like =map= and =filter= place _ like \|> does, but instead of standing for the whole object received from the previous pipeline stage, it stands for an element of the object received (which must be a list).

3 Media🔗ℹ

A preprint of an academic paper about Rash is available here. It is much better documentation than the Rash guide, currently.

I made a quick demo recording of an interactive repl that is on the project website.

Also I gave a talk at RacketCon 2017 about it, which can be viewed here, though it is outdated. There have been various changes since the talk was given, but the core ideas are the same. The biggest change since then is that embedding the line-syntax is encouraged with braces in the Linea syntax rather than string embedding.

4 Rash Reference🔗ℹ

4.1 shell-pipeline re-exports🔗ℹ

Note that all the pipeline things (run-pipeline, =unix-pipe=, =object-pipe=, define-pipeline-operator, etc) are documented in the Pipeline Macro Library module.

But since those functions/macros are essential to using Rash, I’ll also list them here:

Pipeline operators:

• =composite-pipe=

• =basic-unix-pipe=

• =unix-pipe=

• \|

• =basic-object-pipe/expression=

• =basic-object-pipe/form=

• =basic-object-pipe=

• \|>

• =object-pipe/expression=

• =object-pipe/form=

• =object-pipe=

• \|>>

• default-pipeline-starter

Defining pipeline operators:

• define-pipeline-operator

• define-pipeline-alias

• define-simple-pipeline-alias

• current-pipeline-argument

• _

• expand-pipeline-arguments

Pipeline flags:

• &bg

• &pipeline-ret

• &in

• &<

• &out

• &>

• &>!

• &>>

• &err

• &strict

• &permissive

• &lazy

• &lazy-timeout

Pipeline implicit option configuration (note that these are subsumed by with-rash-config):

• with-pipeline-config

• splicing-with-pipeline-config

4.2 linea re-exports🔗ℹ

All the things about reading and line-macros (define-line-macro, #%linea-line, etc) are documented in the Linea: line oriented reader module.

But here is an export list, so it’s visible in this page:

From linea/defaults:

• #%linea-expressions-begin

• #%linea-line

• #%linea-s-exp

• #%linea-default-line-macro

From linea/line-macro (note that with-default-line-macro is subsumed by with-rash-config):

• define-line-macro

• default-line-macro

• with-default-line-macro

• splicing-with-default-line-macro

4.3 Rash original exports🔗ℹ

TODO: document forms for configuring Rash besides the rash macro, document forms for creating customized versions of #lang rash (including reader modifications, default line-macro and pipeline operator, bindings available...), etc

Options:

#:out sets the default output port or transformer for unix pipelines (as run by run-pipeline). The default runs port->string on the output. Note that a custom output transformer should close the received port!

#:in sets the default input port for unix pipelines (as run by run-pipeline). The default is an empty port.

#:err sets the default error port for unix pipelines (as run by run-pipeline). The default is to turn the errors into a string that is put into the exception generated by an error in the pipeline.

#:starter sets the default starting pipeline operator for run-pipeline when one is not explicitly given in the pipeline. The default is =unix-pipe=.

#:line-macro sets the default line-macro for lines that don’t explicitly have one. The default is the run-pipeline line-macro.

Note that in, out, and err are evaluated once for each pipeline run in the body.

Options:

The options are the same as with-rash-config.

TODO - options for changing the reader, etc.

Note that the input/output/error-output have different defaults for the rash macro than for the #lang or repl.

Unstable. I don’t want to commit to this, yet.

(define-syntax my-rash

(make-rash-transformer #:starter =basic-object-pipe=))

Note that the expressions given for default input/output/error ports are evaluated for each pipeline, and so should either be function applications that produce consistent ports (eg. current-input-port) or some expression that gives a fresh port that you want each time.

Unstable. I don’t want to commit to this, yet.

Note that the default #lang rash has its input/output/error-output as stdin/stdout/stderr, which is different than the rash macro.

If no argument is given, it changes to the user’s home directory.

Unstable. The current version does dollar and tilde expansion, and the way that works may change in the future. Additionally, I may add support for things like cd - a la bash and friends, as well as more general directiory history tracking. But I’m not really sure yet. But basic cd my-directory type stuff will definitely keep working the same.

The default takes the result out of terminated pipelines to print rather than the pipeline object itself. I plan to change the default. But the idea of having this parameter is that you can set up your repl to print things in a more useful way, and then have it print the same way in a #lang rash script.

Unstable: the way things are printed may change.

5 Interactive Use🔗ℹ

You can run the repl by running racket -l rash/repl. An executable named rash-repl is installed in Racket’s bin directory, so if you have it on your path you can run rash-repl instead.

Unstable: Various details of the repl will change over time. But that basically just means the repl will get cooler over time. The biggest change to come is that at some point the line-editor will be swapped out.

Note that in the repl the default input/output/error-output are to the stdin/out/err connected to the terminal unless you change them. This is different than the rash macro, and allows you to do things like run curses programs that have access to terminal ports.

The repl can be customized with rc files. First, if $HOME/.config/rash/rashrc.rkt exists, it is required at the top level of the REPL. Then, if $HOME/.config/rash/rashrc (note the lack of .rkt) exists, it is evaluated at the top level more or less as if typed in (much like rc files for bash and friends). Note that rashrc.rkt files are modules, can be in any #lang, and get all the normal and good compilation guarantees that Racket modules enjoy. rashrc is NOT a module, and gets none of them. rashrc is mainly there to let you muck up the namespace that you use interactively. Prefer rashrc.rkt.

Actually, the rc files are found using list-config-files. You can read its docs for specifics, but the gist is that it depends on the value of the XDG_CONFIG_HOME and XDG_CONFIG_DIRS environment variables. $HOME/.config/rash/rashrc is the default on Unix, while the default is %LOCALAPPDATA%/rash/rashrc on Windows.

Note that “the top-level is hopeless”. This applies to the Rash REPL as well as rashrc files. But the hopelessness is mostly to do with defining macros, particularly complicated macros such as mutually recursive or macro-defining macros. So the hopelessness doesn’t affect the types of things most people are likely to do in a shell. But if you don’t remember that, you might put “hopeless” things in a rashrc file. Don’t. Put it in a module like rashrc.rkt (or any other module or library you make). (For more hopelessness, see this.)

A few nice things (like stderr highlighting) are in a demo-rc file you can require. To do so, add this to $HOME/.config/rash/rashrc:

(Rash actually follows the XDG basedir standard – you can have rashrc.rkt or rashrc files in any directory of $XDG_CONFIG_HOME or $XDG_CONFIG_DIRS, and the rash repl will load all of them)

5.1 Unicode and garbled glyphs🔗ℹ

The repl uses the readline module for line-editing and completion. The readline module by default uses libedit instead of the actual libreadline for licensing reasons. Libedit doesn’t seem to handle unicode properly. Installing the readline-gpl package fixes that (raco pkg install readline-gpl). Note that the readline-gpl Racket package needs a libreadline shared library to be installed on your system, so you may need to install a libreadline package using your system package manager. For example, on Debian-based distributions you can install by running sudo apt install –yes libreadline-dev.

5.2 Interactive functions (unstable)🔗ℹ

All the following repl functions are not stable.

Unstable.

Unstable.

The given function’s arity is tested to see if it receives various keywords, so that the protocol can be extended and the user only has to specify the keywords that are needed for a given prompt.

Keywords optionally given:

#:last-return-value - fairly self explanatory. If multiple values were returned, they will be given as a list. This will be (void) for the prompt before the first command. The default prompt function formats the return value with current-rash-top-level-print-formatter before printing it.

#:last-return-index - This increments once for every command run in the repl. It will be 0 for the prompt before the first command. This is the index that can be used for result-n. The default prompt function prints the number of the result before printing the result itself.

#:last-command-duration - This parameter contains the time the last command took to execute in milliseconds. This is 0 for the prompt before the first command.

Unstable. I may change how this works. But probably it actually is stable, I just don’t want to commit to it yet, especially given that the entire line editor will eventually change.

Note that when readline is active, you probably want to use the function readline-prompt in the prompt function, because the readline library itself wants to print something. Eg.

(require readline/pread)

(define (a-prompt #:last-return-value [last-ret #f])

(printf "~v " last-ret)

(readline-prompt #">"))

(current-prompt-function a-prompt)

6 Prompt helpers🔗ℹ

There are currently a few functions that can help you design a custom prompt. Currently, they support things like changing foreground/background color and underlined text and getting git information, and they will be expanded in the future to include more useful ways of getting information for your prompt.

Unstable.

6.1 Styling Strings🔗ℹ

These are some usefull functions for styling strings.

You can use them with (require rash/prompt-helpers/string-style)

The values given for foreground and background are treated the same. If a string is given, it must either be the name of a 4 bit color, e.g. "red" or "bright red" for the high-intensity version, or start with the charachter "#" and represent a hexidecimal color code, e.g. "#f2e455" or "#fff". If a number is given, it is treated as a color code for a 8bit/256color color. If a list is given, it is treated as an RGB value for a color, e.g. '(0 0 0) means black. An object with red, green, and blue methods (like color%) can also be given, and are treated like RGB values. See this stack overflow answer for more information on ansi colors.

The values bold?, italic?, and underlined? do what you’d expect them to.

If reset-before? is #t, then the ANSI escape sequence "\033[0m" will be added to the front of the string. When displayed, it has the effect of clearing all styles set by ANSI escape sequences before itself. Similarily, if reset-after? is #t, the sequence "\033[0m" is appended to the end of the string.

The string custom-commands is placed right before to-style, so any styles in the form of ANSI escape sequences can be added overridden if desired.

If create-function? is #t, then a function is returned instead of a string. This function takes 1 argument and places that argument where to-style would have went, and returns the resulting string.

I cant display the actual colors here, but you can copy the resulting strings into a terminal and print them with echo "string" or display them using display in a terminal window.

If reset-before? is #t for a struct (within its style hash), that struct will ignore the styles provided by an outer struct. If reset-customs? is #t for a struct, that struct will ignore custom commands given by its outer struct and only use it’s own.

6.2 Git functions🔗ℹ

There are also some useful functions that help gather git information.

Remember – these functions are unstable, and may change.

You can use them with (require rash/prompt-helpers/git-info)

If the timeout is reached before all information is gathered, the hash is returned with only those elements that were completed.

7 Demo stuff reference🔗ℹ

Nothing in the demo directory is remotely stable! It can all change or go away at any moment. The stuff documented here is in the rash-demos package. Maybe I should move this documentation into that package...

I’ve written various pipeline operators and line macros that I use, but I haven’t decided what should be in the default language yet. So for now they are sitting in a demo directory. But I need some examples. So here I’m documenting a bit.

Use it with (require rash/demo/setup).

Eg.

The _ argument is appended to =map=’s argument list if it is not written explicitly.

Unstable in that it will probably be moved into the default rash module rather than the demo module.

Eg.

The _ argument is appended to =filter=’s argument list if it is not written explicitly.

Unstable in that it will probably be moved into the default rash module rather than the demo module.

Eg.

in-dir $HOME/projects/* {

make clean

make

}

This is unstable in that the way it does globbing and dollar expansion may change, the way it handles globs that resolve to something that is not a directory may change, and it will probably be moved into the default rash module rather than the demo directory.

8 Code and License🔗ℹ

The code is available on github.

This library is licensed under the terms of the MIT license and the Apache version 2.0 license, at your option.