Hyper-literate programming
1 What is hyper-literate programming?
2 Chunks of code
3 Memorizing and repeating chunks
4 Order of expansion of the program
5 A note on literate programs as subsections of another document
6 Highlighting added, removed and existing parts in literate programs
6.1 Example

Hyper-literate programming

Georges Dupéron

 #lang hyper-literate package: hyper-literate

The hyper-literate metalanguage extends the features of scribble/lp2, with the goal of providing a more modern view on literate programming. It can be parameterized with the language used in the chunks (so that it is possible to directly write typed/racket programs with hyper-literate, for example).

On the first line, which begins with #lang hyper-literate, the language recognises the following options:

#lang hyper-literate lang maybe-no-req maybe-no-auto

  maybe-no-req = 
  | #:no-require-lang
  maybe-no-auto = 
  | #:no-auto-require

where lang is a module name which can be used as a #lang, for example typed/racket or racket/base.

The current implementation of hyper-literate needs to inject a (require lang) in the expanded module, in order to have the arrows properly working in DrRacket for "built-in" identifiers which are provided by the lang itself. The require statement is injected after the whole “code” module has been expanded. It is worth noting that an extra scope is added to the expanded body of the module, in order to make any require form within more specific than the (require lang).

The current implementation of scribble/lp2, on which hyper-literate relies (with a few changes), extracts the require statements from chunks of code, and passes them to (require (for-label )). The goal is to have identifiers from required modules automatically highlighted and hyperlinked to their documentation. However, all meta-levels are smashed into the #f, i.e. for-label meta-level. As a consequence, conflicts can arise at the for-label meta-level between two modules, even if these two modules were originally required at distinct meta-levels in the source program. It is possible in this case to disable the feature using #:no-auto-require, and to manually call (require (for-label )) and handle conflicting identifiers in a more fine-grained way.

NOTE: This #:no-require-lang is deprecated; use , instead. The #:no-require-lang is deprecated starting from version 0.1, and is not needed anymore. It is still accepted for backwards compatibility. Note that version 0.1 of this library requires a fairly recent Racket version to work properly (it needs v. with the commit 8a7852ebbfeb85a8f860700ba5ae481ed7aa9b41, or v. or later). By default, raco will install v0.0 of hyper-literate on older Racket versions.

The extra require statement injected by hyper-literate could in previous versions conflict with user-written require statements. These require statements can shadow some built-ins, and this case would yield conflicts. The #:no-require-lang option disables that behaviour in versions < 0.1, and has the only drawback that built-ins of the lang language do not have an arrow in DrRacket (but they still should be highlighted with -a turquoise background when hovered over with the mouse).

1 What is hyper-literate programming?

Hyper-literate programming is to literate programming exactly what hypertext documents are to regular books and texts. Literate programming is about telling other programmers how the program works (instead of just telling the compiler what it does). Telling this story can be done using non-linear, hyperlinked documents.

For now these utilities only help with manipulating literate programming chunks (e.g. repeating the same chunk in several places in the output document, but keeping a single copy in the source code).

Ultimately, the reading experience should be closer to viewing an interactive presentation, focusing on the parts of the program that are of interest to you: expand on-screen the chunks you are curious about, run some tests and see their result, etc.

2 Chunks of code


(chunk <name> content ...)

Same as chunk from scribble/lp2, with a few tweaks and bug fixes.


(CHUNK <name> content ...)

Same as CHUNK from scribble/lp2, with a few tweaks and bug fixes.

3 Memorizing and repeating chunks


(defck <name> content ...)

Like chunk from scribble/lp2, but remembers the chunk so that it can be re-displayed later using repeat-chunk.


(repeat-chunk <name>)

Shows again a chunk of code previously remembered with defck. If the <name> starts and ends with angle brackets, they are replaced by parentheses to hint that this is not the first occurrence of this chunk, so that the name becomes |(name)|

4 Order of expansion of the program

The file is expanded a first time, in order to identify and aggregate the chunks of code (declared with chunk). Then, the top-level module of the file is constructed using these chunks, and a doc submodule is added, which contains all the surrounding text. The chunks are typeset where they appear using racketblock.

The doc submodule is declared using module*, so that it can use (require (submod "..")) to use functions declared in the chunks. For example, it should be possible to dynamically compute the result of a function, and to insert it into the document, so that the value displayed always matches the implementation.

When the file is expanded for the first time, however, the (submod "..") does not exist yet, and cannot be required. This is the case because the first expansion is performed precisely to extract the chunks and inject them in that module.

To solve this problem, the following macros behave differently depending on whether the code is being expanded for the first time or not (in which case the (submod "..") module can be used).


(if-preexpanding a b)

Expands to a if the code is being pre-expanded, and expands to b if the (submod "..") module can be used.


(when-preexpanding . body)

Expands to (begin . body) if the code is being pre-expanded, and expands to (begin) otherwise.


(unless-preexpanding . body)

Expands to (begin . body) if the (submod "..") module can be used, and expands to (begin) otherwise.

5 A note on literate programs as subsections of another document

To use include-section on hyper-literate programs, a couple of workarounds are required to avoid issues with duplicate tags for identically-named chunks (like <*>, which is likely to always be present).


(chunks-toc-prefix)  (listof string?)

(chunks-toc-prefix prefix-list)  void?
  prefix-list : (listof string?)
We give an example for two files which are part of a hypothetical pkg package:

6 Highlighting added, removed and existing parts in literate programs

 (require hyper-literate/diff1) package: hyper-literate

Highly experimental. Contains bugs, API may change in the future.


(hlite name pat . body)
Like chunk, but highlights parts of the body according to the pattern pat.

The pat should cover the whole body, which can contain multiple expressions. The pat can use the following symbols:

In the following example, the 1 is highlighted as removed (and will not be present in the executable code), the π is highlighted as added, and the rest of the code is dimmed:

#lang hyper-literate #:♦ racket/base
hlite[<my-code> {/ (def args (_ - _ + _ / . _))}
       (define (foo v)
         (+ 1 π . v))]

It produces the result shown below:

(define (foo v)
  (+ 1 π . v))

6.1 Example

You can look at the source code of this document to see how this example is done.

We define the function foo as follows:

(define (foo v)
  (+ 1 v))

However, due to implementation details, we need to add π to this value:

(define (foo v)
  (+ 1 π . v))

In order to optimise the sum of 1 and π, we extract the computation to a global helper constant:

(define π 3.1414592653589795)
(define one-pus-π (+ 1 π))
(define (foo v)
  '(a b c d . e)
  (+ 1 π one-pus-π v))0

'(a b c d . e)
(quote (a b c d . e))
(quote (a b c d . e))
''(a b c d . e)

The whole program is therefore:

1 2 3 4

(x y z)

(0 (x y . z))

(0 ((x x y yy . z)))

(0 ((x x y yy
       . z)))

«*» ::=
(require rackunit)
(check-= (foo 42) (+ 42 1 3.1414592653589795) 0.1)
(check-equal? (list «www»)
              '((a c d . e)
                (a c d . e)
                (a c d . e)
                '(a c d . e)))
(check-equal? '(«aaa») '(2 3 4))
(check-equal? '(0 «bbb» 1) '(0 x z 1))
(check-equal? '«ccc» '(0 x . z))
(check-equal? '«ddd» '(0 x x . z))