ee-lib:   Library Support for DSL Macro Expanders
1 Scope, binding, and hygiene operations for DSL expanders
1.1 Scope
with-scope
scope-tagger?
add-scope
splice-from-scope
syntax-local-introduce-splice
1.2 Binding
bind!
lookup
1.3 Hygiene for expander definitions
define/  hygienic
define/  hygienic-metafunction
1.4 Hygiene for macro application
apply-as-transformer
1.5 Transformer evaluation
eval-transformer
1.6 Integrating with Racket’s expander
current-def-ctx
current-ctx-id
racket-var
racket-var?
1.7 Disappeared uses and bindings
2 Defining literals
define-literal-forms
7.9

ee-lib: Library Support for DSL Macro Expanders

Michael Ballantyne <michael.ballantyne@gmail.com>

This library provides a higher-level API to Racket’s syntax system, designed for implementing macro expanders for DSLs. The paper "Macros for Domain-Specific Languages" serves as the guide-level explanation of the library and associated programming patterns. This page provides reference documentation.

1 Scope, binding, and hygiene operations for DSL expanders

 (require ee-lib) package: ee-lib

Import at phase 1.

1.1 Scope

Intuitively, a scope is a region of program text in which certain bindings are available, determined by scoping forms such as let or block. In the context of macro expansion, syntax may be moved in and out of scopes during the process of expansion, and some scopes aren’t immediately evident in the source program text. The Racket expander uses several kinds of scope values to represent regions of partially expanded programs to implement macro hygiene. This documentation refers to these scope values as scope tags, and reserves the word "scope" for the intuitive notion.

syntax

(with-scope id body ...)

Introduces the elements needed to implement a scope: an outside-edge scope tag, an inside-edge scope tag, and a new binding context segment.

The two scope tags are encapsulated in a scope tagger value accessible via id. The new binding context segment is added to the library local binding context for the dynamic extent of the evaluation of the body forms.

procedure

(scope-tagger? v)  boolean?

  v : any/c
Returns #t if v is a scope tagger created by with-scope and #f otherwise.

procedure

(add-scope stx tagger)  syntax?

  stx : syntax?
  tagger : scope-tagger?
Annotates the stx with the scope tags represented by the tagger.

procedure

(splice-from-scope stx tagger)  syntax?

  stx : syntax?
  tagger : scope-tagger?
Removes the the scope tags represented by the tagger from the stx.

Useful for implementing splicing forms like splicing-let where certain bindings that initially appear to be within a scope in fact splice outside of it.

procedure

(syntax-local-introduce-splice stx)  syntax?

  stx : syntax?
Flips the current macro-introduction scope and removes any use-site scope tags created for the current expansion context.

Useful when moving syntax out of the context of a given macro expansion, as when lifting a definition to a surrounding context.

1.2 Binding

This library implicitly maintains a library local binding context with entries that map bindings to values, similar to the core expander’s local binding context. See current-def-ctx for a way to access the library local binding context as a first-class definition context that can be used with the core expander’s API.

The library local binding context consists of nested binding context segments corresponding to the nested dynamic extents of with-scope uses. The bind! operation adds new entries to the innermost segment. The binding context may be sealed, preventing further bindings within the context until a new segment is added. All operations that use syntax to create or lookup bindings in the binding context first annotate the syntax with the inside-edge scope tag for the scope corresponding to the innermost binding context segment.

procedure

(bind! id v)  identifier?

  id : identifier?
  v : any/c
(bind! ids vs)  (listof identifier?)
  ids : (listof identifier?)
  vs : (listof any/c)
Creates a binding for the given id and extends the current binding context segment with an entry mapping the binding to the value v. The scope set for the binding always includes the current scope’s inside-edge scope tag. Also records a disappeared binding for id.

This operation is legal only in a library local binding context with an unsealed segment.

The second form works like the first, but for lists of corresponding ids and vs.

procedure

(lookup id predicate)  (or/c #f any/c)

  id : identifier?
  predicate : (-> any/c (or/c #f any/c))
Looks for a binding and corresponding entry in the library local binding context for id. If the binding exists, has an value in the library local binding context, and the value satisfies predicate, the value is returned and a disappeared use is recorded for id. Otherwise returns #f.

1.3 Hygiene for expander definitions

As an expander traverses syntax, it needs to enter new expansion contexts and adjust the library local binding context. And hygiene should treat syntax constructed introduced by templates in the expander similarly to syntax introduced by a macro. The following forms define expand functions with these behaviors.

syntax

(define/hygienic (id arg ...) ctx-type body ...)

 
arg = id
     
ctx-type = #:expression
  | #:definition
Defines a function id with positional arguments arg ... and body body. Invocations of the function enter a new expansion context of the type specified the ctx-type: an expression context, or an internal-definition context. Entering an expression context seals the library local binding context; entering an internal-definition context does not.

Invocations of the function are hygienic in the same way macro applications are hygienic: for syntax-valued arguments and returns, arguments are tagged by a fresh use-site scope tag, and syntax returned from the function that was not part of one of the arguments is tagged with a fresh macro-introduction scope tag.

The expansion context type determines the treatment of use-site scope tags at uses of bind, syntax-local-identfier-as-binding, and syntax-local-introduce-splice within. During expansion in an internal-definition context, the expansion context tracks a set of use-site scopes created during expansion of the context. The operations just mentioned remove use-site scopes present in that set. Entering an expression context resets the set to empty.

syntax

(define/hygienic-metafunction (id id ...) ctx-type body ...)

Like define/hygienic, but wrapped as a template metafunction.

1.4 Hygiene for macro application

Expanders also need to apply macro hygiene when invoking macros, via apply-as-transformer. Macros should expand to forms such as define rather than directly extending the binding context with bind!, so apply-as-transformer seals the binding context.

procedure

(apply-as-transformer proc ctx-type arg ...)  any

  proc : procedure?
  ctx-type : (or/c 'expression 'definition)
  arg : any/c
Calls the function proc with the same expansion context and hygiene behavior as calling a function defined by define/hygienic, except that the "library local binding context" is always sealed.

1.5 Transformer evaluation

procedure

(eval-transformer stx)  any/c

  stx : syntax?
Evaluates stx at phase 1 in the library local binding context.

Useful for implementing forms like let-syntax for a DSL.

1.6 Integrating with Racket’s expander

When a DSL’s syntax has Racket subexpression positions, the DSL expander needs to call the Racket expander via local-expand. The following operations help connect the Racket expansion with the library-managed binding context.

Returns a internal-definition context value corresponding to the current library local binding context, or #f when the library local binding context corresponds to the core local binding context. The return value is suitable as the intdef-ctx argument to local-expand.

procedure

(current-ctx-id)  symbol?

Returns a value suitable for use as the context-v argument to local-expand for internal-definition context expansion.

procedure

(racket-var)  racket-var?

procedure

(racket-var? v)  boolean?

  v : any/c
DSL syntax may bind variables that should be accessible from Racket syntax. Use the racket-var datatype to represent Racket variable bindings in the library local binding context via bind!.

1.7 Disappeared uses and bindings

For DrRacket to provide behaviors such as binding arrows, it needs to know which identifiers act as bindings and references. In the case of DSL code these identifiers may not be present as bindings and references in the program’s compilation to Racket. The bind! and lookup operations automatically record such disappeared uses and disappeared bindings.

DrRacket looks for information about disappeared uses and bindings on syntax in fully-expanded modules (see Syntax Properties that Check Syntax Looks For). This library inserts extra syntax without runtime meaning into the expanded module to carry this information.

2 Defining literals

 (require ee-lib/define) package: ee-lib

Import at phase 0.

syntax

(define-literal-forms literal-set-id message (form-id ...))

Binds each form-id as a macro that raises a syntax error with message message. Binds literal-set-id as a literal set containing the form-ids.