2.4.1 Transformer Bindings🔗ℹ

In a top-level or module context, when the expander encounters a form like expr.macro or meta.bridge, the binding that it introduces for the defined names is a macro binding. The value of the binding exists at expansion time, rather than run time (though the two times can overlap), though the binding itself is introduced with phase level 0 (i.e., in the base environment).

The value for the binding is obtained by evaluating the expression on the right-hand side of a meta.bridge form or a similar implied function form for a macro definer like expr.macro. This expression must be parsed before it can be evaluated, and it is parsed at phase level 1 (i.e., in the transformer environment) instead of phase level 0. The result is further wrapped with scope operations to produce a transformer function that can be recorded in a binding and called as described in Expand Steps.

The scope-introduction process for macro expansion (described in Expand Steps) helps keep binding in an expanded program consistent with the lexical structure of the source program. For example, the expanded form of the program

def x = 12

macro 'm $id':

'block:

def x = 10

$id'

m x

is almost equivalent to the expansion of

def x = 12

block:

def x = 10

x

However, the result of the expression produced by m is 12, not 10. The reason is that the transformer bound to m introduces the binding x, but the referencing x is present in the use of the macro. The introduced x is left with one fresh macro scope, while the reference x has a different fresh scope, so the binding x is the one outside the block form.

A use-site scope on a binding identifier is ignored when the definition is in the same context where the use-site scope was introduced. This special treatment of use-site scopes allows a macro to expand to a visible definition. For example, the expanded form of the program

defn.macro 'm $id':

'def $id = 5'

m x

x

is

def x = 5

x

where the x in the def form had a use-site scope that is not present on the final x. The final x nevertheless refers to the definition, because the use-site scope is removed before installing the definition’s binding. In contrast, the expansion of

defn.macro 'm $id':

'block:

def x = 4

def $id = 5

x'

m x

is

block:

def x = 5

def x = 5

x

where the second def x has a use-site scope that prevents it from binding the final x. The use-site scope is not ignored in this case, because the binding is not part of the definition context where m x was expanded.

In addition to using scopes to track introduced identifiers, the expander tracks the expansion history of a form through syntax properties such as #'origin. See Syntax Tracking for more information.

2.4.2 Local Binding Context🔗ℹ

Although the binding of an identifier can be uniquely determined from the combination of its lexical information and the global binding table, the expander also maintains a local binding context that records additional information about local bindings to ensure they are not used outside of the lexical region in which they are bound.

Due to the way local binding forms like block add a fresh scope to both bound identifiers and body forms, it isn’t ordinarily possible for an identifier to reference a local binding without appearing in the body of the block. However, if macros use compile-time state to stash bound identifiers, they can violate this constraint. For example, the following stash_id and unstash_id macros cooperate to move a reference to a locally-bound x identifier outside of the lexical region in which it is bound:

> meta:     def mutable stashed_id = #false

> expr.macro 'stash_id $id':     stashed_id := id     '#void'

> expr.macro 'unstash_id':     stashed_id

> block:     def x = 42     stash_id x     unstash_id

42

> unstash_id

unstash_id: identifier used out of context

In general, an identifier’s lexical information is not sufficient to know whether or not its binding is available in the enclosing context, since the scope set for the identifier stored in stashed_id unambiguously refers to a binding in the global binding table. However, the reference produced by unstash_id in the above program is still illegal, even if it isn’t technically unbound. To record the fact that x’s binding is in scope only within the body of its corresponding block form, the expander adds x’s binding to the local binding context while expanding the block body. More generally, the expander adds all local variable bindings to the local binding context while expanding expressions in which a reference to the variable would be legal. When the expander encounters an identifier bound to a local variable, and the associated binding is not in the current local binding context, it throws a syntax error.

The local binding context also tracks local macro bindings (i.e. bindings bound by forms like meta.bridge or expr.macro in a similar way, except that the context also stores the compile-time value associated with the transformer. When an identifier that is locally bound as a transformer is parsed, the local binding context is consulted to retrieve the value. If the binding is in scope, its associated compile-time value is used; otherwise, the expander throws a syntax error.

2.4.3 Partial Expansion🔗ℹ

In certain contexts, such as an internal-definition context or module context, partial expansion is used to determine whether forms represent definitions, expressions, or other declaration forms. Partial expansion works by cutting off the normal recursive expansion when the relevant binding is for a primitive syntactic form.

2.4.4 Internal Definitions🔗ℹ

An internal-definition context supports local definitions mixed with expressions. Forms that allow internal definitions document such positions using the body meta-variable.

Expansion relies on partial expansion of each body in an internal-definition sequence. Partial expansion of each body produces a form matching one of the following cases:

• A definition form like def: The binding table is immediately enriched with bindings for the definition form. Further expansion of the definition is deferred, and partial expansion continues with the rest of the body.

• A transformer definition form like expr.macro: The right-hand side is expanded and evaluated, and a transformer binding is installed for the body sequence before partial expansion continues with the rest of the body.

After all body forms are partially expanded, if no definitions were encountered, then the expressions are collected into a sequence as the internal-definition context’s expansion. Otherwise, at least one expression must appear after the last definition.

Before partial expansion begins, expansion of an internal-definition context begins with the introduction of a fresh outside-edge scope on the content of the internal-definition context. This outside-edge scope effectively identifies syntax objects that are present in the original form. An inside-edge scope is also created and added to the original content; furthermore, the inside-edge scope is added to the result of any partial expansion. This inside-edge scope ensures that all bindings introduced by the internal-definition context have a particular scope in common.

2.4.5 Module Expansion, Phases, and Visits🔗ℹ

Expansion of a module form proceeds in a similar way to expansion of an internal-definition context: an outside-edge scope is created for the original module content, and an inside-edge scope is added to both the original module and any form that appears during a partial expansion of the module’s top-level forms to uncover definitions and imports.

A import form not only introduces bindings at expansion time, but also visits the referenced module when it is encountered by the expander. That is, the expander instantiates any variables defined in the module within meta, and it also evaluates all expressions for macro bindings via meta.bridge, expr.macro and similar.

Module visits propagate through imports in the same way as module instantiation. Moreover, when a module is visited at phase 0, any module that it imports with import meta is instantiated at phase 1, while further import meta -1s leading back to phase 0 causes the required module to be visited at phase 0 (i.e., not instantiated).

During compilation, the top-level of module context is itself implicitly visited. Thus, when the expander encounters import meta, it immediately instantiates the required module at phase 1, in addition to adding bindings at phase level 1 (i.e., the transformer environment). Similarly, the expander immediately evaluates any form that it encounters within meta.

Phases beyond 0 are visited on demand. For example, when the right-hand side of a phase-0 expr.macro is to be expanded, then modules that are available at phase 1 are visited. More generally, initiating expansion at phase n visits modules at phase n, which in turn instantiates modules at phase n+1. These visits and instantiations apply to available modules in the enclosing evaluator’s module registry; a per-registry lock prevents multiple threads from concurrently instantiating and visiting available modules.

When the expander encounters import and import meta within a module context, the resulting visits and instantiations are specific to the expansion of the enclosing module, and are kept separate from visits and instantiations triggered from a top-level context or from the expansion of a different module.

2.4.6 Macro-Introduced Bindings in the Top Level🔗ℹ

When a top-level definition binds an identifier that originates from a macro expansion, the definition captures only uses of the identifier that are generated by the same expansion due to the fresh scope that is generated for the expansion.

For a top-level definition (outside of a module), the order of evaluation affects the binding of a generated definition for a generated identifier use. If the use precedes the definition, then the use is resolved with the bindings that are in place at that point, which will not include the binding from the subsequently macro-generated definition. (No such dependency on order occurs within a module, since a module binding covers the entire module body.) To support the declaration of an identifier before its use, the meta.bridge form avoids binding an identifier if the body of the definition produces zero values.

Macro-generated import and export forms also introduce and reference generation-specific bindings (due to the added scope) with the same ordering effects as for definitions.