On this page:
1.1 Introduction
1.2 Expansion model for type expanders
1.2.1 Comparison with Te  X’s macro expansion model
1.2.2 Interactions between type expanders and scopes
1.3 The prop:  type-expander structure type property
«prop:type-expander»
«prop-guard»
«prop-guard-field-index»
«prop-guard-field-value»
«prop-guard-procedure»
«prop-guard-else-error»
1.3.1 The type-expander struct
«type-expander-struct»
1.4 Associating type expanders to identifiers
1.4.1 The type-expander syntax class
«expand-type-syntax-classes»
«type-syntax-class»
«type-contract»
«type-expand-syntax-class»
1.4.2 Calling type expanders
«apply-type-expander»
«apply-type-expander-checks»
1.4.3 Associating type expanders to already existing identifiers
«patched»
«patch»
1.4.4 Defining new type expanders
«define-type-expander»
1.4.5 Locally binding type expanders
1.5 Expanding types
«expand-type»
1.5.1 Cases handled by expand-type
1.5.2 Applying type expanders
«expand-type-case-expander»1
«expand-type-case-expander»2
«expand-type-case-app-expander»
«expand-type-case-app-other»
1.5.2.1 Polymorphic types with
«expand-type-case-∀-through»
«shadowed»
«expand-type-case-∀-later»
«expand-type-case-∀-app»
«app-args-error»
1.5.2.2 Recursive types with Rec
«expand-type-case-Rec»
1.5.2.3 Local bindings with Let and Letrec
«expand-type-case-Let»
«expand-type-case-Letrec»
1.5.2.4 Anonymous types with Λ
«eval-anonymous-expander-code»
«expand-type-case-app-Λ»
«expand-type-case-just-Λ/not-applicable»
«expand-type-case-Λ-later»
1.5.2.5 Preventing the expansion of types with No-Expand
«expand-type-case-noexpand»
1.5.2.6 The overloaded : identifier
«expand-type-case-:»
1.5.2.7 Last resort cases:   leaving the type unchanged
«expand-type-case-app-fallback»
«expand-type-case-fallback-T»
1.5.3 Debugging type expanders
«expand-type-debug-outer»1
«debug-type-expander»
«expand-type-debug-outer»2
«expand-type-debug-indent»
«expand-type-debug-before»
«expand-type-debug-after»
«expand-type-debug-rules»
1.6 Overloading typed/  racket forms
1.6.1 syntax classes
«remove-ddd»
«syntax-classes»1
1.6.2 Overview of the overloaded primitives
1.6.3 :
«:»
1.6.4 define-type
«define-type»
1.6.5 define
«define»
1.6.6 lambda
«lambda»
1.6.7 case-lambda
«case-lambda»
1.6.8 struct
«struct»
1.6.9 define-struct/  exec
«define-struct/exec»
1.6.10 ann
«ann»
1.6.11 cast
«cast»
1.6.12 unsafe-cast
«unsafe-cast»
1.6.13 inst
«inst»
1.6.14 row-inst
«row-inst»
1.6.15 let
«let»
1.6.16 let*
«let*»
1.6.17 let-values
«let-values»
1.6.18 make-predicate
«make-predicate»
1.6.19 :  type, :  print-type, :  query-type/  args, :  query-type/  result
«:type»
«:print-type»
«:query-type/args»
«:query-type/result»
1.6.20 Type expanders for the typed classes
«syntax-classes»2
«syntax-classes»3
«syntax-classes»4
«syntax-classes»5
«class»
1.6.21 Other typed/  racket forms
«other-forms»
1.7 Future work
1.8 Conclusion
«module-expander»
«module-main»
«*»
7.1

1 Implementation of the type expander library

This document describes the implementation of the type-expander library, using literate programming. For the library’s documentation, see the Type expander library document instead.

1.1 Introduction

Extensible types would be a nice feature for typed/racket. Unlike require and provide, which come with define-require-syntax and define-provide-syntax, and unlike match, which comes with define-match-expander, typed/racket doesn’t provide a way to define type expanders. The type-expander library extends typed/racket with the ability to define type expanders, i.e. type-level macros.

The Some example type expanders section presents a small library of type expanders built upon the mechanism implemented here.

We redefine the forms :, define, lambda and so on to equivalents that support type expanders. Type expanders are defined via the define-type-expander macro. Ideally, this would be handled directly by typed/racket, which would directly expand uses of type expanders.

1.2 Expansion model for type expanders

Type expanders are expanded similarly to macros, with two minor differences:
  • A form whose first element is a type expander, e.g. (F . args₁), can expand to the identifier of another type expander G. If the form itself appears as the first element of an outer form, e.g. ((F . args₁) . args₂), the first expansion step will result in (G . args₂). The official macro expander for Racket would then expand G on its own, as an identifier macro, without passing the args₂ to it. In contrast, the type expander will expand the whole (G . args₂) form, letting G manipulate the args₂ arguments.

  • It is possible to write anonymous macros,

    The Λ form can be used to create anonymous type expanders. Anonymous type expanders are to type expanders what anonymous functions are to function definitions. The following table presents the expression-level and type-level function and macro forms. Note that Let serves as a type-level equivalent to both let and let-syntax, as anonymous macros can be used in conjunction with Let to obtain the equivalent of let-syntax.

     

    Definitions

     

    Local binding

     

    Anonymous functions

    Functions

     

    define

     

    let

     

    λ

    Macros

     

    define-syntax

     

    let-syntax

     

    N/A

    Type‑level functionsa

     

    define-type

     

    Let

     

    Type‑level macros

     

    define-type-expander

     

    Let

     

    Λ

    a: The type-level functions are simple substitution functions, and cannot perform any kind of computation. They are, in a sense, closer to pattern macros defined with define-syntax-rule than to actual functions.

Combined, these features allow some form of "curried" application of type expanders: The F type expander could expand to an anonymous Λ type expander which captures the args₁ arguments. In the second expansion step, the Λ anonymous type expander would then consume the args₂ arguments, allowing F to effectively rewrite the two nested forms, instead of being constrained to the innermost form.

1.2.1 Comparison with TeX’s macro expansion model

For long-time TeX or LaTeX users, this may raise some concerns. TeX programs are parsed as a stream of tokens. A TeX commands is a macro. When a TeX macro occurs in the stream of tokens, it takes its arguments by consuming a certain number of tokens following it. After consuming these arguments, a TeX macro may expand to another TeX macro, which in turn consumes more arguments. This feature is commonly used in TeX to implement macros which consume a variable number arguments: the macro will initially consume a single argument. Depending on the value of that argument, it will then expand to a macro taking n arguments, or another macro taking m arguments. This pattern, omnipresent in any sufficiently large TeX program, opens the door to an undesirable class of bugs: when a TeX macro invocation appears in the source code, it is not clear syntactically how many arguments it will eventually consume. An incorrect parameter value can easily cause it to consume more arguments than expected. This makes it possible for the macro to consume the end markers of surrounding environments, for example in the code:

\begin{someEnvironment}

\someMacro{arg1}{arg2}

\end{someEnvironment}

the someMacro command may actually expect three arguments, in which case it will consume the \end token, but leave the {someEnvironment} token in the stream. This will result in a badly broken TeX program, which will most likely throw an error complaining that the environment \begin{someEnvironment} is not properly closed. The error may however occur in a completely different location, and may easily cause a cascade of errors (the missing \end{someEnvironment} may cause later TeX commands to be interpreted in a different way, causing them to misinterpret their arguments, which in turn may cause further errors. The end result is a series of mysterious error messages somewhat unrelated to the initial problem.

This problem with TeX macros can be summed up as follows: the number of tokens following a TeX macro invocation that will be consumed by the macro is unbounded, and cannot be easily guessed by looking at the raw source code, despite the presence of programmer-friendly looking syntactic hints, like wrapping arguments with {…}.

We argue that the expansion model for type expanders is less prone to this class of problems, for several reasons:
  • Firstly, macros can only consume outer forms if they appear as the leftmost leaf of the outer form, i.e. while the F macro in the expression

    ((F . args₁) . args₂)

    may access the args₂ arguments, it will be constrained within the (F . args₁) in the following code:

    (H leading-args₂ (F . args₁) . more-args₂)

    The first case occurs much more rarely than the second, so is less likely to happen

  • Secondly, all TeX macros will consume an arbitrary number of arguments in a linear fashion until the end of the enclosing group or a paragraph separation. In contrast, most type expanders will consume all the arguments within their enclosing application form, and no more. “Curried” type expanders, which expand to a lone macro identifier, will likely only represent a small subset of all type expanders. For comparison, consider the following TeX code:

    \CommandOne{argA}\CommandTwo{argB}

    The \CommandOne TeX macro might consume zero, one, two three or more arguments. If it consumes zero arguments, {argA} will not be interpreted as an argument, but instead will represent a scoped expression, similar to (let () argA). If \CommandOne consumes two or more arguments, \CommandTwo will be passed as an argument, unevaluated, and may be discarded or applied to other arguments than the seemingly obvious {argB} argument. The TeX code above could therefore be equivalent to any the following Racket programs:

    (CommandOne)
    (let () argA)
    (CommandTwo)
    (let () argB)
    (CommandOne argA)
    (CommandTwo)
    (let () argB)
    (CommandOne)
    (let () argA)
    (CommandTwo argB)
    (CommandOne argA)
    (CommandTwo argB)
    (CommandOne argA CommandTwo)
    (let () argB)

    (CommandOne argA CommandTwo argB)

    In contrast, the obvious interpretation at a first glance of the TeX program would be written as follows in Racket:

    (CommandOne argA)
    (CommandTwo argB)

    If these appear as “arguments” of a larger expression, then their meaning is unambiguous (unless the larger expression is itself a macro):

    (+ (CommandOne argA)
       (CommandTwo argB))

    If however the (CommandOne argA) is the first element in its form, then, if it is a curried macro, it may consume the the (CommandTwo argB) form too:

    ((CommandOne argA)
     (CommandTwo argB))

    As stated earlier, this case will likely be less common, and it is clearer that the intent of the programmer to pass (CommandTwo argB) as arguments to the result of (CommandOne argA), either as a macro application or as a regular run-time function application.

  • Finally, Racket macros (and type expanders) usually perform a somewhat thorough check of their arguments, using syntax-parse or syntax-case patterns. Arguments to macros and type expanders which do not have the correct shape will trigger an error early, thereby limiting the risk of causing errors in cascade.

1.2.2 Interactions between type expanders and scopes

Our expansion model for type expanders therefore allows a type expander to escape the scope in which it was defined before it is actually expanded. For example, the following type:

(Let ([A Number])
  ((Let ([F (Λ (self T)
               #`(Pairof #,(datum->syntax #'self 'A)
                         T))])
     (Let ([A String])
       F))
   A))

first expands to:

(F
 A)

and then expands to:

(Pairof String A)

and finally expands to:

(Pairof String A)

Effectively, F captures the scope where its name appears (inside all three Let forms), but is expanded in a different context (outside of the two innermost Let forms).

Using Matthew Flatt’s notation to indicate the scopes present on an identifier, we can more explicitly show the expansion steps:

(Let ([A Number])
  ((Let ([F (Λ (self T)
               #`(Pairof #,(datum->syntax #'self 'A)
                         T))])
     (Let ([A String])
       F))
   A))

The first Let form annotates the identifier it binds with a fresh scope, numbered 1 here, and adds this scope to all identifiers within its body. It stores the binding in the (tl-redirections) binding table, as shown by the comment above the code

;A¹ := Number
((Let¹ ([ (Λ¹ (self¹ )
                #`(Pairof¹ #,(datum->syntax¹ #'self¹ ')
                           ))])
       (Let¹ ([ String¹])
             ))
 )

The second Let form then binds the F identifier, adding a fresh scope as before:

;A¹ := Number
;F¹² := (Λ¹ (self¹ T¹)
; #`(Pairof¹ #,(datum->syntax¹ #'self¹ 'A¹)
; T¹))
((Let¹² ([A¹² String¹²])
        F¹²)
 )

The third Let form then binds A within its body, leaving the outer A unchanged:

;A¹ := Number
;F¹² := (Λ¹ (self¹ T¹)
; #`(Pairof¹ #,(datum->syntax¹ #'self¹ 'A¹)
; T¹))
;A¹²³ := String¹²
(F¹²³
 )

The F¹²³ macro is then expanded, passing as an argument the syntax object #'(F¹²³ ). A fresh scope is added to the identifiers generated by the macro, in order to enforce macro hygiene. The identifier is passed as an input to the macro, so it is left unchanged, and A¹²³ is derived from F¹²³, via datum->syntax, and therefore has the same scopes (F¹²³ is also a macro input, so it is not tagged with the fresh scope). The Pairof¹ identifier, generated by the macro, is however flagged with the fresh scope 4. The result of the application of F to this syntax object is:

;A¹ := Number
;F¹² := (Λ¹ (self¹ T¹)
; #`(Pairof¹ #,(datum->syntax¹ #'self¹ 'A¹)
; T¹))
;A¹²³ := String¹²
(Pairof¹⁴ A¹²³ )

The Pairof¹⁴ type is resolved to the primitive type constructor Pairof:

;A¹ := Number
;F¹² := (Λ¹ (self¹ T¹)
; #`(Pairof¹ #,(datum->syntax¹ #'self¹ 'A¹)
; T¹))
;A¹²³ := String¹²
(Pairof A¹²³ )

The type A¹²³ is then resolved to String¹², which in turn is resolved to the String built-in type:

;A¹ := Number
;F¹² := (Λ¹ (self¹ T¹)
; #`(Pairof¹ #,(datum->syntax¹ #'self¹ 'A¹)
; T¹))
;A¹²³ := String¹²
(Pairof String )

And the type is resolved to Number:

;A¹ := Number
;F¹² := (Λ¹ (self¹ T¹)
; #`(Pairof¹ #,(datum->syntax¹ #'self¹ 'A¹)
; T¹))
;A¹²³ := String¹²
(Pairof String Number)

The syntax-local-value function does not support querying the transformer binding of identifiers outside of the lexical scope in which they are bound. In our case, however, we need to access the transformer binding of F¹²³ outside of the scope of the Let binding it, and similarly for A¹²³.

1.3 The prop:type-expander structure type property

Type expanders are identified by the prop:type-expander structure type property. Structure type properties allow the same identifier to act as a rename transformer, a match expander and a type expander, for example. Such an identifier would have to implement the prop:rename-transformer, prop:match-expander and prop:type-expander properties, respectively.

(define-values (prop:type-expander
                has-prop:type-expander?
                get-prop:type-expander-value)
  (make-struct-type-property 'type-expander prop-guard))

The value of the prop:type-expander property should either be a transformer procedure of one or two arguments which will be called when expanding the type, or the index of a field containing such a procedure.

(define (prop-guard val struct-type-info-list)
  (cond «prop-guard-field-index»
        «prop-guard-procedure»
        «prop-guard-else-error»))

If the value is a field index, it should be within bounds. The make-struct-field-accessor function performs this check, and also returns an accessor. The accessor expects an instance of the struct, and returns the field’s value.

[(exact-nonnegative-integer? val)
 (let* ([make-struct-accessor (cadddr struct-type-info-list)]
        [accessor (make-struct-field-accessor make-struct-accessor val)])
   (λ (instance)
     (let ([type-expander (accessor instance)])
       «prop-guard-field-value»)))]

The expander procedure will take one argument: the piece of syntax corresponding to the use of the expander. If the property’s value is a procedure, we therefore check that its arity includes 1.

(cond
  [(and (procedure? type-expander)
        (arity-includes? (procedure-arity type-expander) 2))
   (curry type-expander instance)]
  [(and (procedure? type-expander)
        (arity-includes? (procedure-arity type-expander) 1))
   type-expander]
  [else
   (raise-argument-error 'prop:type-expander-guard
                         (~a "the value of the " val "-th field should"
                             " be a procedure whose arity includes 1 or"
                             " 2")
                         type-expander)])

In the first case, when the property value is a field index, we return an accessor function. The accessor function expects a struct instance, performs some checks and returns the actual type expander procedure.

When the property’s value is directly a type expander procedure, we follow the same convention. We therefore return a function which, given a struct instance, returns the type expander procedure (ignoring the _ argument).

[(procedure? val)
 (cond
   [(arity-includes? (procedure-arity val) 2)
    (λ (s) (curry val s))]
   [(arity-includes? (procedure-arity val) 1)
    (λ (_) val)]
   [else
    (raise-argument-error 'prop:type-expander-guard
                          "a procedure whose arity includes 1 or 2"
                          val)])]

When the value of the prop:type-expander property is neither a positive field index nor a procedure, an error is raised:

[else
 (raise-argument-error
  'prop:type-expander-guard
  (~a "a procedure whose arity includes 1 or 2, or an exact "
      "non-negative integer designating a field index within "
      "the structure that should contain a procedure whose "
      "arity includes 1 or 2.")
  val)]

1.3.1 The type-expander struct

We make a simple struct that implements prop:type-expander and nothing else. It has a single field, expander-proc, which contains the type expander transformer procedure.

(struct type-expander (expander-proc) #:transparent
  #:extra-constructor-name make-type-expander
  #:property prop:type-expander (struct-field-index expander-proc))

1.4 Associating type expanders to identifiers

1.4.1 The type-expander syntax class

The type-expander syntax class recognises identifiers which are bound to type expanders. These fall into three cases:
  • The identifier’s syntax-local-value is an instance of a struct implementing prop:type-expander

  • The identifier has been bound by a type-level local binding form like Let or , and therefore are registered in the (tl-redirections) binding table.

  • The identifier has been patched via patch-type-expander, i.e. a type expander has been globally attached to an existing identifier, in which case the type expander is stored within the patched free identifier table.

(define-syntax-class type-expander
  (pattern local-expander:id
           #:when (let ([b (binding-table-find-best (tl-redirections)
                                                    #'local-expander
                                                    #f)])
                    (and b (has-prop:type-expander? b)))
           #:with code #'local-expander)
  (pattern (~var expander
                 (static has-prop:type-expander? "a type expander"))
           #:when (not (binding-table-find-best (tl-redirections)
                                                #'expander
                                                #f))
           #:with code #'expander)
  (pattern patched-expander:id
           #:when (let ([p (free-id-table-ref patched
                                              #'patched-expander
                                              #f)])
                    (and p (has-prop:type-expander? p)))
           #:when (not (binding-table-find-best (tl-redirections)
                                                #'expander
                                                #f))
           #:with code #'patched-expander))

We also define a syntax class which matches types. Since types can bear many complex cases, and can call type expanders which may accept arbitrary syntax, we simply define the type syntax class as expr. Invalid syntax will be eventually caught while expanding the type, and doing a thorough check before any processing would only make the type expander slower, with little actual benefits. The type syntax class is however used in syntax patterns as a form of documentation, to clarify the distinction between types and run-time or compile-time expressions.

(define-syntax-class type
  (pattern :expr))

(define stx-type/c syntax?)

Finally, we define a convenience syntax class which expands the matched type:

(define-syntax-class type-expand!
  #:attributes (expanded)
  (pattern t:expr
           #:with expanded (expand-type #'t #f)))

1.4.2 Calling type expanders

The apply-type-expander function applies the syntax expander transformer function associated to type-expander-id. It passes stx as the single argument to the transformer function. Usually, stx will be the syntax used to call the type expander, like #'(te arg ...) or just #'te if the type expander is not in the first position of a form.

The identifier type-expander-id should be bound to a type expander, in one of the three possible ways described above.

(define/contract (apply-type-expander type-expander-id stx)
  (-> identifier? syntax? syntax?)
  (let ([val (or (binding-table-find-best (tl-redirections)
                                          type-expander-id
                                          #f)
                 (let ([slv (syntax-local-value type-expander-id
                                                (λ () #f))])
                   (and (has-prop:type-expander? slv) slv))
                 (free-id-table-ref patched type-expander-id #f))]
        [ctxx (make-syntax-introducer)])
    «apply-type-expander-checks»
    (ctxx (((get-prop:type-expander-value val) val) (ctxx stx)))))

The apply-type-expander function checks that its type-expander-id argument is indeed a type expander before attempting to apply it:

(unless val
  (raise-syntax-error 'apply-type-expander
                      (format "Can't apply ~a, it is not a type expander"
                              type-expander-id)
                      stx
                      type-expander-id))

1.4.3 Associating type expanders to already existing identifiers

As explained above, existing identifiers which are provided by other libraries can be “patched” so that they behave like type expanders, using a global table associating existing identifiers to the corresponding expander code:

(define-syntax patch-type-expander
  (syntax-parser
    [(_ id:id expander-expr:expr)
     #`(begin
         (begin-for-syntax
           (free-id-table-set! patched
                               #'id
                               (type-expander #,(syntax/loc this-syntax
                                                  expander-expr)))))]))

1.4.4 Defining new type expanders

The define-type-expander macro binds name to a type expander which uses (λ (arg) . body) as the transformer procedure. To achieve this, we create a transformer binding (with define-syntax), from name to an instance of the type-expander structure.

(define-syntax define-type-expander
  (syntax-parser
    [(_ (name:id arg:id) . body)
     #`(define-syntax name
         (type-expander #,(syntax/loc this-syntax (λ (arg) . body))))]
    [(_ name:id fn:expr)
     #`(define-syntax name
         (type-expander #,(syntax/loc this-syntax fn)))]))

1.4.5 Locally binding type expanders

Some features of the type expander need to locally bind new type expanders:

We use with-bindings (defined in another file) to achieve this. The code

(with-bindings [bound-ids transformer-values]
               rebind-stx
  transformer-body)

evaluates transformer-body in the transformer environment. It creates a fresh scope, which it applies to the bound-ids and the rebind-stx. It associates each modified bound-id with the corresponding transformer-value in the (tl-redirections) binding table. The with-bindings form does not mutate the syntax objects, instead it shadows the syntax pattern variables mentioned in bound-ids and rebind-stx with versions pointing to the same syntax objects, but with the fresh scope flipped on them.

The code

(with-rec-bindings [bound-ids generate-transformer-values rhs]
                   rebind-stx
  transformer-body)

works in the same way, but it also flips the fresh scope on each element of rhs. The generate-transformer-values is expected to be a transformer expression which, given an element of rhs with the flipped scope, produces the transformer value to bind to the corresponding bound-id.

The implementation of with-bindings unfortunately does not play well with syntax-local-value, so the binding table has to be queried directly instead of using syntax-local-value. To our knowledge, the only ways to make new bindings recognised by syntax-local-value are:
  • To expand to a define-syntax form, followed with a macro performing the remaining work

  • Equivalently, to expand to a let-syntax form, whose body is a macro performing the remaining work

  • To call local-expand with an internal definition context which contains the desired bindings

  • To explicitly call syntax-local-value with an internal definition context argument

It is not practical in our case to use the first solution involving define-syntax, as the type expander may be called while expanding an expression (e.g. ann). The next two solutions assume that syntax-local-value will be called in a well-scoped fashion (in the sense of the official expander): in the second solution, syntax-local-value must be called by expansion-time code located within the scope of the let-syntax form, and in the third solution, syntax-local-value must be called within the dynamic extent of local-expand. The last solution works, but requires that the user explicitly passes the appropriate internal definition context.

The second and third solutions cannot be applied in our case, because type expanders can be expanded outside of the scope in which they were defined and used, as explained the Interactions between type expanders and scopes section.

The current version of the type expander does not support a reliable alternative to syntax-local-value which takes into account local binding forms for types (Let, and Rec), but one could be implemented, either by using some tricks to make the first solution work, or by providing an equivalent to syntax-local-value which consults the (tl-redirections) binding table.

1.5 Expanding types

The expand-type function fully expands the type stx. As explained in Locally binding type expanders, shadowing would be better handled using scopes. The expand-type function starts by defining some syntax classes, then parses stx, which can fall in many different cases.

(define (expand-type stx [applicable? #f])
  (start-tl-redirections
   «expand-type-syntax-classes»
   (define (expand-type-process stx first-pass?)
     «expand-type-debug-before»
     ((λ (result) «expand-type-debug-after»)
      (parameterize («expand-type-debug-indent»)
        «expand-type-debug-rules»
        (syntax-parse stx
          «expand-type-case-:»
          «expand-type-case-expander»
          «expand-type-case-∀-later»
          «expand-type-case-Λ-later»
          «expand-type-case-app-expander»
          «expand-type-case-∀-through»
          «expand-type-case-Rec»
          «expand-type-case-app-Λ»
          «expand-type-case-just-Λ/not-applicable»
          «expand-type-case-∀-app»
          «expand-type-case-Let»
          «expand-type-case-Letrec»
          «expand-type-case-noexpand»
 
          ;Must be after other special application cases
          «expand-type-case-app-other»
          «expand-type-case-app-fallback»
          «expand-type-case-fallback-T»))))
   (expand-type-process stx #t)))

1.5.1 Cases handled by expand-type

The cases described below which expand a use of a type expander re-expand the result, by calling expand-type once more. This process is repeated until no more expansion can be performed. This allows type expanders to produce calls to other type expanders, exactly like macros can produce calls to other macros.

We distinguish the expansion of types which will appear as the first element of their parent form from types which will appear in other places. When the applicable? argument to expand-type is #true, it indicates that the current type, once expanded, will occur as the first element of its enclosing form. If the expanded type is the name of a type expander, or a or Λ form, it will be directly applied to the given arguments by the type expander. When applicable? is #false, it indicates that the current type, once expanded, will not appear as the first element of its enclosing form (it will appear in another position, or it is at the top of the syntax tree representing the type).

When applicable? is #true, if the type is the name of a type expander, or a or Λ form, it is not expanded immediately. Instead, the outer form will expand it with the arguments. Otherwise, these forms are expanded without arguments, like identifier macros would be.

1.5.2 Applying type expanders

When a type expander is found in a non-applicable position, it is called, passing the identifier itself to the expander. An identifier macro would be called in the same way.

[expander:type-expander
 #:when (not applicable?)
 (rule id-expander/not-applicable
   (let ([ctxx (make-syntax-introducer)])
     (expand-type (ctxx (apply-type-expander #'expander.code
                                             (ctxx #'expander)))
                  applicable?)))]

When a type expander is found in an applicable position, it is returned without modification, so that the containing application form may expand it with arguments. When the expander e appears as (e . args), it is applicable. It is also applicable when it appears as ((Let (bindings…) e) . args), for example, because Let propagates its applicable? status.

[expander:type-expander
 #:when applicable?
 (rule id-expander/applicable
   #'expander)]

When a form contains a type expander in its first element, the type expander is called. The result is re-expanded, so that a type expander can expand to a use of another type expander.

[(~and expander-call-stx (expander:type-expander . _))
 (rule app-expander
   (let ([ctxx (make-syntax-introducer)])
     (expand-type (ctxx (apply-type-expander #'expander.code
                                             (ctxx #'expander-call-stx)))
                  applicable?)))]

When a form of the shape (f . args) is encountered, and the f element is not a type expander, the f form is expanded, and the whole form (with f replaced by its expansion) is expanded a second time. The applicable? parameter is set to #true while expanding f, so that if f produces a type expander (e.g. f has the shape (Let () some-type-expander)), the type expander can be applied to the args arguments.

[(~and whole (f . args))
 #:when first-pass?
 (rule app-other
   (expand-type-process
    (datum->syntax #'whole
                   (cons (expand-type #'f #true) #'args)
                   #'whole
                   #'whole)
    #f))]

1.5.2.1 Polymorphic types with

When the or All special forms from typed/racket are used, the bound type variables may shadow some type expanders. The type expanders used in the body T which have the same identifier as a bound variable will be affected by this (they will not act as a type-expander anymore). The body of the or All form is expanded with the modified environment. The result is wrapped again with ( (TVar ) expanded-T), in order to conserve the behaviour from typed/racket’s .

[({~and  {~literal }} (tvar:id ) T:type)
 #:when (not applicable?)
 (rule just-∀/not-applicable
   (with-syntax ([(tvar-vars-only ) (remove-ddd #'(tvar ))])
     (with-bindings [(tvar-vars-only ) (stx-map «shadowed»
                                                 #'(tvar-vars-only ))]
                    (T tvar )
       #`( (tvar )
            #,(expand-type #'T #f)))))]

Where «shadowed» is used to bind the type variables tvarᵢ to (No-Expand tvarᵢ), so that their occurrences are left intact by the type expander:

(λ ()
  (make-type-expander
   (λ (stx)
     (syntax-case stx ()
       [self (identifier? #'self) #'(No-Expand self)]
       [(self . args) #'((No-Expand self) . args)]))))

When a polymorphic type is found in an applicable position, it is returned without modification, so that the containing application form may expand it, binding the type parameters to their effective arguments.

[(~and whole ({~literal } (tvar:id ) T:type))
 #:when applicable?
 (rule just-∀/applicable
   #'whole)]

When a polymorphic type is immediately applied to arguments, the type expander attempts to bind the type parameters to the effective arguments. It currently lacks any support for types under ellipses, and therefore that case is currently handled by the «expand-type-case-app-fallback» case described later.

[(({~literal } ({~and tvar:id {~not {~literal }}} ) τ) arg )
 (unless (= (length (syntax->list #'(tvar )))
            (length (syntax->list #'(arg ))))
   «app-args-error»)
 (rule app-∀
   (with-bindings [(tvar ) (stx-map (λ (a) (make-type-expander (λ (_) a)))
                                         #'(arg ))]
                      τ
         (expand-type #'τ applicable?)))]

If the given number of arguments does not match the expected number of arguments, an error is raised immediately:

(raise-syntax-error
 'type-expander
 (format (string-append "Wrong number of arguments to "
                        "polymorphic type: ~a\n"
                        "  expected: ~a\n"
                        "  given: ~a"
                        "  arguments were...:\n")
         (syntax->datum #'f)
         (length (syntax->list #'(tvar )))
         (length (syntax->list #'(arg )))
         (string-join
          (stx-map (λ (a)
                     (format "~a" (syntax->datum a)))
                   #'(arg ))
          "\n"))
 #'whole
 #'
 (syntax->list #'(arg )))

1.5.2.2 Recursive types with Rec

Similarly, the Rec special form will cause the bound variable R to shadow type expanders with the same name, within the extent of the body T. The result is wrapped again with (Rec R expanded-T), in order to conserve the behaviour from typed/racket’s Rec.

[((~literal Rec) R:id T:type)
 (rule Rec
   #`(Rec R #,(with-bindings [R («shadowed» #'R)]
                                 T
                    (expand-type #'T #f))))]

1.5.2.3 Local bindings with Let and Letrec

The Let special form binds the given identifiers to the corresponding type expanders. We use with-bindings, as explained above in (part ("(lib type-expander/type-expander.hl.rkt)" "shadow")), to bind the Vᵢ identifiers to their corresponding Eᵢ while expanding T.

[((~commit (~literal Let)) ([Vᵢ:id Eᵢ] ) T:type)
 (rule Let
   (with-bindings [(Vᵢ )
                   (stx-map (λ (Eᵢ)
                              (make-type-expander
                               (λ (stx)
                                 (syntax-case stx ()
                                   [self (identifier? #'self) Eᵢ]
                                   [(self . argz) #`(#,Eᵢ . argz)]))))
                            #'(Eᵢ ))]
                  T
     (expand-type #'T applicable?)))]

The Letrec special form behaves in a similar way, but uses with-rec-bindings, so that the right-hand-side expressions Eᵢ appear to be within the scope of all the Vᵢ bindings.

[((~commit (~literal Letrec)) ([Vᵢ:id Eᵢ] ) T:type)
 (rule Letrec
   (with-rec-bindings [(Vᵢ )
                       (λ (Eᵢ)
                         (make-type-expander
                          (λ (stx)
                            (syntax-case stx ()
                              [self (identifier? #'self) Eᵢ]
                              [(self . args444) #`(#,Eᵢ . args444)]))))
                       Eᵢ]
                      T
     (expand-type #'T applicable?)))]

1.5.2.4 Anonymous types with Λ

When an anonymous type expander appears as the first element of its enclosing form, it is applied to the given arguments. We use the trampoline-eval function defined in another file, which evaluates the given quoted transformer expression, while limiting the issues related to scopes. The “official” eval function from racket/base removes one of the module scopes which are normally present on the expression to evaluate. In our case, we are evaluating an anonymous type expander, i.e. a transformer function. When using eval, identifiers generated by the transformer function may not have the expected bindings. The alternative trampoline-eval seems to solve this problem.

The auto-syntax-case form is used, so that an anonymous type expander (Λ (_ a b) ) can either use a and b as pattern variables in quoted syntax objects, or as regular values (i.e syntax->datum is automatically applied on the syntax pattern variables when they are used outside of syntax templates, instead of throwing an error).

(trampoline-eval
 #'(λ (stx)
     (define ctxx (make-syntax-introducer))
     (ctxx (auto-syntax-case (ctxx (stx-cdr stx)) ()
             [formals (let () . body)]))))

This case works by locally binding a fresh identifier tmp to a type expander, and then applying that type expander. It would also be possible to immediately invoke the type expander function.

[{~and whole (({~literal Λ} formals . body) . _args)}
 ;; TODO: use the same code as for the not-applicable case, to avoid ≠
 (rule app-Λ
   (with-syntax* ([tmp (gensym '#%Λ-app-)]
                  [call-stx #'(tmp . whole)])
     (with-bindings [tmp (make-type-expander
                          «eval-anonymous-expander-code»)]
                    call-stx
       (expand-type #'call-stx applicable?))))]

When a Λ anonymous type expander appears on its own, in a non-applicable position, it is expanded like an identifier macro would be.

This case is implemented like the «expand-type-case-app-Λ» case, i.e. by locally binding a fresh identifier tmp to a type expander, and then applying that type expander. The difference is that in the identifier macro case, the syntax object given as an argument to the type expander contains only the generated tmp identifier. This allows the type expander to easily recognise the identifier macro case, where the whole syntax form is an identifier, from regular applications, where the whole syntax form is a syntax pair. The whole original syntax is attached consed onto a syntax property named 'original-Λ-syntax, in (unlikely) case the type expander needs to access the original Λ syntax used to call it (this is an experimental feature, and may change without notice in later versions).

[{~and whole ({~literal Λ} formals . body)}
 #:when (not applicable?)
 (rule just-Λ/not-applicable
   (with-syntax* ([tmp (syntax-property
                        (datum->syntax #'whole
                                       (gensym '#%Λ-id-macro-)
                                       #'whole
                                       #'whole)
                        'original-Λ-syntax
                        (cons #'whole
                              (or (syntax-property #'whole
                                                   'original-Λ-syntax)
                                  null)))]
                  [call-stx #'(tmp . tmp)])
         (with-bindings [tmp (make-type-expander
                              «eval-anonymous-expander-code»)]
                        call-stx
           ;; applicable? should be #f here, otherwise it would have been
           ;; caught by other cases.
           (expand-type #'call-stx applicable?))))]

When a Λ anonymous type expander appears on its own, in an applicable position, it is returned without modification, so that the containing application form may expand it with arguments (instead of expanding it like an identifier macro would be).

[(~and whole ({~literal Λ} formals . body))
 #:when applicable?
 (rule just-Λ/applicable
   #'whole)]

1.5.2.5 Preventing the expansion of types with No-Expand

The No-Expand special form prevents the type expander from re-expanding the result. This is useful for example for the implementation of the fancy quote expander, which relies on the built-in quote expander. It is also used to implement shadowing: type variables bound by in non-applicable positions and type variables bound by Rec are re-bound to type expanders returning (No-Expand original-tvar).

[((~literal No-Expand) T)
 (rule just-No-Expand
   #'T)]
[(((~literal No-Expand) T) arg ...)
 (rule app-No-Expand
   #`(T #,@(stx-map (λ (τ) (expand-type τ #f)) #'(arg ...))))]

1.5.2.6 The overloaded : identifier

This case handles the colon identifiers : (overloaded by this library) and : (provided by typed/racket). Wherever the new overloaded : identifier appears in a type, we want to convert it back to the original : from typed/racket. The goal is that a type of the form ( Any Boolean : Integer), using the new :, will get translated to ( Any Boolean : Integer), using the old : so that it gets properly interpreted by typed/racket’s parser.

[(~and c (~literal new-:))
 (rule (datum->syntax #'here ': #'c #'c)
   ':)]

1.5.2.7 Last resort cases: leaving the type unchanged

If the type expression to expand was not matched by any of the above cases, then it can still be an application of a polymorphic type T. The arguments TArg can contain uses of type expanders. We therefore expand each separately, and combine the results.

[{~and whole (T TArg )}
 (rule app-fallback
   (quasisyntax/loc #'whole
     (T #,@(stx-map (λ (a) (expand-type a #f)) #'(TArg ...)))))]

As a last resort, we consider that the type T (which would most likely be an identifier) is either a built-in type provided by typed/racket, or a user-declared type introduced by define-type. In both cases, we just leave the type as-is.

[T
 (rule just-fallback
   #'T)]

1.5.3 Debugging type expanders

In order to facilitate writing type expanders, it is possible to print the inputs, steps and outputs of the expander using (debug-type-expander #t), which sets the value of debug-type-expander?. This can then be undone using (debug-type-expander #f).

(define debug-type-expander? (box #f))

(define-syntax (debug-type-expander stx)
  (syntax-case stx ()
    [(_ #t) (set-box! debug-type-expander? #t) #'(void)]
    [(_ #f) (set-box! debug-type-expander? #f) #'(void)]))

For better readability, each level of recursion indents the debugging information:

(define indent (make-parameter 0))

[indent (+ (indent) 3)]

Before expanding a term, it is printed:

(when (unbox debug-type-expander?)
  (printf "~a~a ~a"
          (make-string (indent) #\space)
          applicable?
          (+scopes stx)))

Once the term has been expanded, the original term and the expanded term are printed:

(when (unbox debug-type-expander?)
  (printf "~a~a ~a\n~a=> ~a (case: ~a)\n"
          (make-string (indent) #\space)
          applicable?
          (+scopes stx)
          (make-string (indent) #\space)
          (+scopes (car result))
          (cdr result))
  (when (= (indent) 0)
    (print-full-scopes)))
(car result)

Finally, each rule for the type expander is wrapped with the rule macro, which prints the name of the rule, and returns a pair containing the result and the rule’s name, so that the debugging information indicates the rule applied at each step.

(define-syntax-rule (rule name e)
  (begin (when (unbox debug-type-expander?)
           (printf "(case:~a)\n"
                   'name))
         (cons e 'name)))

1.6 Overloading typed/racket forms

Throughout this section, we provide alternative definitions of the typed/racket forms :, lambda, define, struct, ann, inst… . We write these definitions with syntax-parse, using the syntax classes defined in section syntax classes.

Most of the time, we will use the experimental template macro from syntax/parse/experimental/template which allows more concise code than the usual #'() and #`().

1.6.1 syntax classes

The syntax classes from typed-racket/base-env/annotate-classes match against the : literal. Since we provide a new definition for it, these syntax classes do not match code using our definition of :. We therefore cannot use the original implementations of curried-formals and lambda-formals, and instead have to roll out our own versions.

We take that as an opportunity to expand the types directly from the syntax classes using #:with, instead of doing that inside the macros that use them.

The colon syntax class records the identifier it matches as a "disappeared use", which means that DrRacket will draw an arrow from the library importing it (either typed/racket or type-expander) to the identifier. Unfortunately, this effect is not (yet) undone by syntax/parse’s backtracking. See https://groups.google.com/forum/#!topic/racket-users/Nc1klmsj9ag for more details about this.

(define (remove-ddd stx)
  (remove #'(... ...) (syntax->list stx) free-identifier=?))

(define-syntax-class colon
  #:attributes ()
  (pattern (~and {~or {~literal new-:} {~literal :}}
                 C
                 {~do (record-disappeared-uses (list #'C))})))
 
(define-splicing-syntax-class new-maybe-kw-type-vars
  #:attributes ([vars 1] maybe)
  (pattern kw+vars:lambda-type-vars
           #:with (vars ) (remove-ddd #'kw+vars.type-vars)
           #:with maybe #'kw+vars)
  (pattern (~seq)
           #:with (vars ) #'()
           #:attr maybe #f))
 
(define-splicing-syntax-class new-maybe-type-vars
  #:attributes ([vars 1] maybe)
  (pattern v:type-variables
           #:with (vars ) (remove-ddd #'v)
           #:with maybe #'v)
  (pattern (~seq)
           #:with (vars ) #'()
           #:attr maybe #f))
 
(define-splicing-syntax-class new-kw-formal
  #:attributes ([expanded 1])
  (pattern (~seq kw:keyword id:id)
           #:with (expanded ...) #'(kw id))
  (pattern (~seq kw:keyword [id:id
                             (~optional (~seq :colon type:type-expand!))
                             (~optional default:expr)])
           #:with (expanded ...)
           (template (kw [id ([email protected] : type.expanded)
                          (?? default)]))))
 
(define-splicing-syntax-class new-mand-formal
  #:attributes ([expanded 1])
  (pattern id:id
           #:with (expanded ...) #'(id))
  (pattern [id:id :colon type:type-expand!]
           #:with (expanded ...)
           (template ([id : type.expanded])))
  (pattern kw:new-kw-formal
           #:with (expanded ...) #'(kw.expanded ...)))
 
(define-splicing-syntax-class new-opt-formal
  #:attributes ([expanded 1])
  (pattern [id:id
            (~optional (~seq :colon type:type-expand!))
            default:expr]
           #:with (expanded ...)
           (template ([id (?? ([email protected] : type.expanded))
                       default])))
  (pattern kw:new-kw-formal
           #:with (expanded ...) #'(kw.expanded ...)))
 
(define-syntax-class new-rest-arg
  #:attributes ([expanded 0])
  (pattern rest:id
           #:with expanded #'rest)
  (pattern (rest:id
            :colon type:type-expand!
            (~or (~and x* (~describe "*" (~or (~literal *)
                                              (~literal ...*))))
                 (~seq (~literal ...) bound:type-expand!)))
           #:with expanded
           (template (rest : type.expanded
                           (?? x*
                               ([email protected] (... ...) bound.expanded))))))
 
(define-syntax-class new-lambda-formals
  (pattern (~or (mand:new-mand-formal ...
                 opt:new-opt-formal ...
                 . rest:new-rest-arg)
                (mand:new-mand-formal ...
                 opt:new-opt-formal ...))
           ;; TODO: once template supports ?? in tail position, use it.
           #:with expanded #`(mand.expanded ...
                              ...
                              opt.expanded ...
                              ...
                              . #,(if (attribute rest)
                                      #'rest.expanded
                                      #'()))))
 
(define-syntax-class (new-curried-formals def-id)
  (pattern (f:id . args:new-lambda-formals)
           #:with expanded #`(#,def-id . args.expanded))
  (pattern ((~var lhs (new-curried-formals def-id))
            . args:new-lambda-formals)
           #:with expanded #'(lhs.expanded . args.expanded)))
 
(define-syntax-class new-curried-formals-id
  (pattern (id:id . _))
  (pattern (lhs:new-curried-formals-id . _)
           #:with id #'lhs.id))
 
(define-splicing-syntax-class new-optionally-annotated-name
  (pattern (~seq name:id (~optional (~seq :colon type:type-expand!)))
           #:with expanded
           (template (name
                      (?? ([email protected] : type.expanded))))))
 
(define-syntax-class new-name-or-parenthesised-annotated-name
  (pattern name:id
           #:with expanded #'name)
  (pattern [id:id :colon type:type-expand!]
           #:with expanded
           (template [id : type.expanded])))

1.6.2 Overview of the overloaded primitives

The following sections merely define overloads for the typed/racket primitives. The process is similar each time: a new primitive is defined, e.g. new-: for :. The new primitive calls the old one, after having expanded (using expand-type) all parts of the syntax which contain types. Aside from heavy usage of syntax-parse, there is not much to say concerning these definitions.

1.6.3 :

«:» ::=
(set-:-impl! (syntax-parser
               [(_ x:id t:expr)
                #`(: x #,(expand-type #'t #f))]))

1.6.4 define-type

(define-syntax new-define-type
  (syntax-parser
    [(_ (~or name:id (name:id maybe-tvar )) . whole-rest)
     #:with (tvar ) (if (attribute maybe-tvar) #'(maybe-tvar ) #'())
     #:with (tvar-not-ooo ) (filter (λ (tv) (not (free-identifier=? tv #'( ))))
                                     (syntax->list #'(tvar )))
     (start-tl-redirections
      (with-bindings [(tvar-not-ooo ) (stx-map «shadowed»
                                                #'(tvar-not-ooo ))]
                     whole-rest
        (syntax-parse #'whole-rest
          [(type:type-expand! . rest)
           (template
            (define-type (?? (name tvar ) name)
              type.expanded
              . rest))])))]))

1.6.5 define

(define-syntax new-define
  (f-start-tl-redirections
   (syntax-parser
     [(_ {~and (~seq :new-maybe-kw-type-vars
                     (~or v:id
                          formals-id:new-curried-formals-id)
                     _ )
              (~with-tvars (tvars new-maybe-kw-type-vars)
                           (~or :id
                                (~var formals (new-curried-formals
                                               #'formals-id.id)))
                           (~optional (~seq :colon type:type-expand!))
                           e ...)})
      (template
       (define (?? ([email protected] . tvars.maybe)) (?? v formals.expanded)
         (?? ([email protected] : type.expanded))
         e ...))])))

1.6.6 lambda

(define-syntax new-lambda
  (f-start-tl-redirections
   (syntax-parser
     [(_ {~with-tvars (tvars new-maybe-kw-type-vars)
              args:new-lambda-formals
              (~optional (~seq :colon ret-type:type-expand!))
              e })
      (template (lambda (?? ([email protected] . tvars.maybe)) args.expanded
                  (?? ([email protected] : ret-type.expanded))
                  e ...))])))

1.6.7 case-lambda

(define-syntax new-case-lambda
  (f-start-tl-redirections
   (syntax-parser
     [(_ {~with-tvars (tvars new-maybe-kw-type-vars)
              [args:new-lambda-formals
               (~optional (~seq :colon ret-type:type-expand!))
               e ]
              })
      (template (case-lambda
                  (?? ([email protected] #:∀ tvars.maybe))
                  [args.expanded
                   (?? (ann (let () e ) ret-type.expanded)
                       ([email protected] e ))]
                  ))])))

1.6.8 struct

The name must be captured outside of the ~with-tvars, as ~with-tvars introduces everything in a new lexical context.

(define-syntax new-struct
  (f-start-tl-redirections
   (syntax-parser
     [(_ (~and
          (~seq :new-maybe-type-vars
                (~and (~seq name+parent )
                      (~or (~seq name:id)
                           (~seq name:id parent:id)))
                _ )
          {~with-tvars (tvars new-maybe-type-vars)
              (~or (~seq :id)
                   (~seq :id :id))
              ([field:id :colon type:type-expand!] ...)
              rest }))
      (template (struct (?? tvars.maybe) name (?? parent)
                  ([field : type.expanded] ...)
                  rest ))])))

1.6.9 define-struct/exec

(define-syntax (new-define-struct/exec stx)
  (syntax-parse stx
    [(_ (~and name+parent (~or name:id [name:id parent:id]))
        ([field:id (~optional (~seq :colon type:type-expand!))] ...)
        [proc :colon proc-type:type-expand!])
     (template (define-struct/exec name+parent
                 ([field (?? ([email protected] : type.expanded))] ...)
                 [proc : proc-type.expanded]))]))

1.6.10 ann

(define-syntax/parse (new-ann value:expr
                              (~optional :colon) type:type-expand!)
  (template (ann value type.expanded)))

1.6.11 cast

(define-syntax/parse (new-cast value:expr type:type-expand!)
  (template (cast value type.expanded)))

1.6.12 unsafe-cast

We additionally define an unsafe-cast macro, which Typed/Racket does not provide yet, but can easily be defined using unsafe-require/typed and a polymorphic function.

(module m-unsafe-cast typed/racket
  (provide unsafe-cast-function)
  (define (unsafe-cast-function [v : Any]) v))
 
(require (only-in typed/racket/unsafe unsafe-require/typed))
(unsafe-require/typed 'm-unsafe-cast
                      [unsafe-cast-function ( (A) ( Any A))])
 
(define-syntax-rule (unsafe-cast/no-expand v t)
  ((inst unsafe-cast-function t) v))
 
(define-syntax/parse (unsafe-cast value:expr type:type-expand!)
  (template (unsafe-cast/no-expand value type.expanded)))

1.6.13 inst

(define-syntax new-inst
  (syntax-parser
    [(_ v (~optional :colon) t:type-expand! ...
        last:type-expand! (~literal ...) b:id)
     (template (inst v
                     t.expanded ...
                     last.expanded (... ...) b))]
    [(_ v (~optional :colon) t:type-expand! ...)
     (template (inst v t.expanded ...))]))

1.6.14 row-inst

(define-syntax/parse (new-inst e row:type-expand!)
  (template (row-inst e row.expanded)))

1.6.15 let

(define-syntax new-let
  (f-start-tl-redirections
   (syntax-parser
     [(_ (~optional (~seq loop:id
                          (~optional
                           (~seq :colon return-type:type-expand!))))
         (~with-tvars (tvars new-maybe-kw-type-vars)
                      ([name:new-optionally-annotated-name e:expr] ...)
                      rest ...))
      (template
       (let (?? ([email protected] loop (?? ([email protected] : return-type.expanded))))
         ([email protected] . tvars)
         ([([email protected] . name.expanded) e] ...)
         rest ...))])))

1.6.16 let*

(define-syntax/parse
    (new-let*
     ([name:new-optionally-annotated-name e:expr] ...)
     . rest)
  (template
   (let* ([([email protected] . name.expanded) e] ...) . rest)))

1.6.17 let-values

(define-syntax/parse
    (new-let-values
     ([(name:new-name-or-parenthesised-annotated-name ...) e:expr] ...)
     . rest)
  (template
   (let-values ([(name.expanded ...) e] ...)
     . rest)))

1.6.18 make-predicate

(define-simple-macro (new-make-predicate type:type-expand!)
  (make-predicate type.expanded))

1.6.19 :type, :print-type, :query-type/args, :query-type/result

(define-syntax/parse (new-:type (~optional (~and verbose #:verbose))
                                type:type-expand!)
  (template (eval #'(#%top-interaction
                     . (:type (?? verbose) type.expanded)))))

(define-syntax/parse (new-:print-type e:expr)
  #'(:print-type e)
  #'(eval #'(#%top-interaction
             . (:print-type e))))

(define-syntax/parse (new-:query-type/args f type:type-expand! )
  #'(eval #'(#%top-interaction
             . (:query-type/args f type.expanded ))))

(define-syntax/parse (new-:query-type/result f type:type-expand!)
  #'(eval #'(#%top-interaction
             . (:query-type/result f type.expanded))))

1.6.20 Type expanders for the typed classes

Not all forms are supported for now.

(define-syntax-class field-decl
  (pattern id:id #:with expanded #'(field id))
  (pattern (maybe-renamed {~optional {~seq :colon type:type-expand!}}
                          {~optional default-value-expr})
           #:with expanded
           (template (maybe-renamed (?? ([email protected] : type.expanded))
                                    (?? default-value-expr)))))

(define-syntax-class field-clause
  #:literals (field)
  (pattern (field field-decl:field-decl )
           #:with expanded (template (field field-decl.expanded ))))

(define-syntax-class super-new-clause
  #:literals (super-new)
  (pattern (super-new . rest)
           #:with expanded (template (super-new . rest))))

(define-syntax-class class-clause
  #:attributes (expanded)
  (pattern :field-clause)
  (pattern :super-new-clause))

(define-syntax new-class
  (f-start-tl-redirections
   (syntax-parser
     [(_ superclass-expr
         {~with-tvars (tvars new-maybe-kw-type-vars)
              clause:class-clause ...})
      (template (class superclass-expr
                  (?? ([email protected] . tvars.maybe))
                  clause.expanded ...))])))

1.6.21 Other typed/racket forms

The other typed/racket forms below do not have an alternative definition yet.

(define-syntax (missing-forms stx)
  (syntax-parse stx
    [(_ name ...)
     (define/with-syntax (tmp ...) (generate-temporaries #'(name ...)))
     #'(begin
         (begin
           (define-syntax (tmp stx)
             (raise-syntax-error
              'name
              (format "~a not implemented yet for type-expander" 'name)
              stx))
           (provide (rename-out [tmp name])))
         ...)]))
 
(missing-forms
 ;;TODO: add all-defined-out in prims.rkt
 ;; top-interaction.rkt
 ;:type
 ;:print-type
 ;:query-type/args
 ;:query-type/result
 ;; case-lambda.rkt
 ;case-lambda
 ;case-lambda:
 pcase-lambda:
 ;; (submod "prims-contract.rkt" forms)
 require/opaque-type
 ;require-typed-struct-legacy
 require-typed-struct
 ;require/typed-legacy
 require/typed
 require/typed/provide
 require-typed-struct/provide
 ;cast
 ;make-predicate
 define-predicate
 ;; prims.rkt
 define-type-alias
 define-new-subtype
 define-typed-struct
 define-typed-struct/exec
 ;ann
 ;inst
 ;:
 define-struct:
 define-struct
 ;struct
 struct:
 λ:
 lambda:
 ;lambda
 ;λ
 ;define
 ;let
 ;let*
 letrec
 ;let-values
 letrec-values
 let/cc
 let/ec
 let:
 let*:
 letrec:
 let-values:
 letrec-values:
 let/cc:
 let/ec:
 for
 for/list
 for/vector
 for/hash
 for/hasheq
 for/hasheqv
 for/and
 for/or
 for/sum
 for/product
 for/lists
 for/first
 for/last
 for/fold
 for*
 for*/list
 for*/lists
 for*/vector
 for*/hash
 for*/hasheq
 for*/hasheqv
 for*/and
 for*/or
 for*/sum
 for*/product
 for*/first
 for*/last
 for*/fold
 for/set
 for*/set
 do
 do:
 with-handlers
 define-struct/exec:
 ;define-struct/exec)

1.7 Future work

We have not implemented alternative type-expanding definitions for all the typed/racket forms, as noted in Other typed/racket forms.

Integrating the type expander directly into typed/racket would avoid the need to provide such definitions, and allow using type expanders in vanilla typed/racket, instead of having to require this library. However, the code wrapping the typed/racket forms could be re-used by other libraries that alter the way typed/racket works, so implementing the remaining forms could still be useful.

Also, we would need to provide a syntax-local-type-introduce function, similar to the syntax-local-match-introduce function provided by match for example.

1.8 Conclusion

When an identifier is required from another module, it is not the same as the one visible within the defining module. This is a problem for :, because we match against it in our syntax classes, using (~literal :), but when it is written in another module, for example (define foo : Number 42), it is not the same identifier as the one used by original definition of :, and therefore the (~literal :) won’t match. I suspect that issue to be due to contract wrappers added by typed/racket.

To get around that problem, we define : in a separate module, and require it in the module containing the syntax classes:

Since our new-: macro needs to call the type-expander, and the other forms too, we cannot define type-expander in the same module as these forms, it needs to be either in the same module as new-:, or in a separate module. Additionally, expand-type needs to be required for-syntax by the forms, but needs to be provided too, so it is much easier if it is defined in a separate module (that will be used only by macros, so it will be written in racket, not typed/racket).

(module expander racket
  (require (for-template typed/racket
                         "identifiers.rkt")
           racket
           (only-in racket/base [... ])
           syntax/parse
           racket/format
           racket/syntax
           syntax/id-table
           syntax/stx
           auto-syntax-e
           "parameterize-lexical-context.rkt"
           debug-scopes
           racket/contract/base)
  ;; TODO: move this in a separate chunk and explain it
 
  (provide prop:type-expander
           (contract-out
            (rename has-prop:type-expander?
                    prop:type-expander?
                    (-> any/c boolean?))
            (rename get-prop:type-expander-value
                    prop:type-expander-ref
                    (-> has-prop:type-expander?
                        any/c)))
           type-expander
           apply-type-expander
           ;bind-type-vars
           expand-type
           type
           stx-type/c
           type-expand!
           debug-type-expander?
           patched
           make-type-expander)
 
  «remove-ddd»
 
  «prop-guard»
  «prop:type-expander»
  «type-expander-struct»
 
  «patched»
 
  «apply-type-expander»
  ;<expand-quasiquote>
  «type-syntax-class»
  «type-contract»
  «expand-type-debug-outer»
  «expand-type»
  «type-expand-syntax-class»)

We can finally define the overloaded forms, as well as the <define-type-expander> form.

(module main typed/racket
  (require (only-in typed/racket/base [... ])
           typed/racket/class
           (for-syntax racket
                       (only-in racket/base [... ])
                       racket/syntax
                       syntax/parse
                       syntax/parse/experimental/template
                       syntax/id-table
                       "parameterize-lexical-context.rkt"
                       syntax/stx)
           (for-meta 2 racket/base syntax/parse)
           "utils.rkt"
           syntax/parse/define
           "identifiers.rkt")
 
  (require (submod ".." expander))
  (require (for-syntax (submod ".." expander)))
  (require (for-syntax typed-racket/base-env/annotate-classes))
 
  (provide prop:type-expander
           prop:type-expander?
           prop:type-expander-ref
           expand-type
           define-type-expander
           patch-type-expander
           Let
           Letrec
           Λ
           ...*
           No-Expand
           unsafe-cast/no-expand
           unsafe-cast
           debug-type-expander
           (rename-out [new-: :]
                       [new-define-type        define-type]
                       [new-define             define]
                       [new-lambda             lambda]
                       [new-lambda             λ]
                       [new-case-lambda        case-lambda]
                       [new-case-lambda        case-lambda:]
                       [new-struct             struct]
                       [new-define-struct/exec define-struct/exec]
                       [new-ann                ann]
                       [new-cast               cast]
                       [new-inst               inst]
                       [new-let                let]
                       [new-let*               let*]
                       [new-let-values         let-values]
                       [new-make-predicate     make-predicate]
                       [new-:type              :type]
                       [new-:print-type        :print-type]
                       [new-:query-type/args   :query-type/args]
                       [new-:query-type/result :query-type/result]
                       ;[new-field              field]
                       ;[new-super-new          super-new]
                       [new-class              class]))
 
  (begin-for-syntax
    (define-syntax ~with-tvars
      (pattern-expander
       (syntax-parser
         [(_ (tv tv-stxclass) pat ...)
          #'{~seq {~var tmp-tv tv-stxclass}
                  {~seq whole-rest (... ...)}
                  {~parse (({~var tv tv-stxclass}) pat ...)
                   ;; rebind tvars:
                   (with-bindings [(tmp-tv.vars (... ...))
                                   (stx-map «shadowed»
                                            #'(tmp-tv.vars (... ...)))]
                                  ;; rebind occurrences of the tvars within:
                                  (tmp-tv whole-rest (... ...))
                     ;; to (re-)parse:
                     #'(tmp-tv whole-rest (... ...)))}}]))))
 
  «debug-type-expander»
 
  «:»
 
  «define-type-expander»
  «patch»
 
  (begin-for-syntax
    «remove-ddd»
    «syntax-classes»
 
    (provide colon))
 
  «define-type»
  «define»
  «lambda»
  «case-lambda»
  «struct»
  «define-struct/exec»
  «ann»
  «cast»
  «unsafe-cast»
  «inst»
  «let»
  «let*»
  «let-values»
  «make-predicate»
  «:type»
  «:print-type»
  «:query-type/args»
  «:query-type/result»
  ;<field>
  «class»
  ;<super-new>
  «other-forms»)

We can now assemble the modules in this order:

«*» ::=