On this page:
check:
pre:
post:
submission-eval
user-data
user-substs
pairs-or-singles-with-warning
teams-in-file
add-header-line!
procedure/  arity?
!defined
!bound
!syntax
!procedure
!procedure*
!integer
!integer*
!test
!test/  exn
!eval
!all-covered

6.3 Checker

The handin-server/checker module provides a higher-level of utilities, helpful in implementing ‘checker’ functions that are intended for a more automated system. This module is a language module—a typical checker that uses it looks like this:
(module checker handin-server/checker
  (check: :language  '(special intermediate)
          :users     pairs-or-singles-with-warning
          :coverage? #t
    (!procedure Fahrenheit->Celsius 1)
    (!test (Fahrenheit->Celsius  32)   0)
    (!test (Fahrenheit->Celsius 212) 100)
    (!test (Fahrenheit->Celsius  -4) -20)
    ...))

syntax

(check: keys-n-vals body ...)

 
keys-n-vals = 
  | :key val keys-n-vals
Constructs (and provides) an appropriate checker function, using keywords to customize features that you want it to have. The body of the checker (following the keywords) can contain arbitrary code, using utility functions from handin-server/utils, as well as additional ones that are defined below. Submission files are arriving to the handin server in binary form (in the GRacket format that is used to store text and other objects like images), and a number of these options involve genrating a textual version of this file. The purpose of these options is to have these text files integrate easily into a course framework for grading, based on these text files.

Keywords for configuring check::

Within the body of check:, users and submission will be bound to the checker arguments—a (sorted) list of usernames and the submission as a byte string. In addition to the functionality below, you can use (!eval expr) (or ((submission-eval) 'expr)) to evaluate expressions in the submitted code context, and you can use (with-submission-bindings (id ...) body ...) to evaluate the body when id’s are bound to their values from the submission code.}

syntax

(pre: body ...)

syntax

(post: body ...)

These two macros define a pre- and a post-checker. In their bodies, users and submission are bound as in check:, but there is nothing else special about these. See the description of the pre-checker and post-checker values for what can be done with these, and note that the check for valid users is always first. An example for a sophisticated post: block is below—it will first set a very long timeout for this session, then it will send an email with a submission receipt, with a CC to the TA (assuming a single TA), and pop-up a message telling the student about it:

(require net/sendmail)
(post:
  (define info
    (format "hw.rkt: ~a ~a"
            (file-size "hw.rkt")
            (file-or-directory-modify-seconds "hw.rkt")))
  (timeout-control 300)
  (log-line "Sending a receipt: ~a" info)
  (send-mail-message
   "[email protected]"
   "Submission Receipt"
   (map (lambda (user) (user-substs user "{Full Name} <{Email}>"))
        users)
   (list (user-substs (car users) "{TA Name} <{TA Email}>"))
   null
   `("Your submission was received" ,info))
  (message (string-append
            "Your submission was successfully saved."
            "  You will get an email receipt within 30 minutes;"
            " if not, please contact the course staff.")
           '(ok)))

parameter

(submission-eval)  (any/c . -> . any)

(submission-eval eval)  void?
  eval : (any/c . -> . any)
Holds an evaluation procedure for evaluating code in the submission context.

procedure

(user-data user)  (listof string?)

  user : string?
Returns a user information given a username. The returned information is a list of strings that corresponds to the configured extra-fields.

procedure

(user-substs user fmt)  string

  user : string?
  fmt : string?
Uses the mappings in user-data to substitute user information for substrings of the form “{some-field-name}” in fmt. This procedure signals an error if a field name is missing in the user data. Also, “{username}” will always be replaced by the username and “{submission}” by the current submission directory.

This is used to process the :student-line value in the checker, but it is provided for additional uses. See the above sample code for post: for using this procedure.

procedure

(pairs-or-singles-with-warning users)  any

  users : (listof string?)
Intended for use as the :users entry in a checker. It will do nothing if there are two users, and throw an error if there are more. If there is a single user, then the user will be asked to verify a single submission. If the student cancels, then an exception is raised so the submission directory is retracted. If the student approves this, the question is not repeated (this is marked by creating a directory with a known name). This is useful for cases where you want to allow free pair submissions—students will often try to submit their work alone, and later on re-submit with a partner.

procedure

(teams-in-file team-file)  ((listof string?) . -> . void?)

  team-file : path-string?
Returns a procedure that can be used for the :users entry in a checker. The team file (relative from the server’s main directory) is expected to have user entries—a sequence of s-expressions, each one a string or a list of strings. The resulting procedure will allow submission only by teams that are specified in this file. Furthermore, if this file is modified, the new contents will be used immediately, so there is no need to restart the server of you want to change student teams. (But remember that if you change ("foo" "bar") to ("foo" "baz"), and there is already a "bar+foo" submission directory, then the system will not allow “foo” to submit with “bar”.)

procedure

(add-header-line! line)  void?

  line : string?
During the checker operation, can be used to add header lines to the text version of the submitted file (in addition to the :extra-lines setting). It will not have an effect if :create-text? is false.

procedure

(procedure/arity? proc arity)  boolean?

  proc : procedure?
  arity : number?
Returns #t if proc is a procedure that accepts arity arguments.

syntax

(!defined id ...)

Checks that the given identifiers are defined in the (evaluated) submission, and throws an error otherwise. The identifiers can be bound as either a plain value or as a syntax.

syntax

(!bound id ...)

Checks that the given identifiers are defined in the (evaluated) submission as a plain value. Throws an error if not, or if an identifier is bound to a syntax.

syntax

(!syntax id arity)

Checks that id is defined, and is bound as a macro.

syntax

(!procedure id arity)

Checks that id is defined, and is bound to a procedure.

syntax

(!procedure* expr arity)

Similar to !procedure but omits the defined check, making it usable with any expression, which is then evaluated in the submission context.

syntax

(!integer id)

syntax

(!integer* expr)

Similar to !procedure and !procedure* for integers.

syntax

(!test expr)

(!test expr result)
(!test expr result equal?)

syntax

(!test/exn expr)

The first !test form checks that the given expression evaluates to a non-#f value in the submission context, throwing an error otherwise. The second form compares the result of evaluation, requiring it to be equal to result. The third allows specifying an equality procedure. The !test/exn form checks that the given expression throws an exn:fail? error, throwing an error otherwise.

For the latter two !test forms, note that the result and equal? forms are not evaluated in the submission context.

syntax

(!eval expr)

Evaluate an arbitrary expression in the submission context. This is a simple shorthand for ((submission-eval) `expr).

procedure

(!all-covered)  void?

(!all-covered proc)  void?
  proc : (string? . -> . any)
When coverage information is enabled (see :coverage? above), checks the collected coverage information and throws an error with source information if some code is left uncovered. If proc is provided, it is applied to a string argument that describes the location of the uncovered expression ("<line>:<col>", "#<char-pos>", or "(unknown position)") instead of throwing an error. The collected information includes only execution coverage by submission code, excluding additional checker tests. You do not have to call this explicitly—it is called at the end of the process automatically when :coverage? is enabled. It is made available so you can call it earlier (e.g., before testing) to show clients a coverage error first, or if you want to avoid an error. For example, you can do this:

(!all-covered
 (lambda (where)
   (case (message (string-append
                   "Incomplete coverage at "where", do you want"
                   " to save this submission with 10% penalty?")
                  '(yes-no))
     [(yes) (add-header-line! "No full coverage <*90%>")
            (message "Handin saved with penalty.")]
     [else (error "aborting submission")])))