checkers: Testing Framework🔗ℹ

1 Introduction to Checkers🔗ℹ

A test is written as a test expression, usually containing one or more check expressions. Tests are actions, not values: a test expression’s body is immediately executed, and the test expression returns (void). Tests may be anonymous or named.

> (test (check (+ 5 -5) #:is 0))

> (test #:name "identity"     (check (+ 7 0) #:is 7)     (check (* 8 1) #:is 8))

If a check fails, information about the failure and the enclosing test is printed and the execution of the enclosing test stops.

> (test #:name "addition"     (check (+ 1 1) #:is 2)     (test #:name "more addition"       (check (+ 2 2) #:is 5) ; whoops       (printf "this is not printed\n"))     (printf "but this line is\n"))

but this line is

-------------------- addition > more addition FAIL location:   eval:4:0 actual:     4 expected:   5 --------------------

The check form catches exceptions and multiple values in the “actual” expression and supports several kinds of assertions about its result.

> (test #:name "arithmetic"     (check (+ 1 2) #:is 3)     (check (+ 4 6) #:with even?)     (check (quotient/remainder 10 3) #:is (values 3 1))     (check (/ 1 0) #:error exn:fail:contract:divide-by-zero?))

A check can contain multiple assertions about the actual result. This is often useful for error and predicate tests.

> (test     (check (modulo 5 0)            #:error exn:fail:contract:divide-by-zero?            #:error #rx"^modulo: "))

> (test     (check (range 10)            #:with list?            #:with (lambda (v) (= (length v) 10))))

Checks return (void) by default, but they can optionally forward the result of the actual expression, allowing the checked computation to be used in other computations and other checks.

> (test     (define-values (n r)       (check (quotient/remainder 10 3) #:values))     (check (+ (* n 3) r) #:is 10))

2 Checkers API🔗ℹ

If a check expression is executed during the evaluation of the test body and fails, then evaluation of the test stops and the test is marked as failed. Otherwise, if evaluation of the test body completes, the test is marked as passed. Check failures are implemented by calling raise with special non-exception values. The test form only catches these values; it does not catch exceptions.

Tests may execute nested tests. Checks only affect the immediately enclosing test; the failure of an inner nested test does not cause the outer test to fail.

The result of actual-expr may be a single value, multiple values, or a raised exception (an instance of exn:fail or a subtype). If actual-expr escapes through a continuation jump or by raising a value that does not satisfy exn:fail?, its result is not caught by check. In particular, check does not catch breaks (exn:break).

The following forms of check-clause are supported:

#:is expected-expr

Succeeds if the actual result is equal (equal?) to the result of expected-expr. The evaluation of expected-expr must produce a single value or multiple values; exceptions in expected-expr are not caught. If expected-expr’s result has multiple values, then actual-expr’s result must have the same number of values, and the values must be pairwise equal?.

Equivalent to #:with (checker:equal expected-expr).

#:is-not unexpected-expr

Succeeds if the actual result has the same number of values as unexpected-expr’s result but is not equal (equal?) to it.

Equivalent to #:with (checker:not-equal unexpected-expr).

#:is-true

Succeeds if the actual result is a single value that is not #f.

Equivalent to #:with (λ (v) v), except for the information accompanying a check failure.

#:error predicate/regexp-expr

Succeeds if the actual expression raised an exception and that exception is accepted by the given predicate or regular expression. If a predicate is given, it is applied to the raised exception. If a regular expression is given, it is used to check the exception’s message (exn-message).

Equivalent to #:with (checker:error predicate/regexp-expr).

#:no-error

Succeeds if the actual expression did not raise an exception—that is, it produced a single value or multiple values.

#:with predicate/checker-expr

If a predicate is given, the check succeeds if the actual expression’s result is a single value and the predicate accepts that value. (See checker:predicate for predicates over multiple values.)

If a checker is given, the checker is applied to the result. The kind of result accepted (single value, multiple values, or raised exception) depends on the checker.

If all check clauses succeed, the result of the check expression is determined by maybe-forward. If maybe-forward is absent, then (void) is returned. Otherwise, maybe-forward must be one of the following:

#:forward

The check expression produces the same result as actual-expr. That is, if actual-expr produced values, the check expression returns those values; if actual-expr raised an exception, the check expression re-raises that exception.

#:values

If actual-expr produced values, the values are returned; otherwise, a check failure is signaled.

Equivalent to #:no-error #:forward.

3 Comparison with RackUnit and Others🔗ℹ

This library adopts a pure “tests as actions” model, unlike RackUnit, which started with a “tests as values” model and then later mixed in partial “tests as actions” support. This library does not distinguish test suites from test cases. A test may execute check expressions, acting as a test case, and it may also execute nested test expressions, acting as a test suite. If a check occurs outside of any test, RackUnit automatically wraps it with an anonymous test, but this library does not.

This library’s test form does not catch exceptions, so it does not support RackUnit’s “test error” status. Instead, this library makes check the sole form responsible for catching exceptions, as well as handling multiple values. As a consequence, single-value checks, multiple-value checks, and error checks all use the same check interface, and check failures due to exceptions are distinguished from test-scripting errors.

The design of this library was influenced by chk, expectations, and test-more, in addition to RackUnit.