interspersing unit tests in-line with implementation code, using the Racket test submodule, or in separate files;
unit testing of individual modules, either from within DrRacket, or from the command line or Emacs;
rapid interactive testing of expressions in the REPL; and
running hierarchical sets of individual module unit tests at once.
values of expressions (single value, or multiple value);
exceptions raised; and
output to current-output-port and current-error-port.
> (test (+ 1 2 3) 6)
> (test (+ 1 2 3) 7)
TEST FAILED [???] Value 6 did not match expected value 7 by equal?.
> (test 'simple-addition (+ 1 2 3) 7)
TEST FAILED [simple-addition] Value 6 did not match expected value 7 by equal?.
(test #:id 'simple-addition #:code (+ 1 2 3) #:val 7)
> (test (+ 1 (error "help!") 3) 3)
TEST FAILED [???] Got exception #(struct:exn:fail "help!"), but expected value 3.
> (test (+ 1 (error "help!") 3) #:exn exn:fail?)
> (test (+ 1 2 3) #:exn exn:fail?)
TEST FAILED [???] Got value 6, but expected exception matched by predicate exn:fail?.
> (test (begin 1 2 3) (values 1 2 3))
TEST FAILED [???] Value 3 did not match expected values (1 2 3) by equal?.
> (test (string-append "a" "b" "c") "abc")
> (define (close-enough-val-check a-values b-values) (and (null? (cdr a-values)) (null? (cdr b-values)) (let ((a (car a-values)) (b (car b-values))) (and (number? a) (number? b) (equal? (round (* 1000 a)) (round (* 1000 b)))))))
> (test 3.142 3.14159 #:val-check close-enough-val-check)
> (test (let ((o (open-output-string))) (display 'a o) (display 'b) (display 'c o) (get-output-string o)) "abc")
TEST FAILED [???] Value "ac" did not match expected value "abc" by equal?. Out bytes #"b" did not match expected #"" by equal?.
> (test (begin (fprintf (current-error-port) "%W%SYS$FROBINATOR_OVERHEAT\n") 0) 42)
TEST FAILED [???] Value 0 did not match expected value 42 by equal?. Err bytes #"%W%SYS$FROBINATOR_OVERHEAT\n" did not match expected #"" by equal?.
> (test (begin (display "blah") (display "blah") (display "blah") (* 44 2)) 88 #:out-check #f)
> (test-section 'fundamentals (test-section 'writing (test 'abcs (string-append "a" "b" "c") "abc")) (test-section 'arithmetic (test 'one-two-three (+ 1 2 3) 6) (test 'for-fife-sax (+ 4 5 6) 666)))
TEST FAILED [fundamentals arithmetic for-fife-sax] Value 15 did not match expected value 666 by equal?.
> (test-section 'foo-constant-with-z-arg-zero (for ((bar? (in-list '(#true #false)))) (test-section bar? (for ((power (in-range 1 9))) (test #:id power #:code (foo bar? power 0) #:val 42)))))
overeasy: Start Test Section [foo-constant-with-z-arg-zero]
overeasy: Start Test Section [foo-constant-with-z-arg-zero #t]
overeasy: Test Passed [foo-constant-with-z-arg-zero #t 1]
overeasy: Test Passed [foo-constant-with-z-arg-zero #t 2]
overeasy: Test Passed [foo-constant-with-z-arg-zero #t 3]
> (test 'basic-arithmetic (plussy 1 2 3) 6 #:fail "bug til move to new library")
overeasy: Test Failed Expectedly [basic-arithmetic] Value 5.9 did not match expected value 6 by equal?. (#:fail "bug til move to new library")
TEST FAILED [basic-arithmetic] Passed unexpectedly. (#:fail "bug til move to new library")
There are some more tricks you can do with test. Most notably, you’ll sometimes want to set up state in the system – Racket parameters, test input files, whatever. Because the test syntax can appear anywhere normal Racket code can, you can set up this state using normal Racket code. No special forms for setup and tear-down are required, nor are they provided.
Sanity-check how keyword argument shortcuts are done, and which combinations make sense.
Document the support for #rx with #:exn.
Document the support for #:notes, but rename it to #:note and support the old name (which some package might have already used, even though undocumented) as an alias.
Document how to use with submodules and DrRacket. Just examples of test submodule, and tests run when hitting the Run button with particular DrRacket option set.
In documentation, fill out the defproc for test form.
For #:out and #:err, force expected value to bytes (convert using UTF-8 if string), and also ensure that port encoding is UTF-8 (or whatever encoding makes sense).
Add conveniences for defining #:val-check predicates. For example, a convenient way to specify eq? of a single value (not multiple values).
Try to mess with the continuation marks for test failues due to unexpected exceptions, so that user can navigate first to the test form, and then to the origin of the exception.
Provide convenience for running tests from multiple files, in test submodules, and possibly entire files/modules.
This package does not yet expose an interface so that alternative means of reporting (e.g., GUI) can be added easily. This is intentional, until we can be comfortable that the existing internal interface won’t be changing soon.
- Version 4:2 —
Commented out some tests that have to be run manually.
- Version 4:1 —
- Version 4:0 —
Moving from PLaneT to new package system.
- Version 3:2 —
Removed documentation that wrongly claimed #:id as the only keyword argument worked, since there might have been a regression 3+ years ago, but only just now reported. (Thanks to David Storrs for reporting.)
- Version 3:1 —
- Version 3:0 —
Now requires Racket 5.3.1.
Converted to use the Racket logger facility, and format of the messages changed. This is reason for the change to the major version number of the PLaneT package.
#:fail feature added.
Messages regarding tests now qualify the test ID with the IDs of the parent test sections.
Internal simplifications to test contexts. There are no longer parent and child test contexts, but a single test context. Test contexts no longer have handlers for the start and end of tests. When there is not yet any test context, previously a test-section would create one kind of transient context, and test would create a different kind of transient context; now, both forms create the same kind of context, and it is not transient.
- Version 2:1 —
Fixed bug regarding “fprintf: format string requires 1 arguments, given 2; arguments were: #<output-port> "TEST BROKEN!~A\ " ...
- Version 2:0 —
Converted to McFly.
The default test context now raises an exception with syntax location info for failed test cases, rather than only writing a message to current-error-port.
The test syntax now preserves syntax location info better.
Added shorthand syntax (test ID CODE VAL RESTn ...).
The new name test-section is now preferred to the old name with-test-section.
In test-section syntax, the #:id keyword itself is now optional.
test-section may have no body forms.
- Version 0.1 —
Version 1:0 — 2011-08-26Initial release.
Copyright 2011, 2012, 2015, 2016 Neil Van Dyke. This program is Free Software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See http://www.gnu.org/licenses/ for details. For other licenses and consulting, please contact the author.