Xsmith’s Bundled Generators (And How to Run Them)
Version xsmith 2.0.1 (42990ec)
The xsmith-examples package has some pre-built fuzzers that vary in their level of quality and/or utility as a learning aid.
Included examples:
- The simple directory, with several fuzzers using the canned-components library. These are the best fuzzers to look at if you want to build a new fuzzer using canned-components (recommended).
simple/python
simple/javascript
simple/lua
simple/racket
The racket-kernel fuzzer is another fuzzer built using the canned-components library, but is a bigger fuzzer aimed at lower-level code.
The schemely fuzzer, which is really a Racket fuzzer because I haven’t yet bothered to turn it into a generic scheme fuzzer instead. It is the best example of a fuzzer that does not use the canned-components library. It is recommended that you do use that library, but if you really don’t want to, the best code to read is in schemely and in the implementation of canned-components. However, everything that schemely does, the simple/racket fuzzer does as well, and simpler.
The cish fuzzer, which is a C fuzzer. cish was the first fuzzer, and it is a little rougher than the others and less useful as a guide for what you ought to do with Xsmith. That said, it does use more of the analysis and refinery machinery than other fuzzers.
The verilog fuzzer is not very featureful, it hasn’t yet had enough work to generate anything interesting.
The pythonesque fuzzer was started before canned-components, and has been superceded by simple/python.
The future directory has some experiments revolving around the design of the refiners feature, and are not really for pedagogical consumption.
When xsmith-examples is installed as a Racket package, executables for some of the bundled generators are placed in your Racket package bin directory. Usually this directory is $HOME/.racket/racket-<version>/bin on Linux, maybe $HOME/Library/Racket/<version>/bin on normal MacOS installs, and maybe /usr/local/bin for MacOS Homebrew installs.
These fuzzers can be run on the command line to generate a single program or as an http server that generates one program per request.
Command-line options for bundled Xsmith generators are all the same, and provided by xsmith-command-line.
1 Cish
Cish is a C program generator made with the Xsmith library. It has co-evolved with Xsmith, and is essentially the reference Xsmith program generator.
The executable for Cish is called xsmith-cish. Additionally, Cish can be run with the command racket -l xsmith/cish -- (the final -- causes further flags to be parsed by cish and not by Racket).
The command-line options available in Cish are:
--help – See all command-line options. The --help list will automatically stay up to date, unlike this documentation.
--version – Prints the version info of Xsmith and exits.
--server <true-or-false> – Whether to run the web server. Defaults to false.
--server-port <port-number> – Port to use when running server. Defaults to 8080.
--server-ip <ip-address> – Listen for connections to <ip-address>. Use false to accept connections from all the machine’s addresses. Defaults to 127.0.0.1.
--server-path <url-path> – Run the server at localhost:8080/url-path. Defaults to /.
--seed – Random seed for program generation. Defaults to whatever Racket does to initialize randomness normally.
--output-file <filename> – Outputs to <filename> instead of standard output when not used with --server.
--max-depth <n> – Maximum depth of the generated program tree.
--tree-on-error <true-or-false> – Whether to show a partial program tree when an error occurs. Defaults to false.
--with-<feature> <bool> – Enables or disables generator-specific features, where <feature> is replaced with concrete feature names.
Cish supports the following features for the --with-<feature> flags:
int – defaults to #t
float – defaults to #t
addition – defaults to #t
subtraction – defaults to #t
multiplication – defaults to #t
division – defaults to #t
modulus – defaults to #t
comparisons – defaults to #t
if-expression – defaults to #t
if-statement – defaults to #t
loop-statement – defaults to #t
null-statement – defaults to #t
structs – defaults to #t
volatile – defaults to #t
unsafe-math/range – defaults to #f
To compile cish output, you need to include cish’s directory and Csmith’s runtime directory in your header path to get safe_math.h.
eg.
xsmith-cish > cish-output.c && gcc -I $XSMITH_EXAMPLES_DIR/cish -I $CSMITH_DIR/runtime -o cish-output cish-output.c
2 Schemely
The executable for Schemely is called xsmith-schemely. Additionally, Schemely can be run with the command racket -l xsmith/schemely -- (the final -- causes further flags to be parsed by cish and not by Racket).
The command-line options available in Schemely are:
--help – See all command-line options. The --help list will automatically stay up to date, unlike this documentation.
--version – Prints the version info of Xsmith and exits.
--server <true-or-false> – Whether to run the web server. Defaults to false.
--server-port <port-number> – Port to use when running server. Defaults to 8080.
--server-ip <ip-address> – Listen for connections to <ip-address>. Use false to accept connections from all the machine’s addresses. Defaults to 127.0.0.1.
--server-path <url-path> – Run the server at localhost:8080/url-path. Defaults to /.
--seed – Random seed for program generation. Defaults to whatever Racket does to initialize randomness normally.
--output-file <filename> – Outputs to <filename> instead of standard output when not used with --server.
--max-depth <n> – Maximum depth of the generated program tree.
--tree-on-error <true-or-false> – Whether to show a partial program tree when an error occurs. Defaults to false.
--with-<feature> <bool> – Enables or disables generator-specific features, where <feature> is replaced with concrete feature names.
Schemely currently has no features for the --with-<feature> flags.
At the time of writing, Schemely really just supports Racket. At some future point it should generate portable Scheme code.
3 Verilog
The program generator for Verilog is called xsmith-verilog. It can be also run with the command racket -l xsmith/verilog -- (the final -- causes further flags to be parsed by the Verilog generator and not by Racket).
The command-line options for the Verilog generator are:
--help – See all command-line options. The --help list will automatically stay up to date, unlike this documentation.
--version – Prints the version info of Xsmith and exits.
--server <true-or-false> – Whether to run the web server. Defaults to false.
--server-port <port-number> – Port to use when running server. Defaults to 8080.
--server-ip <ip-address> – Listen for connections to <ip-address>. Use false to accept connections from all the machine’s addresses. Defaults to 127.0.0.1.
--server-path <url-path> – Run the server at localhost:8080/url-path. Defaults to /.
--seed – Random seed for program generation. Defaults to whatever Racket does to initialize randomness normally.
--output-file <filename> – Outputs to <filename> instead of standard output when not used with --server.
--max-depth <n> – Maximum depth of the generated program tree.
--tree-on-error <true-or-false> – Whether to show a partial program tree when an error occurs. Defaults to false.
--with-<feature> <bool> – Enables or disables generator-specific features, where <feature> is replaced with concrete feature names.
The Verilog generator currently has no features for the --with-<feature> flags.