3 Library reference: the property-based testing DSL
| (require evm-redex/pbt) | package: evm-redex |
The evm-redex/pbt library lets you validate properties of a fixed EVM program — a routine of opcodes or a whole contract/transaction — by running it through the semantics on many generated concrete inputs. You write the property as a Hoare-style contract (pre/post conditions), optionally with per-step invariants and revert/success obligations, and the engine searches for a counterexample, shrinking it to a minimal failing input.
This is property-based testing, so it is incomplete: it finds counterexamples, it does not prove their absence. In exchange it runs on concrete inputs and so places no restriction on the program — cryptographic precompiles, non-linear arithmetic and loops all simply execute. It builds on rackcheck (generation and shrinking) and Redex’s own redex-check.
This chapter documents every exported binding. For a hands-on, step-by-step walkthrough that applies the library to real Solidity contracts, read the tutorial in The evm-redex test suite: tutorial, tests, and examples first; this reference explains each piece it uses.
(require evm-redex/pbt)
3.1 Defining properties
syntax
clause = #:code bytes | #:call tx | #:contract bytes | #:address addr | #:given ([x gen] ...) | #:world world | #:block block | #:stack stack | #:gas gas | #:memory memory | #:msg msg | #:orig-storage store | #:pre expr | #:post proc | #:invariant proc | #:revert-when expr | #:succeed-when expr | #:fuel n | #:trials n | #:seed n
Program (choose exactly one). #:code gives fragment mode: bytes is a list of opcode bytes, run from a fresh machine with run. #:call gives transaction mode: tx is built with make-tx and run with the full transaction machinery; pair it with #:world (a pre-state), or with #:contract and #:address to install a contract’s runtime code in a minimal world.
Inputs and setup. #:given lists the generated inputs; each x is bound (to a concrete value) in every other clause. In fragment mode the initial machine is configured with #:stack (top first), #:gas, #:memory, #:world, #:msg and #:orig-storage; in transaction mode the environment is #:world and #:block.
Obligations (any subset). #:pre is the Hoare antecedent over the inputs — when it is false the trial passes vacuously, so it gates the other obligations. #:post is (lambda (m0 m1) ....) in fragment mode or (lambda (w0 w1 result) ....) in transaction mode, where m0 / w0 is the pre-state (this is how you refer to “old” values) and result is a txn-run. #:invariant is a predicate checked at every step in fragment mode (via run-trace) and at the start/end boundary in transaction mode. #:revert-when / #:succeed-when demand that, under the given condition, the run reverts / succeeds.
Budget. #:fuel caps steps; #:trials sets the number of generated cases (default 1000); #:seed fixes the RNG for reproducibility.
(define-evm-property add-wraps #:code (list 1) ; ADD #:given ([a gen-word] [b gen-word]) #:stack (list a b) ; initial stack, top first #:gas 100 #:post (lambda (m0 m1) (= (stack-top m1) (u-add a b))))
struct
(struct evm-property (prop trials seed) #:transparent) prop : any/c trials : exact-nonnegative-integer? seed : (or/c #f exact-integer?)
3.2 Running properties
procedure
(check-evm-property p [ #:trials trials #:seed seed #:deadline deadline]) → void? p : evm-property? trials : (or/c #f exact-nonnegative-integer?) = #f seed : (or/c #f exact-integer?) = #f deadline : (or/c #f real?) = #f
procedure
(run-evm-property p [ #:trials trials #:seed seed #:deadline deadline]) → evm-result? p : evm-property? trials : (or/c #f exact-nonnegative-integer?) = #f seed : (or/c #f exact-integer?) = #f deadline : (or/c #f real?) = #f
procedure
(evm-property-holds? p [ #:trials trials #:seed seed #:deadline deadline]) → boolean? p : evm-property? trials : (or/c #f exact-nonnegative-integer?) = #f seed : (or/c #f exact-integer?) = #f deadline : (or/c #f real?) = #f
struct
(struct evm-result (status counterexample tests) #:transparent) status : (or/c 'passed 'falsified 'timed-out) counterexample : (or/c #f list?) tests : exact-nonnegative-integer?
3.3 Worlds, accounts, and transactions
A world is an association of addresses to accounts; an account is (list 'account nonce balance code storage transient), where code is a byte list and storage is an association of slot to value. Most of the time you obtain a world from deploy rather than writing one by hand.
procedure
(make-tx #:sender sender #:gas-limit gas-limit [ #:nonce nonce #:to to #:value value #:data data #:gas-price gas-price #:max-fee max-fee #:max-priority max-priority #:access-list access-list #:auth-list auth-list #:blob-hashes blob-hashes #:max-blob-fee max-blob-fee]) → any/c sender : exact-nonnegative-integer? gas-limit : exact-nonnegative-integer? nonce : exact-nonnegative-integer? = 0 to : (or/c #f exact-nonnegative-integer?) = #f value : exact-nonnegative-integer? = 0 data : (listof byte?) = '() gas-price : (or/c #f exact-nonnegative-integer?) = #f max-fee : exact-nonnegative-integer? = 0 max-priority : exact-nonnegative-integer? = 0 access-list : list? = '() auth-list : list? = '() blob-hashes : list? = '() max-blob-fee : exact-nonnegative-integer? = 0
procedure
(install-contract code addr [balance]) → list?
code : (listof byte?) addr : exact-nonnegative-integer? balance : exact-nonnegative-integer? = 0
value
DEFAULT-BLOCK : list?
3.4 Deploying contracts
A Solidity contract compiles to two bytecodes: the creation (init) code, which runs the constructor and returns the runtime code, and the runtime (deployed) code that is stored at the address. Running the creation code is the faithful path — it applies constructor storage writes and patches immutables — whereas installing the runtime directly with #:contract skips the constructor.
procedure
(deploy creation [ #:from from #:value value #:gas gas #:nonce nonce #:world world #:block block]) → deploy-result? creation : (listof byte?) from : exact-nonnegative-integer? = DEFAULT-DEPLOYER value : exact-nonnegative-integer? = 0 gas : exact-nonnegative-integer? = 30000000 nonce : (or/c #f exact-nonnegative-integer?) = #f world : list? = '() block : list? = DEFAULT-BLOCK
(define art (read-artifact "storage/storage.json" #:contract "SimpleStorage")) (define dep (deploy (artifact-creation art))) (deploy-result-ok? dep) ; #t (sload (deploy-result-world dep) (deploy-result-address dep) 0)
struct
(struct deploy-result (world address code ok? gas-used err) #:transparent) world : list? address : exact-nonnegative-integer? code : (or/c #f (listof byte?)) ok? : boolean? gas-used : exact-nonnegative-integer? err : (or/c #f string?)
3.5 Solidity artifacts and the ABI
struct
(struct artifact (name creation runtime abi raw srcmap sources) #:transparent) name : (or/c #f string?) creation : (or/c #f (listof byte?)) runtime : (or/c #f (listof byte?)) abi : any/c raw : any/c srcmap : (or/c #f string?) sources : (or/c #f (listof string?))
srcmap is the runtime source map, verbatim, and sources the file list its indices refer to; both are #f unless the artifact was built with them. They are what lifts bytecode coverage to Solidity lines (see the coverage harness under "tests/coverage/"); nothing in the library itself reads them.
procedure
path : path-string? name : (or/c #f string?) = #f
procedure
str : string? name : (or/c #f string?) = #f
Add srcmap-runtime to that list if you want line-level coverage: solc –combined-json bin,bin-runtime,abi,srcmap-runtime Token.sol. It leaves the bytecode byte-for-byte identical — nothing else in a suite moves — and simply adds the map (and solc’s sourceList) to the JSON.
procedure
(function-selector sig) → exact-nonnegative-integer?
sig : string?
(encode-call "transfer(address,uint256)" recipient amount) ; dynamic types too, matching the canonical Solidity vector: (encode-call "sam(bytes,bool,uint256[])" (list 100 97 118 101) #t (list 1 2 3))
procedure
types : (listof string?) values : list?
procedure
(abi-decode types bs) → list?
types : (listof string?) bs : (listof byte?)
uint<M> / int<M> (bare uint/int = 256), address — an integer (signed for int); bool — #t/#f (or 1/0).
bytes<M> (fixed, 1–32) and bytes (dynamic) — a byte list or a Racket bytes; string — a Racket string (or byte list).
T[] (dynamic) and T[k] (fixed) arrays — a list of element values; tuples (T1,...) — a list of the tuple’s element values.
3.6 Calling into a world
A transaction discards a function’s return value, so transaction-mode #:post sees the world, outcome and logs but not the returned bytes. To read a getter or view function, use call, a raw frame execution that exposes the output.
procedure
(call world addr [ #:from from #:value value #:data data #:gas gas #:block block #:static static?]) → call-result? world : list? addr : exact-nonnegative-integer? from : exact-nonnegative-integer? = DEFAULT-CALLER value : exact-nonnegative-integer? = 0 data : (listof byte?) = '() gas : exact-nonnegative-integer? = 30000000 block : list? = CALL-BLOCK static? : boolean? = #f
(define r (call world addr #:data (encode-call "total()"))) (call-result-outcome r) ; 'return | 'revert | ... (car (abi-decode (list "uint256") (call-result-return r)))
struct
(struct call-result (outcome return world gas-left err logs) #:transparent) outcome : symbol? return : (listof byte?) world : list? gas-left : exact-nonnegative-integer? err : (or/c #f string?) logs : list?
3.7 Lower-level execution
define-evm-property builds on two run functions and their result structs. You can call them directly to reproduce a counterexample or to script an ad-hoc run.
procedure
(run-fragment code [ #:stack stack #:gas gas #:memory memory #:world world #:msg msg #:block block #:tx tx #:orig-storage orig #:fuel fuel #:trace? trace?]) → frag-run? code : (listof byte?) stack : list? = '() gas : exact-nonnegative-integer? = 1000000 memory : list? = '() world : list? = '() msg : any/c = #f block : any/c = #f tx : any/c = #f orig : list? = '() fuel : exact-nonnegative-integer? = 1000000 trace? : boolean? = #f
struct
(struct frag-run (pre post trace outcome err) #:transparent) pre : any/c post : any/c trace : (or/c #f list?) outcome : symbol? err : (or/c #f string?)
struct
(struct txn-run (world0 world1 ok? gas-used logs outcome err) #:transparent) world0 : list? world1 : list? ok? : boolean? gas-used : exact-nonnegative-integer? logs : list? outcome : (or/c 'success 'revert 'error) err : (or/c #f string?)
procedure
(machine-with-field m tag arg ...) → any/c
m : any/c tag : symbol? arg : any/c
value
CALL-BLOCK : list?
3.8 Observation vocabulary
Pure readers for #:pre / #:post / #:invariant bodies. All are total on well-formed machines / worlds.
3.8.1 Machine fields
procedure
(pc m) → exact-nonnegative-integer?
m : any/c
procedure
(gas m) → exact-nonnegative-integer?
m : any/c
procedure
(the-stack m) → list?
m : any/c
procedure
(mem-of m) → any/c
m : any/c
procedure
(code-of m) → list?
m : any/c
procedure
(world-of m) → list?
m : any/c
procedure
(return-data m) → list?
m : any/c
procedure
(logs-of m) → list?
m : any/c
procedure
(refund-of m) → exact-integer?
m : any/c
procedure
(halt-of m) → any/c
m : any/c
3.8.2 Stack
procedure
(stack m i) → exact-nonnegative-integer?
m : any/c i : exact-nonnegative-integer?
procedure
(stack-top m) → exact-nonnegative-integer?
m : any/c
procedure
(stack-depth m) → exact-nonnegative-integer?
m : any/c
procedure
(stack-empty? m) → boolean?
m : any/c
3.8.3 Outcome
procedure
(running? m) → boolean?
m : any/c
procedure
(halted? m) → boolean?
m : any/c
procedure
(stopped? m) → boolean?
m : any/c
procedure
(returned? m) → boolean?
m : any/c
procedure
(reverted? m) → boolean?
m : any/c
procedure
(out-of-gas? m) → boolean?
m : any/c
procedure
(exception? m) → boolean?
m : any/c
procedure
(halt-tag m) → symbol?
m : any/c
3.8.4 Return data and memory
procedure
m : any/c
procedure
(return-word m) → exact-nonnegative-integer?
m : any/c
procedure
(return-size m) → exact-nonnegative-integer?
m : any/c
procedure
(mem-word m off) → exact-nonnegative-integer?
m : any/c off : exact-nonnegative-integer?
procedure
(mem-byte m off) → byte?
m : any/c off : exact-nonnegative-integer?
procedure
m : any/c off : exact-nonnegative-integer? len : exact-nonnegative-integer?
procedure
(memory-size m) → exact-nonnegative-integer?
m : any/c
3.8.5 World, accounts, and storage
procedure
(account-of world addr) → list?
world : list? addr : exact-nonnegative-integer?
procedure
(exists? world addr) → boolean?
world : list? addr : exact-nonnegative-integer?
procedure
(balance-of world addr) → exact-nonnegative-integer?
world : list? addr : exact-nonnegative-integer?
procedure
(nonce-of world addr) → exact-nonnegative-integer?
world : list? addr : exact-nonnegative-integer?
procedure
world : list? addr : exact-nonnegative-integer?
procedure
(storage-at world addr) → list?
world : list? addr : exact-nonnegative-integer?
procedure
(empty-account? world addr) → boolean?
world : list? addr : exact-nonnegative-integer?
procedure
(sload world addr key) → exact-nonnegative-integer?
world : list? addr : exact-nonnegative-integer? key : exact-nonnegative-integer?
procedure
(m-balance m addr) → exact-nonnegative-integer?
m : any/c addr : exact-nonnegative-integer?
procedure
(m-nonce m addr) → exact-nonnegative-integer?
m : any/c addr : exact-nonnegative-integer?
procedure
(m-sload m addr key) → exact-nonnegative-integer?
m : any/c addr : exact-nonnegative-integer? key : exact-nonnegative-integer?
procedure
(m-storage-at m addr) → list?
m : any/c addr : exact-nonnegative-integer?
3.8.6 Constants
value
UINT256-MAX : exact-nonnegative-integer?
value
WORD-MOD : exact-nonnegative-integer?
3.9 Generators
EVM-tuned, edge-biased, shrinkable rackcheck generators for #:given. Edge bias matters: bugs cluster at 0, 1, 2^255, MAX-U256 and byte/word boundaries, so the defaults sample those heavily while still covering the uniform range.
value
value
value
value
procedure
lo : exact-nonnegative-integer? hi : exact-nonnegative-integer?
value
procedure
n : exact-nonnegative-integer? = 64
value
procedure
depth : exact-nonnegative-integer? = 8
procedure
k : exact-nonnegative-integer?
value
procedure
n : exact-nonnegative-integer? = 6
procedure
code : (listof byte?) = '()
procedure
(gen-world-with contract addr [ #:storage storage #:caller caller #:caller-balance caller-balance]) → gen? contract : (listof byte?) addr : exact-nonnegative-integer? storage : gen? = (gen-store) caller : exact-nonnegative-integer? = 0 caller-balance : gen? = gen-word
value
procedure
max-args : exact-nonnegative-integer? = 3
procedure
selector : (listof byte?) arg : any/c
All of rackcheck’s combinators (gen:integer-in, gen:one-of, gen:map, gen:frequency, gen:tuple, …) are re-exported, so you can build custom generators inline.
3.10 Running the DSL’s own tests
The DSL’s tests — correct properties passing, buggy ones falsified and shrunk, each clause form, a transaction property, reproducibility by seed — live under "tests/pbt/":
; raco test tests/pbt/