Tutorial: testing Solidity contracts with evm-redex
Versão em português: Tutorial: testando contratos Solidity com evm-redex.
This tutorial walks, from scratch, through writing a small Solidity contract, compiling it, and testing it with evm-redex/pbt — the property-based testing DSL. Every snippet below is the real, runnable code under "tutorial/" — "counter.rkt", "token.rkt", "ballot.rkt", "auction.rkt", "purchase.rkt", and "explore.rkt"; run them all with raco test tutorial.
You will need solc (the Solidity compiler) to turn a .sol file into the JSON artifact the library reads; the committed artifacts under "tutorial/" were built with solc 0.8.33, so you can follow along without recompiling.
1 The idea in one paragraph
The library runs a contract’s bytecode on an executable specification of the EVM. So the loop is always the same: compile the .sol to bytecode with solc, load that artifact with read-artifact, deploy it into a fresh world, and then either call into it (to read state or send a message) or state a property with define-evm-property and let the engine try to falsify it over many generated inputs.
2 A first contract: Counter
Here is the "hello world" of stateful contracts — a counter, in "tutorial/contracts/Counter.sol":
"Counter.sol"
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;
contract Counter {
uint256 public count; // public -> free getter `count()`
function increment() public { count += 1; }
function add(uint256 n) public { count += n; }
function decrement() public {
require(count > 0, "underflow"); // a guard we can test for reverting
count -= 1;
}
}
2.1 Compile it
Ask solc for the bytecode and the ABI, as one JSON file:
solc --combined-json bin,bin-runtime,abi contracts/Counter.sol > Counter.json
2.2 Load and deploy
read-artifact pulls the creation and runtime bytecode (and the ABI) out of the JSON; deploy runs the constructor and gives you the deployed address and the resulting world.
(require evm-redex/pbt) (define ART (read-artifact "Counter.json" #:contract "Counter")) (define DEPLOYER 2703024129) (define DEP (deploy (artifact-creation ART) #:from DEPLOYER)) (define COUNTER (deploy-result-address DEP)) (define BASE (deploy-result-world DEP)) ; the world just after deployment
2.3 Read state
count is a public variable, so Solidity generates a count() getter. A call runs a message; call-result-return is its return data, which abi-decode turns back into a number:
(define (count-of world) (car (abi-decode (list "uint256") (call-result-return (call world COUNTER #:data (encode-call "count()")))))) (count-of BASE) ; => 0
2.4 Send transactions
encode-call builds the calldata for a function; call returns a call-result, and call-result-world is the world after the call. Thread that world through a couple of calls and read the count back:
(define CALLER 2964324353) (define (send world sig . args) (call-result-world (call world COUNTER #:from CALLER #:data (apply encode-call sig args)))) (let* ([w (send BASE "increment()")] [w (send w "add(uint256)" 5)]) (count-of w)) ; => 6
2.5 State a property
Concrete calls are fine for a sanity check, but the point of the library is to test a property over many generated inputs. define-evm-property in transaction mode (the #:call clause) runs a real transaction; #:given lists the generated inputs, and #:post receives the world before (w0) and after (w1) plus the transaction result:
(define (tx sig . args) (make-tx #:sender CALLER #:to COUNTER #:gas-limit 200000 #:gas-price 0 #:data (apply encode-call sig args))) (define-evm-property add-raises-count-by-n #:given ([n (gen-word-in 0 (expt 2 200))]) #:world BASE #:call (tx "add(uint256)" n) #:post (lambda (w0 w1 r) (= (count-of w1) (+ (count-of w0) n)))) (check-evm-property add-raises-count-by-n #:trials 50)
Running it prints:
✓ property add-raises-count-by-n passed 50 tests. |
A property can also assert that a call reverts. #:revert-when says "under this condition, the transaction must revert"; from a count of zero, decrement() always does:
(define-evm-property decrement-reverts-at-zero #:given () ; no generated inputs — a plain assertion #:world BASE ; count is 0 here #:call (tx "decrement()") #:revert-when #t) (check-evm-property decrement-reverts-at-zero #:trials 1)
3 A richer contract: Token
A minimal token adds balances, a guarded transfer, and an event — "tutorial/contracts/Token.sol":
"Token.sol"
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;
contract Token {
mapping(address => uint256) public balanceOf; // -> `balanceOf(address)`
uint256 public totalSupply;
event Transfer(address indexed from, address indexed to, uint256 value);
function mint(address to, uint256 amount) public {
totalSupply += amount;
balanceOf[to] += amount;
emit Transfer(address(0), to, amount);
}
function transfer(address to, uint256 amount) public returns (bool) {
require(balanceOf[msg.sender] >= amount, "insufficient balance");
balanceOf[msg.sender] -= amount;
balanceOf[to] += amount;
emit Transfer(msg.sender, to, amount);
return true;
}
}
Deploy it and, this time, build a world where ALICE already holds 1000 tokens (mint is unrestricted here, so any account can call it):
(define ART (read-artifact "Token.json" #:contract "Token")) (define ALICE 659918) (define BOB 723712) (define DEP (deploy (artifact-creation ART) #:from 3736076289)) (define TOKEN (deploy-result-address DEP)) (define (u256 r) (car (abi-decode (list "uint256") (call-result-return r)))) (define (bal w a) (u256 (call w TOKEN #:data (encode-call "balanceOf(address)" a)))) (define (total w) (u256 (call w TOKEN #:data (encode-call "totalSupply()")))) (define (send w from sig . args) (call-result-world (call w TOKEN #:from from #:data (apply encode-call sig args)))) (define MINTED (send (deploy-result-world DEP) ALICE "mint(address,uint256)" ALICE 1000)) (bal MINTED ALICE) ; => 1000
3.1 The property that matters: conservation
The invariant a token must never break is that a transfer moves value without creating or destroying any. This one property captures both the success and the revert paths, over amounts that straddle ALICE’s balance:
(define (tx from sig . args) (make-tx #:sender from #:to TOKEN #:gas-limit 200000 #:gas-price 0 #:data (apply encode-call sig args))) (define-evm-property transfer-conserves-supply #:given ([amount (gen-word-in 0 2000)]) ; straddles the 1000 balance #:world MINTED #:call (tx ALICE "transfer(address,uint256)" BOB amount) #:post (lambda (w0 w1 r) (and (= (total w1) (total w0)) ; supply never changes (if (eq? (txn-run-outcome r) 'revert) (= (bal w1 ALICE) (bal w0 ALICE)) ; rolled back (and (= (bal w1 ALICE) (- (bal w0 ALICE) amount)) (= (bal w1 BOB) (+ (bal w0 BOB) amount))))))) (check-evm-property transfer-conserves-supply #:trials 100)
And a focused revert property — a transfer of more than you own must fail:
(define-evm-property transfer-reverts-when-insufficient #:given ([amount (gen-word-in 1001 100000)]) ; always more than ALICE has #:world MINTED #:call (tx ALICE "transfer(address,uint256)" BOB amount) #:revert-when #t) (check-evm-property transfer-reverts-when-insufficient #:trials 50)
When a property fails, the engine shrinks the random inputs to a minimal counterexample and prints it — that shrunk case is the payoff of property-based testing. Try weakening transfer’s require in the contract and re-running: the conservation property falsifies with a tiny transfer.
4 More examples from Solidity by Example
The two contracts above cover the whole loop; the rest of this section applies it to the classic contracts from the Solidity documentation’s Solidity by Example. Each is a full runnable module under "tutorial/" — the Solidity source, the deployment, and the property that pins its central rule.
4.1 Voting
"tutorial/ballot.rkt" tests Ballot, the delegated-voting contract. Two things are new here: the constructor takes arguments (an array of bytes32 proposal names), which you ABI-encode and append to the creation bytecode; and a getter can return a struct, which you decode as a tuple.
(define ART (read-artifact "Ballot.json" #:contract "Ballot")) (define CHAIR 3298922497) ; proposal names are bytes32 — pad each to 32 bytes (define NAMES (list (name->b32 "alpha") (name->b32 "beta") (name->b32 "gamma"))) ; constructor args are ABI-encoded and appended to the creation code (define DEP (deploy (append (artifact-creation ART) (abi-encode (list "bytes32[]") (list NAMES))) #:from CHAIR)) (define BALLOT (deploy-result-address DEP))
The chairperson (the deployer) grants a right to vote, the voter casts it, and the tally updates — and the rule worth checking is that only the chairperson may grant voting rights, so a call from anyone else must revert:
(define OUTSIDER 50157932545) (define-evm-property only-chair-grants-rights #:given ([who gen-address]) #:world BASE #:call (make-tx #:sender OUTSIDER #:nonce 0 #:to BALLOT #:gas-limit 300000 #:gas-price 0 #:data (encode-call "giveRightToVote(address)" who)) #:revert-when #t) ; a non-chairperson caller always reverts (check-evm-property only-chair-grants-rights #:trials 30)
4.2 An open auction
"tutorial/auction.rkt" tests SimpleAuction. Its calls carry ether: make-tx’s #:value funds the bid. Accounts that bid need a balance, so we top a couple up by hand — the same thing deploy does for the deployer:
(require (only-in evm-redex world-ref world-set acct-balance acct-with-balance)) (define (fund w a wei) (world-set w a (acct-with-balance (world-ref w a) (+ (acct-balance (world-ref w a)) wei))))
The auction’s central rule is that a bid is accepted exactly when it beats the current highest, and then becomes the new highest — one property captures both the accept and the reject path:
(define-evm-property bid-raises-the-highest #:given ([v (gen-word-in 0 500)]) #:world W1 ; highest bid is 100 here #:call (make-tx #:sender BOB #:nonce (nonce-of W1 BOB) #:to AUCTION #:value v #:gas-limit 300000 #:gas-price 0 #:data (encode-call "bid()")) #:post (lambda (w0 w1 r) (if (> v (highest-bid w0)) (and (eq? (txn-run-outcome r) 'success) (= (highest-bid w1) v)) (eq? (txn-run-outcome r) 'revert)))) (check-evm-property bid-raises-the-highest #:trials 60)
4.3 Safe remote purchase
"tutorial/purchase.rkt" tests Purchase, a four-state escrow (Created → Locked → Release → Inactive). The constructor is payable — the seller locks twice the item value — so you deploy it with #:value:
(define DEP (deploy (artifact-creation ART) #:from SELLER #:value 200)) ; value = 100
The happy path is buyer confirmPurchase (matching the deposit) then confirmReceived. The rule worth pinning is that from the Locked state only the buyer can confirm receipt:
(define-evm-property only-buyer-confirms-receipt #:given () #:world LOCKED-W #:call (make-tx #:sender OUTSIDER #:nonce 0 #:to PURCHASE #:gas-limit 300000 #:gas-price 0 #:data (encode-call "confirmReceived()")) #:revert-when #t) (check-evm-property only-buyer-confirms-receipt #:trials 1)
The fourth Solidity by Example contract, the Micropayment Channel, is not included: it verifies an off-chain ECDSA signature with ecrecover, and producing that signature happens outside the EVM this library models — so testing it would need an off-chain signer, beyond the scope of this tutorial.
5 Exploring by hand with #lang evm-redex/sim
Properties are for checking; when you just want to poke at a contract, the transaction simulator reads like a script. "tutorial/explore.rkt" is a whole session — run it with racket tutorial/explore.rkt:
"explore.rkt"
#lang evm-redex/sim
.account ALICE balance=1eth
.account BOB
.deploy TOKEN from=ALICE code=@Token.json:Token
tx from=ALICE to=TOKEN sig="mint(address,uint256)" args=(ALICE, 1000)
tx from=ALICE to=TOKEN sig="transfer(address,uint256)" args=(BOB, 100)
tx from=BOB to=TOKEN sig="transfer(address,uint256)" args=(ALICE, 5000) ; reverts
call from=ALICE to=TOKEN sig="balanceOf(address)" args=(BOB) returns=uint256
It prints a receipt per transaction — with the Transfer event and the revert reason decoded for you — and a final state diff:
tx ALICE -> TOKEN |
status: success |
log: Transfer(from=0x0, to=0x4e8a…8f7, value=0x3e8) |
tx ALICE -> TOKEN |
status: success |
log: Transfer(from=0x4e8a…8f7, to=0x28e4…4c38, value=0x64) |
tx BOB -> TOKEN |
status: revert (Error("insufficient balance")) |
call TOKEN.balanceOf(address) = (100) |
|
--- final state --- |
state root: 0x4b40…1730 |
… |
6 Where to go next
The complete evm-redex/pbt reference — every clause of define-evm-property, the observation vocabulary, and the generators — is in Library reference: the property-based testing DSL.
The #lang evm-redex/sim scenario language is documented in #lang evm-redex/sim — simulating transactions.
Larger worked examples — real ERC-20/721, OpenZeppelin contracts, and a benchmark reproducing real audit bugs — live under "tests/" and are described in The evm-redex test suite: tutorial, tests, and examples.