Contains a few configurations of the GTP benchmark programs, code for continuously testing their performance as Racket changes, and data from past runs.
To run a performance test on your machine:
- Clone the repo
Either via git clone, or better yet via raco pkg install --clone gtp-checkup
See results on STDOUT, with a summary of errors at the bottom.
The Makefile compiles and runs all scripts that match the pattern benchmarks/*/main.rkt. Each compile job and each run job has a time limit. Run racket main.rkt --help for more information.
Changed in version 0.1: Changed style of benchmarks to focus on one worst-case configuration instead of spot-checking N mixed ones.
Changed in version 0.1: Renamed quadBG to quadU and replaced quadMB with quadT. The version notes in the GTP Benchmarks documentation explain why. Updated other benchmarks to match the GTP benchmarks versions.
(gtp-checkup bin #:iterations i #:timeout time-limit) → void? bin : racket-bin-dir/c i : (or/c #f exact-positive-integer?) time-limit : (or/c #f (cons/c exact-positive-integer? exact-positive-integer?))
The value of i determines the number of times to run each benchmark configuration.
The plots in this section show the performance of different snapshots of the Racket language. A snapshot begins with one commit to the racket/racket repository and includes contemporaneous commits to other main distribution repositories.
Quick guide to plots:
x-axis = time, commits occur from left to right
y-axis = runtime (seconds), lower is better.
Each plot shows all commits for one benchmark, each commit is represented by one point.
If one commit is much worse than the previous one, then the line is labeled with a short hash of the new/bad commit.
The points labeled typed-worst-case are for a configuration where: (1) every module is typed and (2) every import is guarded with contracts via require/typed.
The benchmarks come from GTP benchmarks v1.0.