GTP Checkup
1 Basic Usage
2 Version History
3 Checkup API
gtp-checkup
import-benchmark
gtp-checkup-logger
racket-bin-dir/  c
4 Checkup data for racket/  racket
4.1 acquire
4.2 dungeon
4.3 forth
4.4 fsm
4.5 fsmoo
4.6 gregor
4.7 jpeg
4.8 kcfa
4.9 lnm
4.10 mbta
4.11 morsecode
4.12 quad  T
4.13 quad  U
4.14 sieve
4.15 snake
4.16 suffixtree
4.17 synth
4.18 take5
4.19 tetris
4.20 zombie
4.21 zordoz
7.4

GTP Checkup

Source: https://github.com/bennn/gtp-checkup

Contains a few configurations of the GTP benchmark programs, code for continuously testing their performance as Racket changes, and data from past runs.

1 Basic Usage

To run a performance test on your machine:

The Makefile compiles and runs all scripts that match the pattern benchmarks/*/main.rkt. Each compile job and each run job has a time limit. Run racket main.rkt --help for more information.

2 Version History

3 Checkup API

Main entry point, for running the benchmarks and for importing a new benchmark.

procedure

(gtp-checkup bin    
  #:iterations i    
  #:timeout time-limit)  void?
  bin : racket-bin-dir/c
  i : (or/c #f exact-positive-integer?)
  time-limit : (or/c #f (cons/c exact-positive-integer? exact-positive-integer?))
Run the benchmarks using the given executables. Post results to gtp-checkup-logger at the 'info level.

The value of i determines the number of times to run each benchmark configuration.

The value of time-limit determines how long to wait for each configuration to compile and run. If time-limit is a cons-pair, then its car is the compile-time limit and its cdr is the run-time limit.

procedure

(import-benchmark dir)  void?

  dir : directory-exists?
Extract code from the given directory and create a new folder in the "benchmarks/" directory of the gtp-checkup repo. The directory dir must have a different name than any existing benchmark, and it must match the subdirectory structure of a GTP benchmark.

Receives messages about running times and timeouts. To subscribe to this logger, set PLTSTDERR="error info@gtp-checkup"

procedure

(racket-bin-dir/c x)  boolean?

  x : any/c
Predicate for a directory that exists and contains racket and raco executables.

4 Checkup data for racket/racket

The plots in this section show the performance of different snapshots of the Racket language. A snapshot begins with one commit to the racket/racket repository and includes contemporaneous commits to other main distribution repositories.

Quick guide to plots:

Each subsection has data for one benchmark. The data comes from the following machines:

The points labeled typed-worst-case are for a configuration where: (1) every module is typed and (2) every import is guarded with contracts via require/typed.

The benchmarks come from GTP benchmarks v1.0.

4.1 acquire

imageimage

4.2 dungeon

imageimage

4.3 forth

imageimage

4.4 fsm

imageimage

4.5 fsmoo

imageimage

4.6 gregor

imageimage

4.7 jpeg

imageimage

4.8 kcfa

imageimage

4.9 lnm

imageimage

4.10 mbta

imageimage

4.11 morsecode

imageimage

4.12 quadT

imageimage

4.13 quadU

imageimage

4.14 sieve

imageimage

4.15 snake

imageimage

4.16 suffixtree

imageimage

4.17 synth

imageimage

4.18 take5

imageimage

4.19 tetris

imageimage

4.20 zombie

imageimage

4.21 zordoz

imageimage