Gradual Typing Across the Spectrum

Benchmarks

Reticulated Python

https://github.com/nuprl/retic_performance?path=benchmarks

Suite of Python programs adapted from: case studies reported by Vitousek, Kent, Siek, and Baker; the module-level evaluation of Big Types in Little Runtime; and open-source programs. Each function in these benchmarks may be typed or untyped. In other words, for a program with 10 functions the benchmark explores 1024 configurations of gradual typing.

Appeared in:

Typed Racket

https://github.com/bennn/gtp-benchmarks

Suite of Typed Racket programs adapted from open-source projects. Each module in these benchmarks may be typed or untyped.

Appeared in:

Grift

https://github.com/Gradual-Typing/benchmarks

Suite of programs adapted from different sources including "The Computer Language Benchmarks Game", "Kernighan and Van Wyk benchmarks", "Numerical Recipes in C", "John C. Hull's Options, Futures, and Other Derivatives", and "Paul Graham's book ANSI Common Lisp". Each benchmark has two Grift and Racket versions, fully typed and fully untyped. They are also ported to OCaml, Chez Scheme, and Gambit Scheme. They are intended to benchmark systems that support fine-grained gradual typing where each type place holder can be fully typed, partially typed, or untyped. Currently the benchmarking infrastructure supports Grift only for the gradual typing experiments but also benchmarks fully typed and fully untyped Grift against other statically typed and dynamically typed languages mentioned before. There are plans to support other fine-grained gradually-typed systems such as HiggsCheck and Reticulated Python.

Appeared in: