This page contains supplemental information for the paper:
Benchmarking Derivative-Free Optimization Algorithms
by J. Moré and S. Wild. SIAM J. Optimization, Vol. 20 (1), pp.172-191, 2009.
Mathematics and Computer Science Division,
Argonne National Laboratory,
Users interested in the use of the techniques described
in this paper (data and performance profiles)
for benchmarking derivative-free solvers should consult the
results of our (ever-changing)
shootout of derivative-free solvers. We hope to
update these results as additional codes are
developed and submitted.
We are currently soliciting problems and solvers for two new efforts:
The following information (used in ) is provided to encourage
the benchmarking of derivative-free optimization solvers.
Additional solvers, application problems,
comments, questions, and suggestions should be addressed to Stefan Wild at
Plotting the Profiles
We provide the following Matlab scripts for producing basic data and performance profiles from data:
Code for plotting a basic data profile.
Code for plotting a basic performance profile.
The following source files were used to define the benchmark problems in :
Code for evaluating the 22 CUTEr problems considered (needs dfovec)
Code producing component vectors for the 22 CUTEr problems considered.
Code specifying the standard starting points.
Data file specifying the benchmark problem set P through the
integer parameters (nprob, n, m, ns).
Additional details concerning the problems may be found on the
The following solvers were used to obtain the performance data in :
Source code for this Nelder-Mead algorithm can be obtained via the
Computation Toolbox of N Highman.
Source code and documentation for the APPSPACK 5.0.1
solver can be obtained via the APPSPACK web page.
Source code for this trust-region algorithm can be obtained directly from