This page contains supplemental information for the paper:

Benchmarking Derivative-Free Optimization Algorithms
by J. Moré and S. Wild. SIAM J. Optimization, Vol. 20 (1), pp.172-191, 2009.
[More information]
Formerly
Mathematics and Computer Science Division,
Argonne National Laboratory,
Preprint ANL/MCS-P1471-1207,
May 2008.

Users interested in the use of the techniques described
in this paper (data and performance profiles)
for benchmarking derivative-free solvers should consult the
results of our (ever-changing)
shootout of derivative-free solvers. We hope to
update these results as additional codes are
developed and submitted.

We are currently soliciting problems and solvers for two new efforts:

Simulation-based problems

Bound-constrained problems

The following information (used in [1]) is provided to encourage
the benchmarking of derivative-free optimization solvers.
Additional solvers, application problems,
comments, questions, and suggestions should be addressed to Stefan Wild at
.

Plotting the Profiles

We provide the following Matlab scripts for producing basic data and performance profiles from data:

data_profile
[MATLAB]
Code for plotting a basic data profile.

perf_profile
[MATLAB]
Code for plotting a basic performance profile.

Benchmark Problems

The following source files were used to define the benchmark problems in [1]:

calfun
[Fortran] [MATLAB]
Code for evaluating the 22 CUTEr problems considered (needs dfovec)

dfovec
[Fortran] [MATLAB]
Code producing component vectors for the 22 CUTEr problems considered.

dfoxs
[Fortran] [MATLAB]
Code specifying the standard starting points.

dfo.dat
[Data file]
Data file specifying the benchmark problem set P through the
integer parameters (nprob, n, m, ns).

Additional details concerning the problems may be found on the
CUTEr webpage.

Sample Solvers

The following solvers were used to obtain the performance data in [1]:

nmsmax
[Matlab]
Source code for this Nelder-Mead algorithm can be obtained via the
Computation Toolbox of N Highman.

appspack_serial [C++]
Source code and documentation for the APPSPACK 5.0.1
solver can be obtained via the APPSPACK web page.

newuoa [Fortran]
Source code for this trust-region algorithm can be obtained directly from
Mike Powell.