S. M. Wild, R. B. Gramacy, and M. A. Taddy, "Variable Selection and Sensitivity Analysis via Dynamic Trees with an Application to Computer Code Performance Tuning," Journal, vol. 7, no. 1, The Annals of Applied Statistics, 2011, pp. 51-80. Also Preprint ANL/MCS-P1961-1011, October 2011. [pdf]
We show how the newly developed dynamic tree model can support variable selection and a sensitivity analysis of inputs, two tasks usually requiring disparate model structure. To this end, we adapt methods used in conjunction with static tree models and Gaussian process models (GPs). Compared with static trees, this approach allows for dynamic (sequential) variable selection with fully Bayesian evidence and sensitivity indices not previously enjoyed. Compared with GPs, it facilitates sensitivity analysis and variable selection on problems that are several orders of magnitude larger in size than previously possible. Importantly, this allows a unifying approach to mixed data applications. The same computational techniques can be used, for example, in regression and classification applications with trivial modification. We illustrate our methodology on several instructive data sets and then analyze automatic computer code tuning that combines classification and regression elements, demanding both variable selection and input sensitivity analysis on large data sets.