  Calibrating simulation models that take large quantities of multi-dimensional
data as input is a hard simulation optimization problem. Existing adaptive
sampling strategies offer a methodological solution. However, they may not
sufficiently reduce the computational cost for estimation and solution
algorithm's progress within a limited budget due to extreme noise levels and
heteroskedasticity of system responses. We propose integrating stratification
with adaptive sampling for the purpose of efficiency in optimization.
Stratification can exploit local dependence in the simulation inputs and
outputs. Yet, the state-of-the-art does not provide a full capability to
adaptively stratify the data as different solution alternatives are evaluated.
We devise two procedures for data-driven calibration problems that involve a
large dataset with multiple covariates to calibrate models within a fixed
overall simulation budget. The first approach dynamically stratifies the input
data using binary trees, while the second approach uses closed-form solutions
based on linearity assumptions between the objective function and concomitant
variables. We find that dynamical adjustment of stratification structure
accelerates optimization and reduces run-to-run variability in generated
solutions. Our case study for calibrating a wind power simulation model, widely
used in the wind industry, using the proposed stratified adaptive sampling,
shows better-calibrated parameters under a limited budget.
