  We introduce COCO, an open source platform for Comparing Continuous
Optimizers in a black-box setting. COCO aims at automatizing the tedious and
repetitive task of benchmarking numerical optimization algorithms to the
greatest possible extent. The platform and the underlying methodology allow to
benchmark in the same framework deterministic and stochastic solvers for both
single and multiobjective optimization. We present the rationales behind the
(decade-long) development of the platform as a general proposition for
guidelines towards better benchmarking. We detail underlying fundamental
concepts of COCO such as the definition of a problem as a function instance,
the underlying idea of instances, the use of target values, and runtime defined
by the number of function calls as the central performance measure. Finally, we
give a quick overview of the basic code structure and the currently available
test suites.
