  Algorithmic differentiation (AD) allows exact computation of derivatives
given only an implementation of an objective function. Although many AD tools
are available, a proper and efficient implementation of AD methods is not
straightforward. The existing tools are often too different to allow for a
general test suite. In this paper, we compare fifteen ways of computing
derivatives including eleven automatic differentiation tools implementing
various methods and written in various languages (C++, F#, MATLAB, Julia and
Python), two symbolic differentiation tools, finite differences, and
hand-derived computation.
  We look at three objective functions from computer vision and machine
learning. These objectives are for the most part simple, in the sense that no
iterative loops are involved, and conditional statements are encapsulated in
functions such as {\tt abs} or {\tt logsumexp}. However, it is important for
the success of algorithmic differentiation that such `simple' objective
functions are handled efficiently, as so many problems in computer vision and
machine learning are of this form.
  Of course, our results depend on programmer skill, and familiarity with the
tools. However, we contend that this paper presents an important datapoint: a
skilled programmer devoting roughly a week to each tool produced the timings we
present. We have made our implementations available as open source to allow the
community to replicate and update these benchmarks.
