  Stochastic scientific models and machine learning optimization estimators
have a large number of variables; hence computing large sparse Jacobians and
Hessians is important. Algorithmic differentiation (AD) greatly reduces the
programming effort required to obtain the sparsity patterns and values for
these matrices. We present forward, reverse, and subgraph methods for computing
sparse Jacobians and Hessians. Special attention is given the the subgraph
method because it is new. The coloring and compression steps are not necessary
when computing sparse Jacobians and Hessians using subgraphs. Complexity
analysis shows that for some problems the subgraph method is expected to be
much faster. We compare C++ operator overloading implementations of the methods
in the ADOL-C and CppAD software packages using some of the MINPACK-2 test
problems. The experiments are set up in a way that makes them easy to run on
different hardware, different systems, different compilers, other test problem
and other AD packages. The setup time is the time to record the graph, compute
sparsity, coloring, compression, and optimization of the graph. If the setup is
necessary for each evaluation, the subgraph implementation has similar run
times for sparse Jacobians and faster run times for sparse Hessians.
