  We review strategies for differentiating matrix-based computations, and
derive symbolic and algorithmic update rules for differentiating expressions
containing the Cholesky decomposition. We recommend new `blocked' algorithms,
based on differentiating the Cholesky algorithm DPOTRF in the LAPACK library,
which uses `Level 3' matrix-matrix operations from BLAS, and so is
cache-friendly and easy to parallelize. For large matrices, the resulting
algorithms are the fastest way to compute Cholesky derivatives, and are an
order of magnitude faster than the algorithms in common usage. In some
computing environments, symbolically-derived updates are faster for small
matrices than those based on differentiating Cholesky algorithms. The symbolic
and algorithmic approaches can be combined to get the best of both worlds.
