  We show that reverse mode automatic differentiation and symbolic
differentiation are equivalent in the sense that they both perform the same
operations when computing derivatives. This is in stark contrast to the common
claim that they are substantially different. The difference is often
illustrated by claiming that symbolic differentiation suffers from "expression
swell" whereas automatic differentiation does not. Here, we show that this
statement is not true. "Expression swell" refers to the phenomenon of a much
larger representation of the derivative as opposed to the representation of the
original function.
