The RNN hierarchy can be "collapsed" into a single RNN, by "distilling" a higher level "chunker" network into a lower level "automatizer" network. In 1993, a chunker solved a deep learning task whose CAP depth exceeded 1000.
