prove the speed or quality of training, with most ending in failure. The main survivors were <a href="https://arxiv.org/pdf/1503.02531.pdf">knowledge distillation (Hinton, Vinyals &amp; Dean, 2015)</a>, <a href="https://arxiv.org/pdf/1803.05407.pdf">stochastic weight averaging (SWA) (Izmailov, Podoprikhin, Garipov, Vetrov &amp; Wilson, 2019)</a>, <em>auxiliary training targets</em> (<a href="https://arxiv.org/pdf/1902.10565.pdf">Wu, 2020</a>; and <a href="https://medium.com/oracledevs/lessons-from-alphazero