">KataGo, Wu (2020)</a> makes use of <a href="https://arxiv.org/pdf/1803.05407.pdf">stochastic weight averaging (SWA) (Izmailov, Podoprikhin, Garipov, Vetrov &amp; Wilson, 2019)</a> to generate stronger self-play training data. Experiments showed that Wu's decaying weight schedule also performed best in ChessCoach and added approximately 50 to 100 Elo in strength to networks in early training. The process of training, with thousands of weights continually being updated, is like traveling in a highly dimensi