Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Orthogonalising gradients to speed up neural network optimisation   The optimisation of neural networks can be sped up by orthogonalising the
gradients before the optimisation step, ensuring the diversification of the
learned representations. We orthogonalise the gradients of the layer's
components/filters with respect to each other to separate out the intermediate
representations. Our method of orthogonalisation allows the weights to be used
more flexibly, in contrast to restricting the weights to an orthogonalised
sub-space. We tested this method on ImageNet and CIFAR-10 resulting in a large
decrease in learning time, and also obtain a speed-up on the semi-supervised
learning BarlowTwins. We obtain similar accuracy to SGD without fine-tuning and
better accuracy for na\"ively chosen hyper-parameters.

28
The categories of this paper are: alg-geom
11
The categories of this paper are: hep-lat
29
The categories of this paper are: funct-an
0
The categories of this paper are: hep-ph
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
19
The categories of this paper are: eess
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
17
The categories of this paper are: q-fin
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
6
The categories of this paper are: astro-ph