Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: NanoBatch DPSGD: Exploring Differentially Private learning on ImageNet
  with low batch sizes on the IPU   Differentially private SGD (DPSGD) has recently shown promise in deep
learning. However, compared to non-private SGD, the DPSGD algorithm places
computational overheads that can undo the benefit of batching in GPUs.
Microbatching is a standard method to alleviate this and is fully supported in
the TensorFlow Privacy library (TFDP). However, this technique, while improving
training times also reduces the quality of the gradients and degrades the
classification accuracy. Recent works that for example use the JAX framework
show promise in also alleviating this but still show degradation in throughput
from non-private to private SGD on CNNs, and have not yet shown ImageNet
implementations. In our work, we argue that low batch sizes using group
normalization on ResNet-50 can yield high accuracy and privacy on Graphcore
IPUs. This enables DPSGD training of ResNet-50 on ImageNet in just 6 hours (100
epochs) on an IPU-POD16 system.

13
The categories of this paper are: quant-ph
2
The categories of this paper are: math
30
The categories of this paper are: q-alg
14
The categories of this paper are: math-ph
17
The categories of this paper are: q-fin
32
The categories of this paper are: atom-ph
26
The categories of this paper are: bayes-an
12
The categories of this paper are: nucl-th
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
21
The categories of this paper are: adap-org
8
The categories of this paper are: hep-ex
0
The categories of this paper are: hep-ph
28
The categories of this paper are: alg-geom
24
The categories of this paper are: dg-ga
3
The categories of this paper are: physics