Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Information Theoretic Limits of Data Shuffling for Distributed Learning   Data shuffling is one of the fundamental building blocks for distributed
learning algorithms, that increases the statistical gain for each step of the
learning process. In each iteration, different shuffled data points are
assigned by a central node to a distributed set of workers to perform local
computations, which leads to communication bottlenecks. The focus of this paper
is on formalizing and understanding the fundamental information-theoretic
trade-off between storage (per worker) and the worst-case communication
overhead for the data shuffling problem. We completely characterize the
information theoretic trade-off for $K=2$, and $K=3$ workers, for any value of
storage capacity, and show that increasing the storage across workers can
reduce the communication overhead by leveraging coding. We propose a novel and
systematic data delivery and storage update strategy for each data shuffle
iteration, which preserves the structural properties of the storage across the
workers, and aids in minimizing the communication overhead in subsequent data
shuffling iterations.

15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph
26
The categories of this paper are: bayes-an
17
The categories of this paper are: q-fin
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
31
The categories of this paper are: ao-sci
28
The categories of this paper are: alg-geom
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
29
The categories of this paper are: funct-an
19
The categories of this paper are: eess
32
The categories of this paper are: atom-ph
18
The categories of this paper are: econ