Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models   The memory capacity of embedding tables in deep learning recommendation
models (DLRMs) is increasing dramatically from tens of GBs to TBs across the
industry. Given the fast growth in DLRMs, novel solutions are urgently needed,
in order to enable fast and efficient DLRM innovations. At the same time, this
must be done without having to exponentially increase infrastructure capacity
demands. In this paper, we demonstrate the promising potential of Tensor Train
decomposition for DLRMs (TT-Rec), an important yet under-investigated context.
We design and implement optimized kernels (TT-EmbeddingBag) to evaluate the
proposed TT-Rec design. TT-EmbeddingBag is 3 times faster than the SOTA TT
implementation. The performance of TT-Rec is further optimized with the batched
matrix multiplication and caching strategies for embedding vector lookup
operations. In addition, we present mathematically and empirically the effect
of weight initialization distribution on DLRM accuracy and propose to
initialize the tensor cores of TT-Rec following the sampled Gaussian
distribution. We evaluate TT-Rec across three important design space dimensions
-- memory capacity, accuracy, and timing performance -- by training MLPerf-DLRM
with Criteo's Kaggle and Terabyte data sets. TT-Rec achieves 117 times and 112
times model size compression, for Kaggle and Terabyte, respectively. This
impressive model size reduction can come with no accuracy nor training time
overhead as compared to the uncompressed baseline.

19
The categories of this paper are: eess
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
12
The categories of this paper are: nucl-th
16
The categories of this paper are: stat
35
The categories of this paper are: mtrl-th
26
The categories of this paper are: bayes-an
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
2
The categories of this paper are: math