Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: First-Generation Inference Accelerator Deployment at Facebook   In this paper, we provide a deep dive into the deployment of inference
accelerators at Facebook. Many of our ML workloads have unique characteristics,
such as sparse memory accesses, large model sizes, as well as high compute,
memory and network bandwidth requirements. We co-designed a high-performance,
energy-efficient inference accelerator platform based on these requirements. We
describe the inference accelerator platform ecosystem we developed and deployed
at Facebook: both hardware, through Open Compute Platform (OCP), and software
framework and tooling, through Pytorch/Caffe2/Glow. A characteristic of this
ecosystem from the start is its openness to enable a variety of AI accelerators
from different vendors. This platform, with six low-power accelerator cards
alongside a single-socket host CPU, allows us to serve models of high
complexity that cannot be easily or efficiently run on CPUs. We describe
various performance optimizations, at both platform and accelerator level,
which enables this platform to serve production traffic at Facebook. We also
share deployment challenges, lessons learned during performance optimization,
as well as provide guidance for future inference hardware co-design.

23
The categories of this paper are: solv-int
2
The categories of this paper are: math
8
The categories of this paper are: hep-ex
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
22
The categories of this paper are: chao-dyn
16
The categories of this paper are: stat
30
The categories of this paper are: q-alg
32
The categories of this paper are: atom-ph
27
The categories of this paper are: comp-gas
34
The categories of this paper are: chem-ph
36
The categories of this paper are: cmp-lg
26
The categories of this paper are: bayes-an
12
The categories of this paper are: nucl-th
4
The categories of this paper are: cond-mat
33
The categories of this paper are: plasm-ph