Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: High-Throughput CNN Inference on Embedded ARM big.LITTLE Multi-Core
  Processors   IoT Edge intelligence requires Convolutional Neural Network (CNN) inference
to take place in the edge devices itself. ARM big.LITTLE architecture is at the
heart of prevalent commercial edge devices. It comprises of single-ISA
heterogeneous cores grouped into multiple homogeneous clusters that enable
power and performance trade-offs. All cores are expected to be simultaneously
employed in inference to attain maximal throughput. However, high communication
overhead involved in parallelization of computations from convolution kernels
across clusters is detrimental to throughput. We present an alternative
framework called Pipe-it that employs pipelined design to split convolutional
layers across clusters while limiting parallelization of their respective
kernels to the assigned cluster. We develop a performance-prediction model that
utilizes only the convolutional layer descriptors to predict the execution time
of each layer individually on all permitted core configurations (type and
count). Pipe-it then exploits the predictions to create a balanced pipeline
using an efficient design space exploration algorithm. Pipe-it on average
results in a 39% higher throughput than the highest antecedent throughput.

31
The categories of this paper are: ao-sci
8
The categories of this paper are: hep-ex
11
The categories of this paper are: hep-lat
5
The categories of this paper are: gr-qc
14
The categories of this paper are: math-ph
25
The categories of this paper are: patt-sol
33
The categories of this paper are: plasm-ph
20
The categories of this paper are: acc-phys
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
7
The categories of this paper are: hep-th
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
9
The categories of this paper are: nlin