  The emergence of various intelligent mobile applications demands the
deployment of powerful deep learning models at resource-constrained mobile
devices. The device-edge co-inference framework provides a promising solution
by splitting a neural network at a mobile device and an edge computing server.
In order to balance the on-device computation and the communication overhead,
the splitting point needs to be carefully picked, while the intermediate
feature needs to be compressed before transmission. Existing studies decoupled
the design of model splitting, feature compression, and communication, which
may lead to excessive resource consumption of the mobile device. In this paper,
we introduce an end-to-end architecture, named BottleNet++, that consists of an
encoder, a non-trainable channel layer, and a decoder for more efficient
feature compression and transmission. The encoder and decoder essentially
implement joint source-channel coding via convolutional neural networks (CNNs),
while explicitly considering the effect of channel noise. By exploiting the
strong sparsity and the fault-tolerant property of the intermediate feature in
a deep neural network (DNN), BottleNet++ achieves a much higher compression
ratio than existing methods. Furthermore, by providing the channel condition to
the encoder as an input, our method enjoys a strong generalization ability in
different channel conditions. Compared with merely transmitting intermediate
data without feature compression, BottleNet++ achieves up to 64x bandwidth
reduction over the additive white Gaussian noise channel and up to 256x bit
compression ratio in the binary erasure channel, with less than 2% reduction in
accuracy. With a higher compression ratio, BottleNet++ enables splitting a DNN
at earlier layers, which leads to up to 3x reduction in on-device computation
compared with other compression methods.
