  The rapid uptake of intelligent applications is pushing deep learning (DL)
capabilities to Internet-of-Things (IoT). Despite the emergence of new tools
for embedding deep neural networks (DNNs) into IoT devices, providing
satisfactory Quality of Experience (QoE) to users is still challenging due to
the heterogeneity in DNN architectures, IoT devices, and user preferences. This
paper studies automated customization for DL inference on IoT devices (termed
as on-thing inference), and our goal is to enhance user QoE by configuring the
on-thing inference with an appropriate DNN for users under different usage
scenarios. The core of our method is a DNN selection module that learns user
QoE patterns on-the-fly and identifies the best-fit DNN for on-thing inference
with the learned knowledge. It leverages a novel online learning algorithm,
NeuralUCB, that has excellent generalization ability for handling various user
QoE patterns. We also embed the knowledge transfer technique in NeuralUCB to
expedite the learning process. However, NeuralUCB frequently solicits QoE
ratings from users, which incurs non-negligible inconvenience. To address this
problem, we design feedback solicitation schemes to reduce the number of QoE
solicitations while maintaining the learning efficiency of NeuralUCB. A
pragmatic problem, aggregated QoE, is further investigated to improve the
practicality of our framework. We conduct experiments on both synthetic and
real-world data. The results indicate that our method efficiently learns the
user QoE pattern with few solicitations and provides drastic QoE enhancement
for IoT devices.
