  The prevalence of Internet of things (IoT) devices and abundance of sensor
data has created an increase in real-time data processing such as recognition
of speech, image, and video. While currently such processes are offloaded to
the computationally powerful cloud system, a localized and distributed approach
is desirable because (i) it preserves the privacy of users and (ii) it omits
the dependency on cloud services. However, IoT networks are usually composed of
resource-constrained devices, and a single device is not powerful enough to
process real-time data. To overcome this challenge, we examine data and model
parallelism for such devices in the context of deep neural networks. We propose
Musical Chair to enable efficient, localized, and dynamic real-time recognition
by harvesting the aggregated computational power from the resource-constrained
devices in the same IoT network as input sensors. Musical chair adapts to the
availability of computing devices at runtime and adjusts to the inherit
dynamics of IoT networks. To demonstrate Musical Chair, on a network of
Raspberry PIs (up to 12) each connected to a camera, we implement a
state-of-the-art action recognition model for videos and two recognition models
for images. Compared to the Tegra TX2, an embedded low-power platform with a
six-core CPU and a GPU, our distributed action recognition system achieves not
only similar energy consumption but also twice the performance of the TX2.
Furthermore, in image recognition, Musical Chair achieves similar performance
and saves dynamic energy.
