  Running neural networks (NNs) on microcontroller units (MCUs) is becoming
increasingly important, but is very difficult due to the tiny SRAM size of MCU.
Prior work proposes many algorithm-level techniques to reduce NN memory
footprints, but all at the cost of sacrificing accuracy and generality, which
disqualifies MCUs for many important use cases. We investigate a system
solution for MCUs to execute NNs out of core: dynamically swapping NN data
chunks between an MCU's tiny SRAM and its large, low-cost external flash.
Out-of-core NNs on MCUs raise multiple concerns: execution slowdown, storage
wear out, energy consumption, and data security. We present a study showing
that none is a showstopper; the key benefit -- MCUs being able to run large NNs
with full accuracy and generality -- triumphs the overheads. Our findings
suggest that MCUs can play a much greater role in edge intelligence.
