  The emerging trend of deploying complex algorithms, such as Deep Neural
Networks (DNNs), increasingly poses strict memory and energy efficiency
requirements on Internet-of-Things (IoT) end-nodes. Mixed-precision
quantization has been proposed as a technique to minimize a DNN's memory
footprint and maximize its execution efficiency, with negligible end-to-end
precision degradation. In this work, we present a novel hardware and software
stack for energy-efficient inference of mixed-precision Quantized Neural
Networks (QNNs). We introduce Flex-V, a processor based on the RISC-V
Instruction Set Architecture (ISA) that features fused Mac&Load mixed-precision
dot product instructions; to avoid the exponential growth of the encoding space
due to mixed-precision variants, we encode formats into the Control-Status
Registers (CSRs). Flex-V core is integrated into a tightly-coupled cluster of
eight processors; in addition, we provide a full framework for the end-to-end
deployment of DNNs including a compiler, optimized libraries, and a
memory-aware deployment flow. Our results show up to 91.5 MAC/cycle and 3.26
TOPS/W on the cluster, implemented in a commercial 22nm FDX technology, with up
to 8.5x speed-up, and an area overhead of only 5.6% with respect to the
baseline. To demonstrate the capabilities of the architecture, we benchmark it
with end-to-end real-life QNNs, improving performance by 2x - 2.5x with respect
to existing solutions using fully flexible programmable processors.
