  Low-precision formats have recently driven major breakthroughs in neural
network (NN) training and inference by reducing the memory footprint of the NN
models and improving the energy efficiency of the underlying hardware
architectures. Narrow integer data types have been vastly investigated for NN
inference and have successfully been pushed to the extreme of ternary and
binary representations. In contrast, most training-oriented platforms use at
least 16-bit floating-point (FP) formats. Lower-precision data types such as
8-bit FP formats and mixed-precision techniques have only recently been
explored in hardware implementations. We present MiniFloat-NN, a RISC-V
instruction set architecture extension for low-precision NN training, providing
support for two 8-bit and two 16-bit FP formats and expanding operations. The
extension includes sum-of-dot-product instructions that accumulate the result
in a larger format and three-term additions in two variations: expanding and
non-expanding. We implement an ExSdotp unit to efficiently support in hardware
both instruction types. The fused nature of the ExSdotp module prevents
precision losses generated by the non-associativity of two consecutive FP
additions while saving around 30% of the area and critical path compared to a
cascade of two expanding fused multiply-add units. We replicate the ExSdotp
module in a SIMD wrapper and integrate it into an open-source floating-point
unit, which, coupled to an open-source RISC-V core, lays the foundation for
future scalable architectures targeting low-precision and mixed-precision NN
training. A cluster containing eight extended cores sharing a scratchpad
memory, implemented in 12 nm FinFET technology, achieves up to 575 GFLOPS/W
when computing FP8-to-FP16 GEMMs at 0.8 V, 1.26 GHz.
