  Sparse linear algebra is crucial in many application domains, but challenging
to handle efficiently in both software and hardware, with one- and two-sided
operand sparsity handled with distinct approaches. In this work, we enhance an
existing memory-streaming RISC-V ISA extension to accelerate both one- and
two-sided operand sparsity on widespread sparse tensor formats like compressed
sparse row (CSR) and compressed sparse fiber (CSF) by accelerating the
underlying operations of streaming indirection, intersection, and union. Our
extensions enable single-core speedups over an optimized RISC-V baseline of up
to 7.0x, 7.7x, and 9.8x on sparse-dense multiply, sparse-sparse multiply, and
sparse-sparse addition, respectively, and peak FPU utilizations of up to 80% on
sparse-dense problems. On an eight-core cluster, sparse-dense and sparse-sparse
matrix-vector multiply using real-world matrices are up to 4.9x and 5.9x faster
and up to 2.9x and 3.0x more energy efficient. We explore further applications
for our extensions, such as stencil codes and graph pattern matching. Compared
to recent CPU, GPU, and accelerator approaches, our extensions enable higher
flexibility on data representation, degree of sparsity, and dataflow at a
minimal hardware footprint, adding only 1.8% in area to a compute cluster. A
cluster with our extensions running CSR matrix-vector multiplication achieves
9.9x and 1.7x higher peak floating-point utilizations than recent highly
optimized sparse data structures and libraries for CPU and GPU, respectively,
even when accounting for off-chip main memory (HBM) and on-chip interconnect
latency and bandwidth effects.
