  Sparse tensor algebra computations have become important in many real-world
applications like machine learning, scientific simulations, and data mining.
Hence, automated code generation and performance optimizations for tensor
algebra kernels are paramount. Recent advancements such as the Tensor Algebra
Compiler (TACO) greatly generalize and automate the code generation for tensor
algebra expressions. However, the code generated by TACO for many important
tensor computations remains suboptimal due to the absence of a scheduling
directive to support transformations such as distribution/fusion.
  This paper extends TACO's scheduling space to support kernel
distribution/loop fusion in order to reduce asymptotic time complexity and
improve locality of complex tensor algebra computations. We develop an
intermediate representation (IR) for tensor operations called branched
iteration graph which specifies breakdown of the computation into smaller ones
(kernel distribution) and then fuse (loop fusion) outermost dimensions of the
loop nests, while the innermost dimensions are distributed, to increase data
locality. We describe exchanges of intermediate results between space iteration
spaces, transformation in the IR, and its programmatic invocation. Finally, we
show that the transformation can be used to optimize sparse tensor kernels. Our
results show that this new transformation significantly improves the
performance of several real-world tensor algebra computations compared to
TACO-generated code.
