  Sparse tensor networks are commonly used to represent contractions over
sparse tensors. Tensor contractions are higher-order analogs of matrix
multiplication. Tensor networks arise commonly in many domains of scientific
computing and data science. After a transformation into a tree of binary
contractions, the network is implemented as a sequence of individual
contractions. Several critical aspects must be considered in the generation of
efficient code for a contraction tree, including sparse tensor layout mode
order, loop fusion to reduce intermediate tensors, and the interdependence of
loop order, mode order, and contraction order. We propose CoNST, a novel
approach that considers these factors in an integrated manner using a single
formulation. Our approach creates a constraint system that encodes these
decisions and their interdependence, while aiming to produce reduced-order
intermediate tensors via fusion. The constraint system is solved by the Z3 SMT
solver and the result is used to create the desired fused loop structure and
tensor mode layouts for the entire contraction tree. This structure is lowered
to the IR of the TACO compiler, which is then used to generate executable code.
Our experimental evaluation demonstrates very significant (sometimes orders of
magnitude) performance improvements over current state-of-the-art sparse tensor
compiler/library alternatives.
