  This paper presents a light-weight, high-quality texture synthesis algorithm
that easily generalizes to other applications such as style transfer and
texture mixing. We represent texture features through the deep neural
activation vectors within the bottleneck layer of an auto-encoder and frame the
texture synthesis problem as optimal transport between the activation values of
the image being synthesized and those of an exemplar texture. To find this
optimal transport mapping, we utilize an N-dimensional probability density
function (PDF) transfer process that iterates over multiple random rotations of
the PDF basis and matches the 1D marginal distributions across each dimension.
This achieves quality and flexibility on par with expensive back-propagation
based neural texture synthesis methods, but with the potential of achieving
interactive rates. We demonstrate that first order statistics offer a more
robust representation for texture than the second order statistics that are
used today. We propose an extension of this algorithm that reduces the
dimensionality of the neural feature space. We utilize a multi-scale
coarse-to-fine synthesis pyramid to capture and preserve larger image features;
unify color and style transfer under one framework; and further augment this
system with a novel masking scheme that re-samples and re-weights the feature
distribution for user-guided texture painting and targeted style transfer.
