  We report the design of diffractive surfaces to all-optically perform
arbitrary complex-valued linear transformations between an input (N_i) and
output (N_o), where N_i and N_o represent the number of pixels at the input and
output fields-of-view (FOVs), respectively. First, we consider a single
diffractive surface and use a matrix pseudoinverse-based method to determine
the complex-valued transmission coefficients of the diffractive
features/neurons to all-optically perform a desired/target linear
transformation. In addition to this data-free design approach, we also consider
a deep learning-based design method to optimize the transmission coefficients
of diffractive surfaces by using examples of input/output fields corresponding
to the target transformation. We compared the all-optical transformation errors
and diffraction efficiencies achieved using data-free designs as well as
data-driven (deep learning-based) diffractive designs to all-optically perform
(i) arbitrarily-chosen complex-valued transformations including unitary,
nonunitary and noninvertible transforms, (ii) 2D discrete Fourier
transformation, (iii) arbitrary 2D permutation operations, and (iv) high-pass
filtered coherent imaging. Our analyses reveal that if the total number (N) of
spatially-engineered diffractive features/neurons is N_i x N_o or larger, both
design methods succeed in all-optical implementation of the target
transformation, achieving negligible error. However, compared to data-free
designs, deep learning-based diffractive designs are found to achieve
significantly larger diffraction efficiencies for a given N and their
all-optical transformations are more accurate for N < N_i x N_o. These
conclusions are generally applicable to various optical processors that employ
spatially-engineered diffractive surfaces.
