  Inverse problems and, in particular, inferring unknown or latent parameters
from data are ubiquitous in engineering simulations. A predominant viewpoint in
identifying unknown parameters is Bayesian inference where both prior
information about the parameters and the information from the observations via
likelihood evaluations are incorporated into the inference process. In this
paper, we adopt a similar viewpoint with a slightly different numerical
procedure from standard inference approaches to provide insight about the
localized behavior of unknown underlying parameters. We present a variational
inference approach which mainly incorporates the observation data in a
point-wise manner, i.e. we invert a limited number of observation data
leveraging the gradient information of the forward map with respect to
parameters, and find true individual samples of the latent parameters when the
forward map is noise-free and one-to-one. For statistical calculations (as the
ultimate goal in simulations), a large number of samples are generated from a
trained neural network which serves as a transport map from the prior to
posterior latent parameters. Our neural network machinery, developed as part of
the inference framework and referred to as Neural Net Kernels (NNK), is based
on hierarchical (deep) kernels which provide greater flexibility for training
compared to standard neural networks. We showcase the effectiveness of our
inference procedure in identifying bimodal and irregular distributions compared
to a number of approaches including Markov Chain Monte Carlo sampling
approaches and a Bayesian neural network approach.
