  Nonlocal quantum theory of one-component scalar field in $D$-dimensional
Euclidean spacetime is studied in representations of $\mathcal{S}$-matrix
theory for both polynomial and nonpolynomial interaction Lagrangians. The
theory is formulated on coupling constant $g$ in the form of an infrared smooth
function of argument $x$ for space without boundary. Nonlocality is given by
evolution of Gaussian propagator for the local free theory with ultraviolet
form factors depending on ultraviolet length parameter $l$. By representation
of the $\mathcal{S}$-matrix in terms of abstract functional integral over
primary scalar field, the $\mathcal{S}$ form of a grand canonical partition
function is found. And, by expression of $\mathcal{S}$-matrix in terms of the
partition function, the representation for $\mathcal{S}$ in terms of basis
functions is obtained. Derivations are given for discrete case where basis
functions are Hermite functions, and for continuous case where basis functions
are trigonometric functions. The obtained expressions for the
$\mathcal{S}$-matrix are investigated within the framework of variational
principle based on Jensen inequality. Equations with separable kernels
satisfied by variational function $q$ are found and solved, yielding results
for both the polynomial theory $\varphi^{4}$ and the nonpolynomial sine-Gordon
theory. A new definition of the $\mathcal{S}$-matrix is proposed to solve
additional divergences which arise in application of Jensen inequality for the
continuous case. Analytical results are illustrated numerically. For simplicity
of numerical calculation: the $D=1$ case is considered, and propagator for the
free theory $G$ is in the form of Gaussian function typically in the
Virton-Quark model. The formulation for nonlocal QFT in momentum $k$ space of
extra dimensions with subsequent compactification into physical spacetime is
discussed.
