  We propose two novel nonparametric two-sample kernel tests based on the
Maximum Mean Discrepancy (MMD). First, for a fixed kernel, we construct an MMD
test using either permutations or a wild bootstrap, two popular numerical
procedures to determine the test threshold. We prove that this test controls
the probability of type I error non-asymptotically. Hence, it can be used
reliably even in settings with small sample sizes as it remains
well-calibrated, which differs from previous MMD tests which only guarantee
correct test level asymptotically. When the difference in densities lies in a
Sobolev ball, we prove minimax optimality of our MMD test with a specific
kernel depending on the smoothness parameter of the Sobolev ball. In practice,
this parameter is unknown and, hence, the optimal MMD test with this particular
kernel cannot be used. To overcome this issue, we construct an aggregated test,
called MMDAgg, which is adaptive to the smoothness parameter. The test power is
maximised over the collection of kernels used, without requiring held-out data
for kernel selection (which results in a loss of test power), or arbitrary
kernel choices such as the median heuristic. We prove that MMDAgg still
controls the level non-asymptotically, and achieves the minimax rate over
Sobolev balls, up to an iterated logarithmic term. Our guarantees are not
restricted to a specific type of kernel, but hold for any product of
one-dimensional translation invariant characteristic kernels. We provide a
user-friendly parameter-free implementation of MMDAgg using an adaptive
collection of bandwidths. We demonstrate that MMDAgg significantly outperforms
alternative state-of-the-art MMD-based two-sample tests on synthetic data
satisfying the Sobolev smoothness assumption, and that, on real-world image
data, MMDAgg closely matches the power of tests leveraging the use of models
such as neural networks.
