  Statistical hypothesis testing and effect size measurement are routine parts
of quantitative research. Advancements in computer processing power have
greatly improved the capability of statistical inference through the
availability of resampling methods. However, many of the statistical practices
used today are based on traditional, parametric methods that rely on
assumptions about the underlying population. These assumptions may not always
be valid, leading to inaccurate results and misleading interpretations.
Permutation testing, on the other hand, generates the sampling distribution
empirically by permuting the observed data, providing distribution-free
hypothesis testing. Furthermore, this approach lends itself to a powerful
method for multiple comparison correction - known as max correction - which is
less prone to type II errors than conventional correction methods. Parametric
methods have also traditionally been utilized for estimating the confidence
interval of various test statistics and effect size measures. However, these
too can be estimated empirically using permutation or bootstrapping techniques.
Whilst resampling methods are generally considered preferable, many popular
programming languages and statistical software packages lack efficient
implementations. Here, we introduce PERMUTOOLS, a MATLAB package for
multivariate permutation testing and effect size measurement.
