  1. Joint Species Distribution models (JSDMs) explain spatial variation in
community composition by contributions of the environment, biotic associations,
and possibly spatially structured residual covariance. They show great promise
as a general analytical framework for community ecology and macroecology, but
current JSDMs, even when approximated by latent variables, scale poorly on
large datasets, limiting their usefulness for currently emerging big (e.g.,
metabarcoding and metagenomics) community datasets. 2. Here, we present a
novel, more scalable JSDM (sjSDM) that circumvents the need to use latent
variables by using a Monte-Carlo integration of the joint JSDM likelihood and
allows flexible elastic net regularization on all model components. We
implemented sjSDM in PyTorch, a modern machine learning framework that can make
use of CPU and GPU calculations. Using simulated communities with known
species-species associations and different number of species and sites, we
compare sjSDM with state-of-the-art JSDM implementations to determine
computational runtimes and accuracy of the inferred species-species and
species-environmental associations. 3. We find that sjSDM is orders of
magnitude faster than existing JSDM algorithms (even when run on the CPU) and
can be scaled to very large datasets. Despite the dramatically improved speed,
sjSDM produces more accurate estimates of species association structures than
alternative JSDM implementations. We demonstrate the applicability of sjSDM to
big community data using eDNA case study with thousands of fungi operational
taxonomic units (OTU). 4. Our sjSDM approach makes the analysis of JSDMs to
large community datasets with hundreds or thousands of species possible,
substantially extending the applicability of JSDMs in ecology. We provide our
method in an R package to facilitate its applicability for practical data
analysis.
