  The availability of genomic data is essential to progress in biomedical
research, personalized medicine, etc. However, its extreme sensitivity makes it
problematic, if not outright impossible, to publish or share it. As a result,
several initiatives have been launched to experiment with synthetic genomic
data, e.g., using generative models to learn the underlying distribution of the
real data and generate artificial datasets that preserve its salient
characteristics without exposing it. This paper provides the first evaluation
of both utility and privacy protection of six state-of-the-art models for
generating synthetic genomic data. We assess the performance of the synthetic
data on several common tasks, such as allele population statistics and linkage
disequilibrium. We then measure privacy through the lens of membership
inference attacks, i.e., inferring whether a record was part of the training
data. Our experiments show that no single approach to generate synthetic
genomic data yields both high utility and strong privacy across the board.
Also, the size and nature of the training dataset matter. Moreover, while some
combinations of datasets and models produce synthetic data with distributions
close to the real data, there often are target data points that are vulnerable
to membership inference. Looking forward, our techniques can be used by
practitioners to assess the risks of deploying synthetic genomic data in the
wild and serve as a benchmark for future work.
