  Privacy concerns around sharing personally identifiable information are a
major practical barrier to data sharing in medical research. However, in many
cases, researchers have no interest in a particular individual's information
but rather aim to derive insights at the level of cohorts. Here, we utilize
Generative Adversarial Networks (GANs) to create derived medical imaging
datasets consisting entirely of synthetic patient data. The synthetic images
ideally have, in aggregate, similar statistical properties to those of a source
dataset but do not contain sensitive personal information. We assess the
quality of synthetic data generated by two GAN models for chest radiographs
with 14 different radiology findings and brain computed tomography (CT) scans
with six types of intracranial hemorrhages. We measure the synthetic image
quality by the performance difference of predictive models trained on either
the synthetic or the real dataset. We find that synthetic data performance
disproportionately benefits from a reduced number of unique label combinations.
Our open-source benchmark also indicates that at low number of samples per
class, label overfitting effects start to dominate GAN training. We
additionally conducted a reader study in which trained radiologists do not
perform better than random on discriminating between synthetic and real medical
images for intermediate levels of resolutions. In accordance with our benchmark
results, the classification accuracy of radiologists increases at higher
spatial resolution levels. Our study offers valuable guidelines and outlines
practical conditions under which insights derived from synthetic medical images
are similar to those that would have been derived from real imaging data. Our
results indicate that synthetic data sharing may be an attractive and
privacy-preserving alternative to sharing real patient-level data in the right
settings.
