  Radiology reports are an instrumental part of modern medicine, informing key
clinical decisions such as diagnosis and treatment. The worldwide shortage of
radiologists, however, restricts access to expert care and imposes heavy
workloads, contributing to avoidable errors and delays in report delivery.
While recent progress in automated report generation with vision-language
models offer clear potential in ameliorating the situation, the path to
real-world adoption has been stymied by the challenge of evaluating the
clinical quality of AI-generated reports. In this study, we build a
state-of-the-art report generation system for chest radiographs,
$\textit{Flamingo-CXR}$, by fine-tuning a well-known vision-language foundation
model on radiology data. To evaluate the quality of the AI-generated reports, a
group of 16 certified radiologists provide detailed evaluations of AI-generated
and human written reports for chest X-rays from an intensive care setting in
the United States and an inpatient setting in India. At least one radiologist
(out of two per case) preferred the AI report to the ground truth report in
over 60$\%$ of cases for both datasets. Amongst the subset of AI-generated
reports that contain errors, the most frequently cited reasons were related to
the location and finding, whereas for human written reports, most mistakes were
related to severity and finding. This disparity suggested potential
complementarity between our AI system and human experts, prompting us to
develop an assistive scenario in which Flamingo-CXR generates a first-draft
report, which is subsequently revised by a clinician. This is the first
demonstration of clinician-AI collaboration for report writing, and the
resultant reports are assessed to be equivalent or preferred by at least one
radiologist to reports written by experts alone in 80$\%$ of in-patient cases
and 60$\%$ of intensive care cases.
