  Molecular biologists frequently interpret gene lists derived from
high-throughput experiments and computational analysis. This is typically done
as a statistical enrichment analysis that measures the over- or
under-representation of biological function terms associated with genes or
their properties, based on curated assertions from a knowledge base (KB) such
as the Gene Ontology (GO). Interpreting gene lists can also be framed as a
textual summarization task, enabling the use of Large Language Models (LLMs),
potentially utilizing scientific texts directly and avoiding reliance on a KB.
  We developed SPINDOCTOR (Structured Prompt Interpolation of Natural Language
Descriptions of Controlled Terms for Ontology Reporting), a method that uses
GPT models to perform gene set function summarization as a complement to
standard enrichment analysis. This method can use different sources of gene
functional information: (1) structured text derived from curated ontological KB
annotations, (2) ontology-free narrative gene summaries, or (3) direct model
retrieval.
  We demonstrate that these methods are able to generate plausible and
biologically valid summary GO term lists for gene sets. However, GPT-based
approaches are unable to deliver reliable scores or p-values and often return
terms that are not statistically significant. Crucially, these methods were
rarely able to recapitulate the most precise and informative term from standard
enrichment, likely due to an inability to generalize and reason using an
ontology. Results are highly nondeterministic, with minor variations in prompt
resulting in radically different term lists. Our results show that at this
point, LLM-based methods are unsuitable as a replacement for standard term
enrichment analysis and that manual curation of ontological assertions remains
necessary.
