  Data curation tasks that prepare data for analytics are critical for turning
data into actionable insights. However, due to the diverse requirements of
applications in different domains, generic off-the-shelf tools are typically
insufficient. As a result, data scientists often have to develop
domain-specific solutions tailored to both the dataset and the task, e.g.
writing domain-specific code or training machine learning models on a
sufficient number of annotated examples. This process is notoriously difficult
and time-consuming. We present SEED, an LLM-as-compiler approach that
automatically generates domain-specific data curation solutions via Large
Language Models (LLMs). Once the user describes a task, input data, and
expected output, the SEED compiler produces an executable pipeline composed of
LLM-generated code, small model, and data access modules. SEED uses these
generated modules to process most of the data records and dynamically decides
when the LLM should step in to directly process some individual records,
possibly using the data-access modules to retrieve relevant information from
the data sources to assist the LLM in solving the task. To validate this new,
revolutionary approach, we conducted experiments on 9 datasets spanning over 5
data curation tasks. The results show that SEED generates domain-specific
solutions that significantly outperform their generic counterparts, often
approaching the performance of the manually curated solutions that use
thousands of labeled training examples. Moreover, in comparison to solutions
that use the LLM on every data record, SEED achieves state-of-the-art or
comparable few-shot performance, while significantly reducing the number of LLM
calls.
