 using "textbook-quality" data, which is clear, self-contained, instructive, and balanced.

The training data for phi-1 consists of three main datasets: a filtered code-language dataset, a synthetic textbook dataset, and a small synthetic exercises dataset. The filtered code-language dataset is a subset of The Stack and StackOverflow, obtained using a language model-based classifier. The synthetic textbook dataset consists of Python textbooks generated by GPT-3.5, and the synthetic exercises dataset consist