Buel's ancestry includes several prominent and influential families, such as the McDougalls, Hansons, Wilmers and Hillhouses, the latter of which produced many statesmen and scholars in Connecticut. The Buel family was also prominent in Connecticut, and was one of the oldest in New England. His earliest American ancestor, William Buel, arrived in 1630 from England. Two of Buel's brothers, Samuel and Clarence, went on to become Episcopal priests.
