Crutchfield was born in Greeneville, Tennessee, the son of Thomas Crutchfield Sr., a brick contractor, and Sarah (Cleage) Crutchfield. He attended common schools. He moved to McMinn County, Tennessee, in 1840, and remained there for four years before settling in Jacksonville, Alabama, in 1844. He established a large farm at Jacksonville that specialized in grain production and utilized innovative farming techniques. He was also elected a captain in the local militia. Inspired by Kentucky politician Henry Clay, he supported local Whig Party candidates.
