James Healy was born in Jones County, Georgia, on April 6, 1830. His father, Michael Morris Healy (1796–1850), was a native of County Roscommon, Ireland who became a wealthy cotton planter after settling in Georgia. Healy owned more than 1,500 acres of land near the Ocmulgee River as well as 49 to 60 enslaved people. James's mother was a mixed-race enslaved woman named Mary Eliza (c. 1813–1850), whom Michael had purchased for $3,700 along with her family. He took Mary Eliza as his common-law wife in 1829.
