His parents, John and Martha (Finley) Montgomery, had immigrated from Ireland. One known sibling is "Nancy" Agnes Montgomery (Mrs. Alexander McCorkle) who is buried Thyatira Presbyterian Church Cemetery, Rowan County, N.C., and whose son, a nephew of Joseph Montgomery, was also a noted Presbyterian divine: Dr Samuel Eusebius McCorkle (Princeton baccalaureate, Dickinson College Doctorate of Divinity). Montgomery was on his mother's side related to 1761-1766 Princeton President, Samuel Finley (1715–66).
