He was a Justice of the Peace for Nottinghamshire and in 1904 was made KCVO. He married in 1894 but his first wife died in 1912 without issue. He married for the second time in 1916; the marriage produced George Halifax Lumley-Savile, heir to the title. Henry Lumley-Savile (1923–2001) was the younger son from the marriage.
